If you talk to the Chocobo that's standing by the fence and choose the second reply, they'll do a little dance for you, after which you'll get your first Summon Materia, Choco/Mog! Also make sure you talk to the Ranch's owners, Choco Bob and Choco Billy, who will give you some information about catching and riding chocobos. You can also buy a Chocobo Lure from Choco Billy for 2000 gil. Unfortunatly you need it, so you have no choice. Just cough up the cash and leave.
To catch a chocobo, have someone equip the chocobo lure, then run around on the grass over the chocobo tracks. You'll eventually fight a battle with some monsters who are with a chocobo. To catch the chocobo, you must kill these monsters without hurting the chocobo. Also, if you take too long, the chcobo will simply run away. A good strategy is to use greens to delay the chocobo. Use a greens item on the bird, which will stop and eat them. The more expensive the greens were, the longer the chocobo will take to eat them. Meanwhile, kill the monsters.
Ok, once you've got a Chocobo, you'll be able to cross the swamp before the Midgar Zolem (that big nasty snake that chases you) gets you. If you're curious and want to try fighting it, go ahead, but it won't do you much good this early in the game. He's got a spell called Beta (a learnable enemy skill, btw) that will roast you for 1000-2000 HP, which you really can't afford to lose right now. You will later, though, so leave the Midgar Zolem alone for now, but don't forget about him. Once you get across the swamp, ditch the chocobo and enter the Mythril Mine.