Jason Van Orden is an American guy who talks about literally everything you need to know about podcasting and how to get started. He's great - very clear and helpful.
Click here for his 'how to podcast' tutorial.
To record sound: I recommend Olympus recorders as a cheap and cheerful way to get started. They record in 'wma' format but you can easily convert these files into an mp3 using a free converter like jodix.
To buy an Olympus recorder online,
click here
Buy a good microphone though as the Olympus inbuilt mic isn't great (it's more of a dictaphone). To buy a decent sony mic
click here
I recommend searching online for the cheapest places to buy the kit first - for example you can get some great bargains on
ebay.
To edit sound: Use audacity editing software to get you started. It's free and easy to use. Download it here: http://audacity.sourceforge.net/
Jason Van Orden does a really good online tutorial -
click here. If you can use microsoft word, you can use audacity - just think 'copy, cut, paste' etc.
To upload a podcast: Use www.switchpod.com - it's as easy as setting up an email address and it's free! It will host your podcast and create a 'feed' which people can subscribe to. This means whenever you update your podcast, your subscriber will automatically get your latest episode.