The most important difference between audio file formats is the type of compression (that's data compression - not to be confused with audio level compression, which you can review in our article Should You Use Compression In Audio Recording? if you'd like or see my article and video - Vocal Compression Using Reaper’s ReaComp Effect Plugin.
But we are not talking about THAT kind of compression here. This is about making very large files much much smaller, while still having it sound good.
This is similar to what you do to pictures you want to put on a website. You reduce the size of the image first. That gives you a smaller file size. Then you reduce the "quality" of image so that it still looks good, but has an even smaller file size.
Basically THAT is compression. So this is the same thing, just for audio.
There are uncompressed formats (.wav, .aiff), lossless compressed formats (.flac), and lossy compressed formats (.mp3, .wma). As you might guess, uncompressed formats are in their original size and quality.
These formats are best for recording, but due to their larger size, may not be appropriate for distribution. That's why you usually see mp3 files on the internet, or being sent through email. It's because of their smaller size.
Lossless formats compress the size of the file without losing any of the quality, and lossy formats compress size with some loss of quality.
Knowing the key differences between the formats can help you to choose the right file type for each step in your production process.
Get a review on the mp3 format from my article What Is An MP3?
You can read the article describing the other formats here: http://www.audio-issues.com/home-recording-studio/demystifying-audio-formats-what-format-should-you-record-in/