Back when humans were still working on the basics (language, fire, agriculture, etc.) we only had a couple things consistently going for us: hunting and foraging. Foraged nuts, berries, mushrooms, and other plants were an important source of food -- herbs and roots became the original basis for medicine as we know it, and still play important roles in understanding and treating diseases using naturally-occurring chemical compounds.
With the renewal of interest in local, seasonal foods, foraging has started to make a comeback. People are reconnecting with the way their grandparents and great grandparents used to supplement their suppers, gathering wild raspberries, Chicken of the Woods mushrooms, asparagus, morels and other fruits of the forest. Amateur mushroom foragers often start by looking for the "Foolproof Four" -- four easy-to-recognize mushrooms (including the sulfur shelf / Chicken of the Woods, giant puffballs, morels, and chanterelles) with no dangerous look-alikes.
Some basic guidelines for those who forage: know the law, and know your target. From a legal perspective, it's usually best to forage on private land with permission, as opposed to park land -- if you do forage in a park, understand what's acceptable in terms of where you can walk and what you can take. As for the plants themselves, it's not enough to think you know what you're eating -- you should also know what else looks like it, and whether those similar plants are safe, toxic, or deadly. It's always best to travel with an experienced forager before harvesting anything outside of your comfort zone -- books are a start, but until you've smelled and touched a plant in the wild, you don't really know it. It's also important to keep in mind that plants change in appearance (sometimes radically) throughout a growing season and from plant to plant.
Some books to get started with:
Keep in mind that wherever you live, there are probably regional (or even state) foraging guides that will be more accurate and useful than general guides, and can prepare you for starting to understand the specific environment in which you live. And if you're going for mushrooms, there are often local Mycological societies who can lead you on forages and make sure your foraged supper is a safe and delicious one.