When you walk in a prairie or forest each October there won't be animals putting detritus into compost piles or plastic bags to be hauled away, and I challenge you to get a picture of that occurance. Removing spent material from the garden is removing fertilizer, rich topsoil, habitat, and food -- or simply put, it's gardening against nature.
Do you see how trees drop leaves around their root zone? Yes, unless it's super windy. But trees are smart -- they are fertilizing themselves. This is one reason many organic lawn care firms advocate mulching leaves over a lawn (and mulching the lawn during the growing season), because dead leaves are rich in nutrients. The same goes for perennial flowers and grasses in a garden -- leave most of what you cut down in the garden, breaking it up a little by hand if you want to. You can do this in fall but, as we'll see below, it's better to wait until spring.
Topsoil
Leaves and other plant bits are broken down naturally over time by soil life, incorporated into the soil by bugs and micro organisms and bacteria and worms and ants and moles. This adds fertility to the soil and increases water penetration and storage -- it's the cycle of life. When you remove leaves and stems from the garden you are literally removing healthy soil.
Leaving flower heads full of seed can be critical for winter songbirds whose resources dwindle as the season goes along. Seeds tend to be high in fat content and lots of other good nutrients critical to a bird's ability to produce both energy and heat in the colder months. Of course, once spring comes, flower stems and grass clumps used for food and shelter from snowstorms become prime nesting material.
Habitat
Plenty of creatures overwinter in garden litter, from queen bumblebees to mourning cloak butterflies to black swallowtail larvae to all manor of frogs, spiders, beetles, and bugs. Inside plant stems may be larvae of native bees, since roughly 25% of native bee species use cavities found in wood or stone as nesting sites. This is one reason why, in early to mid spring, you should leave 12-18 inches of stem when cutting back the garden -- you will soon see swarms of bees coming to lay eggs in hollow stems or to excavate pith before egg laying begins. After a few weeks the "ugly" dead stems will soon be covered by new green growth as insects keep doing their thing.