The crowds are generally smaller, but some attractions close up shop this time of year. In the fall, hotel rates and airfares drop with the temperatures, especially after Labor Day. Additionally, if you don't like crowds and throngs of families, you'll want to plan your trip for another time. Early spring temperatures can be a bit brisk, so don't forget your windbreaker.
This can be a great time of year to visit if you want to avoid the crowds, just book before the Memorial Day mayhem begins in May.