Your question is very timely. An article addressing this important question was just published online in the Archives of Internal Medicine on August 27, 2012.
The researchers looked at fitness levels of more than 18,600 men and women. When the study began, the median age was 49. The researchers kept track of people for about 26 years.
Once people reached age 65, researchers used Medicare claims information to follow what happened with their health. People who were most fit at midlife not only lived longer. They spent less time being sick.
The fittest group had slightly over half the rate of chronic diseases as people who were least fit. Diseases included coronary artery disease, Alzheimer's, heart failure, diabetes and others. About 2,400 people died during the study. In their last 5 years of life, the people who had been most fit at midlife spent about 50% less time with more chronic diseases than the least fit group.
The amount of time you exercise and the intensity of your exercise routines obviously play a major role in your fitness. But other factors, such as genetics, also make a difference. So there cannot be a precise connection between time spent exercising and level of fitness.
Personally, my goal is one hour of moderate to high intensity exercise 6 days per week to maintain fitness. If I had more time for exercise, I might increase my fitness level a bit more. But I suspect not that much for the amount of extra time I would need to spend exercising.
And its important to not let yourself get overly fatigued from too much exercise. You will actually lose some of the gains in fitness.