Over here, virtually everyone watches at least a bit anime, and even more people read manga, no matter what age they are. When I catch the train to uni in the mornings, there will invariably be at least one, often several, salarymen engrossed in some manga or other, or somehow managing to read the latest issue of Shounen Jump despite how crowded the trains get.
Admittedly Japan is a bit different to the west, but even so - you are most definitely not too old for it, even if there were such a thing as being too old in the first place. It's just a cultural difference in terms of what formats have become popular, really, rather than anything to do with age.