Is it a desirable thing to be an anime fanatic? There are a lot of strange anime fans. I think the average person will consider you to have an obsession if your room has posters of anime, you have figurines, or you have a body pillow. Especially if any of those things have a little girl on them, or an 'anime babe' etc.
I think one should take pride in being discerning and picky. You might be a fanatic if you watch anything just because it's anime, or if you have strong and vocalized opinions about anime / anime culture. It's not something that will get you much respect. But if you treat anime the way you treat other mediums, including Hollywood films, games, music, books and western cartoons, then it is okay even if you've watched tonnes and tonnes. As long as you can talk to someone who doesn't have much interest in anime and argue the merits of any particular show you like that happens to be anime, you do not have to be labeled a fanatic.
You and I are likely thinking of it differently though, which is why I began my post this way. Some people want to wear the label otaku proudly (which to the Japanese is just a very negative word meaning obsessed or addicted). I think it's the same with fanatic. You might love anime so much that being outcast is just the proof you want to show how much you like it. For me though, there's a lot about anime I don't like (the same as there is a lot about all mediums that I don't like), so I want to be critical of it and I want the things I do like not to be overlooked by people who are afraid of the anime fanatics/otakus and their anime body pillows, or by extension, to overlook the Japanese culture in general because of otaku pedophiles and giant robots.
tl;dr: for me it's not really how much you watch, it's your personality and what you watch and how you interpret it that determines whether you are a weird fanatic or just a person who explores anime as another legitimate medium with its own strengths and weaknesses.