Well, I think I never liked math because it's abstract, but in this case I mean abstract as in it means nothig. It explains nothing. Math is usefull for the math's sake and as support for other paths. (I know, I know, but bear with it.) When physics actually explains everything around us and beyond our planet. I also liked lasers. That's a good reason eh. Like "Why did you bacame writter?" - "I like to sniff ink."
See, the problem I have with physics is that, once you get past simple mechanics, it gets very abstract. I remember trying to learn how to solve some problem involving gravity and rotating frames of reference in a radian coordinate system... blah, this was in 1995, so that's as far as I remember, but I couldn't connect it in my head to anything in everyday existence, so I never solved it. (That was the beginning of my turning point, when I realized I needed to abandon my path.) What in the hell, for instance, is an 'up' quark or a 'down' quark; stuff like that has no relevance in the world at the macro level.
I look at math as being akin to learning a language -- it's like learning words and phrases and grammar, but it doesn't have much of a connection with anything until you apply it to physics or to some other real-world application. The math is essentially the language that allows physics to actually carry a precise meaning; you can't really separate the two, just like you can't write a novel without having a language to precisely communicate the story. Not a perfect analogy, I know, but it's how I see the connection between the two. I bet Nikkoru with his near-perfect eloquence could explain it better.