In regards to the asked question. I can see how sometimes it could get annoying seeing fans correct people all the time. On the other hand, why should fans let people be wrong? When you're in school and get a question wrong on a test, the teacher doesn't just "let it go."
But that largely depends on how people go about correcting them, as well as the kind of person claiming it as a cartoon/comic. If a person mistakes an anime for a cartoon or vice-versa without knowing better and the fans start bombarding him with hate for that, then the fans are the bad guys. However, if the person who mistakes an anime for a cartoon starts mocking the fans for watching something "kiddy," then it's understandable that the fans would retaliate for their show (although badmouthing is a bad way to go about it.)
There's a big difference between:
Person: What's this cartoon called?
Fan: It's an ANIME, dumbass. Learn the difference.
Person: Jeez sorry.
and
Person: haha omg this is so kiddy! why would anyone watch these sh*tty cartoons?
Fan: It's an ANIME, dumbass. Learn the difference.
Person: rlly now? Cuz all I see is sh*t
*initiate flame war*
Of course, the second example is obviously a troll, but I digress.
Is it annoying when people mistake anime/manga for cartoons/comics? Yes, if they're spouting it as a fact. No, if they're obviously new to this and don't know any better.
Is it wrong to correct them? Of course not. Just don't act like a jerk when doing it. If both sides are civil about it, then there shouldn't be any problems (yes yes, I know the day when that happens on the Internet is the day when hell freezes over, but I'm just saying, if people want to correct each other, then that's the best possible way to go about it.)
But if all else fails and the person just won't get it (like example #2), THEN the fans can ignore him and move on. It's better to stop arguing than continuing with a pointless flame war, I'd say.