Nazi ideology is violence in itself. Antifascist violence is self-defense. If you don't understand that, get out of the ****ing way so you don't get hurt.
For those who want a more "intellectual" look at things, we call upon Karl Popper -- a man who, by all means, wanted to be even-handed and who advocated for being a tolerant person. He broached an argument in The Open Society and Its Enemies, volume one, that stated that, if we choose to tolerate everything, eventually we will be forced to tolerate intolerant ideals, to a point that includes those ideas that demand the destruction of others or even tolerance itself; as such, as he put it, "[w]e should therefore claim, in the name of tolerance, the right not to tolerate the intolerant." The "so much for the tolerant left" people are idiots. The destruction of Nazism or white nationalism or whatever you want to call it this week -- of intolerance, simply speaking -- is the path of greatest tolerance, despite appearances.
edit: Also, I forgot to post this the first time, but for reference, I don't like the violence-first order either. Plenty of people genuinely do not know what they are following along with or the consequences of their beliefs and some of them can be talked to. Of course, people who organize marches? Not so much. People who have jumped around between several philosophies that share similar threads, for years? Also probably not so much.