"No, I'm an equal opportunity thug. I just like maiming people."
"Oh, well, that's better than doing it because of the group they represent. We won't punish you as harshly as we would someone who only did that to transvestites."
The motive in a crime does matter; it's one of the elements that predicts how dangerous the criminal is likely to be in the future. And maybe hate criminals really are worse than other kinds of criminals, that they poison society more than the actions of other kinds of sadists and thugs do. There's an argument to be made for that. People change their behavior somewhat out of fear of random crime -- not jogging alone at night, or avoiding dangerous neighbors. But people who fear persecution change their behavior a lot more, because they can predict that holding hands or kissing their SO in public will increase their chances of being physically harmed. That's much worse.
So maybe the important thing in singling out hate crimes for legislation is not that this particular criminal act is worse, per se, but that it has such a poisonous effect on society that it's more important that it be stamped out than that random crime be stopped.
I do know there's another category of crime that does bother me more than other sorts, but that no one ever suggests legislating. Call it a "violation of trust" crime. The kind where criminals wait by the side of a highway, pretending their car broke down. Then when someone stops to help, they mug that person. Or people who knock on doors and ask to use the phone so they can get inside and rob the house.
Those specific ones are a lot rarer in the age of cellphones, because it's rarer that someone in trouble really wouldn't be able to call for help. But I'm sure there are other kinds that have taken their place. I hate these kinds more than burglars who rob your house while you're at work, or people who steal cars out of parking lots or whatever. Because when your house gets broken into because your lock isn't very good, you might think "maybe I should get a bolt for the door" or "maybe I should get a security system".
But when someone victimizes you because you were nice enough to offer help to a stranger, you think "maybe I shouldn't help strangers."
And I really hate the lesson that teaches.