There are lots of things I think are bad— which I consider wrong— that as far as I can tell don't necessarily violate anyone's natural human rights. I would not personally use force to stop someone from doing these things, nor would I advocate legislation or support sending cops after anyone for doing them. Even if I consider them wrong.
Sometimes people don't understand the distinction. I'll say I believe something is wrong, maybe even evil, but they then assume I'm going to support legislation to punish those who engage in it. Nope. I don't advocate or support legislation, and I'm probably not going to personally use force against those doing it. Especially if I'm too far removed from the situation to know extra factors.
It might not be smart to test the limits on such things with people I care about, though. Because there is a line that could be crossed— even I'm not sure where it is in every case. If it impacts me or someone who matters to me-- or even some stranger in my sphere-- it might be different as far as me, personally, using force to defend them. In the heat of the moment, I could do things I consider wrong if it feels necessary. What is "necessary"?