In contrast, if an action only indirectly affects society without violating any fixed obligation, then "the inconvenience is one which society can afford to bear, for the sake of the greater good of human freedom." Society has a person's entire childhood to nurture values; if the person fails to accept those values, or remains immature, it is society's own fault. No further influence is necessary. Also, if an action is harmful then people will see its negative effects, and this should be enough of an example to them of why they should not act in such a way.
Mill says the strongest argument against interference, though, is that when society does interfere, it will likely do so wrongly. He writes, "there is no parity between the feeling of a person for his own opinion, and the feeling of another who is offended at his holding it." Mill argues that there is a universal tendency of people to extend the bounds of "moral police" unjustly. He writes about how a Muslim majority might insist that pork not be eaten in their country, or that married clergy be punished in Spain. He writes, "we must beware of admitting a principle of which we should resent as a gross injustice the application to ourselves." If people want to be able to impose their morality, they must be willing to accept the imposition by others. Mill complains about unjust violations of freedom such as the banning of alcohol, the banning of recreation on the Sabbath, and the persecution of Mormons for polygamy. People can preach against such activities, and try to change people's minds, but they should not be coercive.
Mill spends significant time in this chapter defending and delineating his "harm principle": that actions can only be punished when they harm others. Perhaps the most basic issue in this chapter, then, is whether Mill's harm principle actually makes sense. Mill acknowledges that people are not fully isolated from society, and that their actions can affect others. In principle then, one could make a case that any particular activity causes such harm to other people that the need to respect individuality is outweighed. Is it unfairly arbitrary that Mill therefore limits social intervention to those actions that directly violate obligations? Perhaps more importantly, does Mill leave too much room for someone to say that it would be acceptable to limit liberty any time it could harm society in any way?
In response to these questions, Mill would likely remind the reader that his approach is operating under a broadly construed conception of social good. In Chapter 3, he tried to show many of the beneficial effects of nonconformity. Any social interest in restricting actions would therefore have to overcome the broad social value of individuality. While Mill's utilitarian approach does leave open the possibility that social interest could require major limitations on freedom, his discussions in previous chapters about the social value of liberty makes such a possibility unlikely. The reason his standard for "harm" is so high is that the good that comes from individuality is so socially beneficial.
In many ways, Mill uses the same arguing technique in this chapter that he did in defending freedom of opinion in Chapter 2. Mill points out that societies frequently declare perfectly legitimate activities to be immoral. Therefore, if a person wants to say that it is acceptable to punish bad activities, he must also accept that others have the right to do the same to him. Mill starts with examples that would seem obvious to his audience, like the unfairness of banning pork in Muslim countries, to make much more radical claims, such as the unfairness of banning polygamy. Thus, the fallibility of society is an important aspect of Mill's defense of liberty of action.
Mill's discussion is also interesting in the ways in which he leaves some openings for social criticism of actions. Such criticism is appropriate when it cannot be helped; it is simply natural that people will find some activities to be distasteful and will therefore judge the action inappropriate. However, Mill sets boundaries on any punitive action emerging from this criticism. Just as Mill believes opinions must be free while actions are subject to at least some regulation, he gives free rein to criticism while limiting punishment, an action.