In this chapter, Mill attempts to delineate when the authority of society can rightly limit individuality and the "sovereignty of the individual over himself." Mill's answer is that society and the individual should each receive control over that part of human life that it is particularly interested in.
While rejecting the idea of a social contract, Mill writes that since people receive the protection of society, they owe certain conduct in return. Individuals must not injure those interests of other people that should be considered rights. Individuals must fairly share the burden of defending society and its members from injury. Finally, individuals may be censured by opinion, though not by law, for harming others while not violating their rights. Thus, society has jurisdiction over any aspect of human behavior that "affects prejudicially the interests of others."
However, society does not have an interest in those aspects of life that affect no one but the person acting, or only affects people by their consent. Mill writes that such behavior should be both legally permitted and socially accepted. People should encourage others to make full use of their faculties. They should not, however, try to keep a person from doing with his life what he wishes. Mill justifies this position by observing that anybody else's interests in or knowledge about a particular person's well being is "trifling" compared to the individual's own interest and knowledge.
Mill says that he does not mean that people should not be allowed to point out what they see as faults in other people's behavior. In addition, he is not proscribing avoiding a person or warning others about that person. These "penalties" are acceptable because they are natural reactions to some behavior--they are not intended to punish a person. However, People do not have the right to express moral reprobation, and they should not try to make the person uncomfortable. He should not be treated with anger or resentment, or seen as an enemy if he engages in unpopular activities that only affect himself.
Mill then addresses potential criticism of his argument. How "can any part of the conduct of a member of society be a matter of indifference to the other members?" No human is fully isolated, and actions can create bad examples, hurt those who depend on the person and diminish community resources. Furthermore, why can't society interfere on behalf of mature people incapable of "self- government?"
Mill replies that he agrees that some behavior may affect the "sympathies" and interests of others, and hurt the well-being of society at large. When an action violates a person's obligations then it does not only affect himself, and he can be properly face moral reprobation for breaking those obligations. Mill forwards the example of a person who is unable to pay debts because of extravagant living. He says that such behavior is subject to punishment because the person fails to fulfill a duty to his creditors. However, the person should not be punished for the extravagance itself--that is a personal decision that must be respected.