In other of his writings (including Considerations on Representative Government), Mill writes in favor of imperialism and despotic rule over “inferior” peoples. This stance would appear at odds given his commit to individual liberty, however, it is important to realize that Mill does not believe freedom to be an inherent right belonging to all men simply because they are human. Mill specifically rejects trying to justify liberty claims in this manner (by things like natural law or divine will). Rather, Mill wants to show that liberty is beneficial to the individual and to society; his book is an attempt to show the utility of individuality. As a result, he sets limits on how far liberty should extend. It would seem natural that Mill’s support of liberty extends to support self-government, and in general it does.
However, he believes that children and “barbarians” lack the necessary tools to enjoy liberty. For these people, it is the state’s job to try to provide them with the civilized ability to enjoy freedom. For children, this results in measures like mandating public education. For barbarians, Mill leaves open the possibility of imperial rule, by which people are ruled with the hope that they can one day rule themselves. Thus, Mill accepts imperialism because he has a hierarchical conception of societies, where only some are advances enough to benefit from the protection of individuality.
Mill sees barbarians as inferior peoples, in some sense childlike. As a result, the most beneficial way of treating them is as children. Mill thus would accept a kind of benevolent imperialism whose goal was to civilize people to a state where they could benefit from self-government. For those people who were capable of self-government, however, liberty protections would still hold.