Fitness is normally defined as the number of offspring an individual will produce. Natural selection works to maximize fitness, because traits can only be selected for if they are passed down to progeny. Natural selection therefore seems to favor selfish actions that promote one's own fitness over the fitness of other members of one's species. This seeming truth runs into difficulties, however, when faced with actions of social altruism that are quite prevalent in animal society. For example, in eusocial colonies, some individuals forgo reproduction altogether. How can such an action be explained in light of what we know about natural selection and evolution?
The answer comes when we consider an individual's inclusive fitness, which is the sum of an individual's direct fitness, the number of offspring produced, and indirect fitness, the number of relatives (nieces and nephews) produced multiplied by the degree of relatedness of those individuals.
The degree of relatedness (r) is the probability of being identical by descent through a common ancestor. As we shall see, this value is dependent on the genetic nature of the species involved. The degree of relatedness between diploid animals will defer than that for haplodiploid animals.
The parent to offspring relationship is r=.5 because each parent has half its genes in the offspring. The offspring to parent relationship is also r=.5 because half the offspring's genes come from each parent. Full siblings are related by r=.5 and half siblings are related by r=.25. shows how to calculate relatedness.
Let's now examine the case for haplodiploid organisms, such as ants, in which females inherit a set of genes from both parents, but males are haploid, resulting from unfertilized eggs, and receive only half of their mother's genes.
Now that we have included indirect fitness in our definition of inclusive fitness, we can see that individuals can derive some benefit by helping to raise their siblings or other relatives: through these siblings and relatives the genes of the helping individual are passed on. This is really not "altruism" in the true sense of the word because the individual does indirectly benefit. When and who should an individual help? The answer lies in the relative costs and benefits of the aid. Danger or giving up your own chance to reproduce are costs (C) of altruism. The benefit (B) is the aided individual's reproductive success. Therefore, individuals should only act altruistically when the indirect benefit is greater than the cost, or mathematically, when B > C.