Ernst Mayr, a leading figure in twentieth century evolutionary thought, saw behavior as a continuum ranging from completely closed, or fixed by the genotype, to completely open, extremely flexible and dependent on the environment. Behavioral genetics has become an increasingly important field as modern technology has allowed researchers to locate specific genes and alleles responsible for some behaviors. The field has also stirred controversy: people are hesitant to attribute behavior to genetics in the age- old nature versus nurture debate. But nature versus nurture is a false dichotomy. The truth is that both can play a key role in determining a behavior. The genotype determines the potential for a behavior to exist, while nature determines what phenotypic behavior actually results.
Behavioral genetics are more complicated than Mendelian genetics because while Mendel's pea plants showed clearly distinctive characteristics, animal behavior does not always fit into such hard and fast categories. Successful identification of behavioral genes is most likely when the gene shows a high penetrance, that is, the genotype is expressed in the phenotype. Degree of expression is also important for success in determining behavioral genes. Behaviors that can be readily observed and quantified or categorized are easier to work with.
While some behaviors are indeed attributable to a single gene, most behavioral traits are polygenic, or influenced by several genes. Further, more often than not, the environment will mediate the affects of those genes. Such a complex situation can be troublesome for researchers seeking to isolate the genetic mechanisms of behavior. In order to create some standard to measure specific behaviors in relation to genes and environment, scientists have devised a scale of heritability. Measured on a scale between 0 and 1, heritability is a measure of how much of the variance seen in a trait is due to its inherited components rather than its environmental components. A heritability value of 0 means that a certain behavior is completely independent of genetic makeup; a heritability value of 1 means that a behavior is totally dependent on genes. The actual calculation of the variance due to heredity can be extremely complicated, and we need not be concerned with that calculation here. The important thing to remember is that a heritability value that is above 0 and less than one indicates that neither nature nor nurture is the sole factor in determining a behavior.
Behavioral genetics as an experimental field can be extremely technical and complicated. Here we will examine some of the important techniques and approaches to the study of behavioral genetics.
In order to tease apart the genetic and environmental factors contributing to a behavior, it is useful to be able to hold one factor constant. Inbreeding animals over many generations will produce a population that is homozygous and genetically identical. Once experimental subjects are genetically identical, variations in a behavior due to environmental differences can be identified and their relative importance analyzed. Mice are commonly used for such inbreeding experiments. After about thirty generations, the population is about 98 to 100 percent homozygous.
To examine the relative importance of genetic components of behavior while holding environmental components constant, two or more inbred strains may be used. Two different inbred strains that are different from each other can be assessed while holding the environment constant across both strains.