Mrs. Alving is feeling rebellious, and she goes on to suggest that she would approve of a marriage between Regina and Oswald, as long as the couple made the decision in full knowledge of their true genealogy. She goes so far as to remind the Pastor of all the incestuous marriages he must ordain throughout the countryside. The Pastor avoids the subject and reverts to an earlier topic, asking her what she meant by calling herself a coward. She explains that she is haunted by ghosts in her mind. She describes the ghosts as being more than that which we "inherit" from our parents; they are all the ideas and habits that are passed down between generations as larger units and which we are afraid to give up—which make us afraid of "the light."
Pastor Manders dismisses such thoughts as the result of the books Mrs. Alving has been reading, but she says that they are rather the result of the sense of "duty" the Pastor infused her with when she fled to him. The Pastor is moved, asking if this cramping sense of "duty" is all that came from his life's greatest battle with himself; he says it was a struggle for him to send her back because he, too, had feelings for her. But she insists that when she came to him and said, "here I am—take me," it was a crime for him to turn her away. They agree that they do not understand one another at all. The conversation has turned back to Engstrand when a knock is heard at the door.
Mrs. Alving's notion of "ghosts" gets at the heart of the play. Not only are there ghosts in the sense of specific persons come back to life in new forms—as one could figuratively say that Regina is a ghost of Johanna—there are ideas that "haunt" generation after generation. As a priest, Pastor Manders is a purveyor of these ghosts. His emphasis on ideas of familial loyalty is a plague on all the characters in the play. Regina does not want to return to Engstrand; Oswald does not believe that marriages must be sanctioned by the church; Mrs. Alvin does not believe that she should have been loyal to her husband. The Pastor, however, is not the only source of these ideas. They truly are ghosts; they permeate, and hover within the entire community; they haunt each character.
Mrs. Alving says that these ghostly ideas make us afraid of the light. Throughout the play, gloom, clouds, and rain symbolize hypocrisy, fear, duty, and the general cowardice generated by an obsession with public reputation. Later in the play, we will see how several sources of light upset this gloom.
In the ensuing conversation, we see the Pastor and Mrs. Alvin at their most intimate. Apparently, their attraction was mutual. The Pastor has over-dramatized it in his memory, considering the episode his greatest battle with himself. Here, Ibsen demonstrates how people can make themselves unhappy out of a sense of duty. Mrs. Alvin resents the fact that the Pastor denied both of them happiness. Here we see the root of her belief that "law and order" cause unhappiness, the root of her fear of cowardice, and the root of her strong desire to break free. Indeed, if she is a radical who never acts on her radical principles, then we can think of the time she tried to leave her husband for the Pastor as the one time she did act on principle. Because the object of her desire refused her, her impulse to act died.