Jude travels to Sue's school in Shaston. He finds the schoolroom empty and begins playing a tune on the piano. Sue joins him, and they discuss their friendship. Jude accuses Sue of being a flirt, and she objects. They discuss her marriage, and Sue tells Jude to come to her house the next week. Later he walks to her house and sees her through the window looking at a photograph. The next morning Sue writes saying that he should not come to dinner, and he writes back in agreement. On Easter Monday, he hears that his aunt is dying. When he arrives, she has already passed away. Sue comes to the funeral. She tells Jude she is unhappy in her marriage, but that she still must go back to Shaston on the six o'clock train. Jude convinces her to spend the night at Mrs. Edlin's house instead. He tells her that he is sorry he did not tell her not to marry Phillotson, and she suspects he still has tender feelings for her.
Jude denies it, saying that he no longer feels love since he has seen Arabella and is going to live with her. Sue realizes he is lying. She confesses that she likes Phillotson but finds it tortuous to live with him. Jude asks if she would have married him if not for his marriage to Arabella, but Sue leaves without answering. In the middle of the night, Jude hears the cry of a trapped rabbit and goes outside to free it. He kills the rabbit and looks up to see Sue watching him through a window. She says she wishes there was a way to undo a mistake such as her marriage. She kisses Jude on the top of his head and shuts the window.
Jude decides that he cannot in good conscience become a minister, considering his feelings toward Sue. He burns his books. Back in Shaston, Sue hints at her indiscretionary feelings to her husband. At night she goes to sleep in a closet instead of her bedroom, and Phillotson is alarmed. She asks if he would mind living apart from her. He questions her motives and asks if she intends to live alone. She says that she wants to live with Jude. In the morning, Phillotson and Sue continue their discussion through notes passed by their students. She asks to live in the same house, but not as husband and wife, and he says he will consider it. They take separate rooms in the house, but by habit one night, Phillotson returns to the room they once shared, and sees Sue leap out the window. However, she is not badly hurt and claims that she was asleep when she did it.
Phillotson goes to see his friend Gillingham and tells him of his marital troubles. He speaks of his intention to let her go to Jude, and Gillingham is shocked. He says that such thoughts threaten the sanctity of the family unit. At breakfast the next day, Phillotson tells Sue that she may leave and do as she wishes. He says he does not wish to know anything about her in the future.
Jude meets Sue's train and tells her he has arranged for them to travel to Aldbrickham because it is a larger town and no one knows them there. He has booked one room at the Temperance Hotel, and Sue is surprised. She explains that she is not prepared to have a sexual relationship with him yet. He asks whether she has been teasing him. They go to a different hotel, the one where he stayed with Arabella. When Jude is out of the room, the maid tells Sue that she saw him with another woman a month earlier. Sue accuses him of deceiving her, but he objects by saying that if they are only friends, it does not matter. She accuses him of treachery for sleeping with Arabella, but he argues that Arabella is his legal wife. Jude tells Sue that Arabella has married a second husband, but he will never inform against her. He adds that he is comparatively happy just to be near Sue.
Back in Shaston, Phillotson is threatened with dismissal for letting his wife commit adultery. He defends himself at a meeting but falls ill. A letter reaches Sue, and she returns to him. She tells Phillotson that Jude is seeking a divorce from his wife, and Phillotson decides to attempt the same.
The moral implications of the friendship and romance between Jude and Sue emerge as an important issue. Hardy dwells on the question of marriage and its ramifications, and his portrayal of the tragic effects of marital confinement, beginning largely in Part IV, did not sit well with critics of the time. Hardy was accused of attempting to undermine the institution of marriage, and Sue in particular was thought to have inappropriate beliefs for a young female character. In many ways, she is a feminist before her time. She recognizes her own intellect and her potential for a satisfying career in teaching, and marries Phillotson partly out of a desire for a pleasant work environment. She resists a romantic relationship with Jude, but falls in love with him despite her misgivings. However, when it comes time to marry, she does not wish to enter into a legal contract in which she would again be confined.
By marrying Phillotson, Sue hopes to protect her reputation and achieve the traditional lifestyle of a married woman. She likes Phillotson despite his age, but is surprised at her inability to find him attractive. She even comes to be repulsed by him and later admits to jumping out of the window for fear that he would enter her bed. Phillotson tries very hard to preserve at least the external appearance of a typical marriage. As a man, he is legally permitted to force her to stay in his bed and even sleep with him. For this reason he is viewed with contempt for letting her leave him. However, his understanding brings him only more difficulty, as he is personally blamed for Sue's disobedience of convention.
Jude's relationship with Arabella is equally complicated. He does not love her as much as he cares for Sue, but he sleeps with her when she returns from Australia. Again, Hardy's casual depiction of people acting against established societal norms of marital and sexual behavior aroused controversy in Britain and the United States, and Hardy resolved to give up writing fiction as a result.