The novel begins in the midst of the love triangle between Violet, Joe and Dorcas. Violet and Joe are unhappily married and living together in an apartment in Harlem when Joe falls in love with a seventeen-year old girl named Dorcas. Joe and Dorcas meet when Joe comes to Dorcas's aunt's house to sell ladies cosmetics, and their affair lasts from October of 1925 to the first of January 1926. Joe talks with Malvonne, an upstairs neighbor, and negotiates the use of her empty apartment so that he and Dorcas can meet there. This arrangement continues for several months and neither Violet nor Alice Manfred, Dorcas's aunt, have any knowledge of the affair.

Although Joe brings Dorcas presents every time they meet, eventually Dorcas begins to get tired of the older man and starts going out with younger boys, attending parties with her best friend Felice, and making up excuses so as not to meet with Joe. When Joe finally confronts Dorcas about this, she cruelly tells him that he makes her sick and that he should not bother her anymore. Dorcas prefers the attentions of a popular and good-looking young man named Acton, with whom she dances at a party on New Year's Day. Dorcas knows that Joe has not gotten over her and will come looking for her, so she is only half-surprised when he tracks her down at the party and sees her dancing with Acton. Joe, however, brings a gun and shoots Dorcas in the shoulder. Dorcas tells the alarmed witnesses not to call an ambulance, even though she would survive if she allowed someone to help her, and she consequently bleeds to death. Everyone knows that Joe shot Dorcas and rumor of their affair begins to spread in the community after the young girl's death. Violet appears unexpectedly at Dorcas's open-casket funeral and slashes Dorcas's face with a knife. Several weeks later, she begins to visit Dorcas's mourning aunt, Alice Manfred, and the two women begin to develop a friendship as a result of their shared tragedy. In the spring, Joe mourns Dorcas's death and he and Violet patch things up in their relationship, mediated in part by their new friendship with Dorcas's best friend, Felice.

As the narrator tells the story of Violet, Joe, and Dorcas in Harlem she follows a stream of associations and digressive details to create a complex web of people, places, and stories extending back to the late nineteenth century. Violet grew up in a poor household in Virginia with her mother Rose Dear. Her grandmother, True Belle, came from Baltimore to live with them when Violet's father abandoned the family. Soon afterwards Violet's mother, Rose Dear, committed suicide by throwing herself into a well. Joe also grew up in Virginia. He was orphaned at birth and raised by adoptive parents. As a young man he wondered about his birth mother's identity and tried on several occasions to find her. His mentor, a hunter named Henry LesTroy and called "Hunters Hunter," hinted to Joe that his mother was the local mystery, a crazy homeless rover named Wild. When Joe finally tracked Wild down in the woods he asked her to confirm somehow that she was indeed his mother. Wild responded with a hand gesture that Joe could not make out, leaving him to question his own identity. Joe and Violet met in a town called Palestine where they were working the fields. They got married and moved to Harlem, which is referred to simply as "the City" throughout the novel.

In the course of telling Joe and Violet's story, the narrator recounts the stories of periphery characters such as Vera Louise Gray and her son Golden Gray. The narrator shows the connections between the characters, focusing on the perspectives and experiences of individuals and sometimes allowing them to narrate their stories in their own words. Golden Gray, the mixed-race child of a white woman, Vera Louise, and a Black slave, Henry LesTroy, was raised by his mother and True Belle in Baltimore. He believed all his life that he was a white adopted orphan, but when True Belle told him the truth about his father, he set out for Virginia to confront Henry LesTroy. When he arrived near Vienna, Virginia, Golden Gray spotted Wild hiding alongside the road. When she turned quickly and knocked herself unconscious, he decided to take her with him to his father's home. Wild was very pregnant and gave birth to Joe when they arrived at Henry LesTroy's house. Golden Gray never returned to Baltimore after this incident but lived with Wild in the woods, totally apart from civilization. These stories about Harlem and Virginia are recapitulated and fleshed out several times throughout the novel in flashbacks and digressions.