A Gathering of Old Men takes place on the Marshall Plantation in Bayonne Louisiana. The plantation's Cajun work boss, Beau Baton, has been murdered just before the novel begins. Candy Marshall, the partial owner and general overseer of the plantation, discovers Beau's dead body outside of Mathu's house. Candy believes that Mathu killed Beau, but as Mathu is virtually her foster father she wants to make every effort to protect him. Candy immediately gathers everyone in the plantation to Mathu's house. When Miss Merle, the mistress of a local plantation arrives, Candy promptly confesses to the murder. Miss Merle does not believe her, but agrees to help. Candy decides that the best course of action would be to bring all the local men to Mathu's house with as many twelve gauge shotguns and empty shells as possible. With numerous men and guns at the crime scene, Candy believes that the local Sheriff will not be able to solve the crime. Miss Merle spreads Candy's plan through the community. Within a few hours, eighteen men have gathered at Mathu's house. All of these men are in their seventies and their eighties, but they have all brought their shotguns and are ready to confess.

In addition to the arrival of the Sheriff, most people at the plantation await the arrival of local Cajun named Fix. Historically, Fix led many lynch mobs against the local blacks. Everyone at the plantation believes that he will soon arrive once again to seek revenge, especially since the murdered man, Beau, is his son. Before anything else happens though, two other men arrive: Lou Dimes, Candy's boyfriend, and Sheriff Mapes. After the Sheriff sees Beau's dead body, he instructs his deputy to find Fix and keep him off the plantation as to avoid a lynching. Candy promptly tells the Sheriff that she committed the murder, but he does not believe her. The Sheriff eyes the old men and decides to question them about the crime.

The first two men that the Sheriff Mapes questions, Billy Washington and Gable, both confess to the murder. When their answers displease him, the Sheriff strikes them each twice. Next, the Sheriff queries the Reverend Jameson to find out what is happening. When the Reverend tells him nothing, the Sheriff punches him so forcibly that the Reverend falls to the ground. The watching men respond by all physically lining up so that the Sheriff can easily hit them all. Their willingness to confess and their indifference to being struck startle the Sheriff. He stops his questioning and thinks about the murder. The Sheriff knows that Charlie, who is not present, works most closely with Beau, but believes that Charlie is too weak to have committed the act. The Sheriff decides that the only culprit could be Mathu because Mathu is the only black man who has ever demonstrated the strength to stand up to local whites. For the Sheriff, Mathu is the only man who is man enough to murder.

Although the Sheriff has a suspect, he cannot arrest anyone because everyone confesses. As the afternoon goes on, many other black men confess to Beau's murder. They explain that they killed Beau because of what they have suffered— a sister raped; a son executed for crime he did not commit; and a brother killed for beating a tractor in a race. Beau died, they explain, for their past sufferings. The Sheriff still believes that Mathu murdered Beau, but can do nothing so everyone just waits to see if Fix Bauton and a lynch mob will show up.

Beau Bauton's brother, Gil, is a star football player on the Louisiana State University team who plays closely with a black halfback named Cal. The two players rely upon one another for their success. Due to their racial combination the press has dubbed them "Salt and Pepper." When Gil learns of his brother's murder, he treats Cal coldly. He visits the Marshall Plantation where he sees the gathering of old black men with guns. This sight, and Candy's immediate confession, combined with his brother's death grieves him greatly. Eventually, Gil makes it to home to where his father, Fix Bauton, and others are waiting. Fix, old family friends, and some local racist whites want to revenge Beau's death. Gil begs his father to let justice take its course and argues that the days of lynching are over. Gil refuses to assist his family in violence, citing his desire become an All-American football player. Luke Will, a local ruffian, criticizes Gil's perspective, and so too does Gil's father. Still, Fix will not revenge Beau's death without all of his sons. Without Gil, Fix decides to stay home. Luke Will, on the other hand, is up for the job and gathers some men to help him. They stop first at a local bar to drink some whisky in order to be prepared for the lynching.

Back at the plantation, Sheriff Mapes has found out that Fix is not coming. Mathu peacefully agrees to go to jail. Before they leave however, Charlie returns. Charlie confesses that he killed Beau after Beau threatened him. Charlie then asked Mathu to take the blame and fled. Charlie has spent the day hiding in the swamps, but felt called back to confess and show that he is truly a man.

Before Sheriff Mapes can take Charlie in, Luke Will and his crew arrive. They demand that Mapes hand Charlie over. When Mapes refuses, Luke Will shoots Mapes in the arm. The old black men grab live shotgun shells that they have been hiding in their pockets and start shooting at the whites. Sheriff Mapes remains fallen and injured in Mathu's front yard. The shooting surprises the whites and one of them, Leroy, is lightly injured. As the battle continues, Charlie and Luke Will become the primary fighters, dead set on eliminating the other. Eventually, Charlie rises from his hiding spot and manages to shoot Luke Will before he himself is shot to death. Everyone in the community pays homage to Charlie's dead body.

A trial for all blacks and whites involved in the shooting takes place a week after the event. The judge places everyone on probation for five years. At the trial's end, Mathu disappears with the other black men in a car, while Candy lingers on the courthouse steps with Lou. She grips his hand tightly as the novel closes in a gesture that reaffirms her commitment to their relationship.