One day in early summer, Dicey Tillerman's mother begins a trip with her children in their battered station wagon to see their Aunt Cilla in Bridgeport, Connecticut. When they have driven about half the distance between their home in Provincetown and Bridgeport, Momma stops the car and gets out, telling the three younger children, James, Maybeth, and Sammy to mind Dicey. Their Momma does not return, and the next day the children, under Dicey's determined leadership, set out to walk to Bridgeport, with only a map, a change of underwear, and seven dollars. The children walk for days on end, buying food as cheaply as possible, scrounging for change alongside of the road, and sleeping in or near empty houses. After several days, they stop to rest in a state park, where clams, mussels, and fish abound, and they meet a young runaway couple, Edie and Louis. In the park, Sammy, to Dicey's dismay, steals both food and money, and they leave in a fright when the police begin looking for them.

By the time they reach the Connecticut River, they are completely out of money. Dicey decides to earn money by washing windows, and soon has the children carrying bags in the parking lot of a grocery store. Before long, they have enough money to continue on. They cross the forbidding river in a rowboat, and continue on towards the west. When they reach New Haven, however, they are once again out of money, and Dicey is near despair. A couple of Yale students, Stewart and Windy befriend them, take them in for the night, and feed them. That night, James steals twenty dollars from their benefactors, which he grudgingly returns the next morning. Dicey chastises him severely, and Stewart talks seriously to him about the moral implications of his actions. The next day, Stewart drives them to Bridgeport, leaving them at their aunt's door.

The children find that Aunt Cilla is dead, and their fussy and pious cousin, Eunice, occupies the house alone. Eunice takes the children in, and soon the three younger children are attending a church camp every day while Dicey helps Eunice care for the house. Dicey also begins to earn money by washing windows around town. In Bridgeport, Dicey talks to the police about Momma's disappearance, and before long, the police have located her: she is comatose in a mental hospital in Boston. The Tillerman children are unhappy in Bridgeport. Sammy gets in fights, upsetting Eunice, the nuns want to label Maybeth as retarded and give her special schooling, and Dicey feels the children growing apart from one another. When Eunice decides to give up her dream of becoming a nun and adopt the children—although she still considers handing the belligerent Sammy over to the state—Dicey acts quickly. While in Bridgeport, they have learned of their grandmother, Abigail Tillerman, who lives in Crisfield, Maryland, and Dicey decides they must meet her.

Dicey and her siblings leave without Eunice's permission, boarding a bus one morning for New York. They eventually take a bus to Wilmington, where Dicey panics when she finds they have missed the bus to Crisfield and buys tickets for Annapolis, worried that Eunice may have the police looking for them. In Annapolis, they convince two boys, Tom and Jerry to sail them across the bay, and during the trip, Dicey falls in love with sailing. Once across, Dicey, determined to save money for a possible return trip, decides they must walk again. They find a circus and befriend the proprietor, Will. After a couple of days, they see signs advertising work picking tomatoes, and Dicey and the children approach a local farmer, Rudyard, asking for work. The children sense the man's untrustworthiness, but work through the morning. By the time evening comes, they are sure the man means them harm, and when he pulls up in his truck, they dash away into the night, crossing a nearby river. He pursues, but soon gives up his chase. The next day, however, he finds them in a nearby town and pursues them hotly. The circus has come to the town, however, and Will protects the Tillermans from the evil man.

The Tillermans spend the next several days with the circus, and Will offers to drive them to Crisfield when the circus moves to a town near it. When they finally arrive in Crisfield, Will wants to stay and make sure the children are all right, but Dicey explains they must face this challenge alone. The children find their belligerent and eccentric grandmother on a run-down farm seven miles outside of town. The grandmother agrees to let them stay the night, but adamantly refuses to let them stay for good. Dicey and the others, however, start to fall in love with their grandmother's beautiful farm on the shore, and Dicey has found an old sailboat in the barn, which she is determined to use. The children decide that they will start doing work around the farm, postponing their departure one day at a time until their grandmother gets used to them. The children work diligently for about five days, until Will pays a visit, bringing the children used bicycles as a gift. That night, however, their grandmother explodes when Sammy rides off without permission and Dicey refuses to allow their grandmother to punish him by sending him to bed without supper. When Dicey resignedly asks if they can stay, their grandmother refuses.

That night, Dicey finds her grandmother writing a letter to Eunice in the kitchen. She takes this opportunity to explain to Dicey the reasons she cannot take the children in: she has little money and no income, she cherishes the freedom and independence she gained when her stern husband died four years ago, and she is afraid of repeating the mistakes she made as a mother that drove her children away. Dicey's grandmother admits to Dicey, however, that she likes the children and wants them to stay. The next morning, their grandmother seems to remember little of her proclamation of the night before, and several days later, she takes the children to town to register for school, explaining that it may take weeks for Eunice to write back to her letter, expressing whether she will take the children back. At school, Maybeth passes a test allowing her to study in the third grade, much to the joy of all her family members. As they are preparing to return home, Dicey realizes her grandmother has forgotten to mail the letter to Eunice. She reminds her grandmother of the letter, but before she can get to the letterbox, Dicey stops her, telling her that she should take the children in, even if she does not want to. To her surprise, her grandmother agrees, stating in mock exasperation that the children have worn her down. Overjoyed, the five return home together, united in their sense of hope and relief.