Shortly before Christmas, an unidentified sailor jumps overboard and swims toward the harbor of Queen of France in the middle of the night. Unable to reach shore, he climbs aboard a small yacht and stows away. When the yacht lands, he disembarks onto a small island called Isle des Chevaliers, and he hides again, this time at a house called L’Arbe de la Croix. Valerian Street and his wife, Margaret, live in the house, along with their servants, including Ondine and Sydney. Jadine Childs, the niece of Ondine and Sydney, has come to visit. Before coming back to the island, she studied at the Sorbonne, an education for which Valerian paid, and worked as a model in Paris.
Margaret and Valerian fight a lot, mostly because Margaret has invited a number of guests to come stay, against Valerian’s wishes. One of these guests is the Streets’ son, Michael, but Valerian doubts that he will really come. After a particularly fierce argument at dinner one night, Margaret goes to her room, but she quickly returns to the dining room screaming. Sydney runs to Margaret’s room and returns to announce that there is a man hiding in her closet. Everyone but Valerian is terrified. Valerian invites the man, whose name is Son, to stay the night.
The next morning, Margaret, who is extremely upset with Valerian, locks herself in her room. Meanwhile Jadine’s rich, white boyfriend, Ryk, has sent her a luxurious sealskin coat. Son shows up in her room as she tries on the coat, and they talk in a flirtatious way that eventually starts to frighten Jadine. After Son makes some sexually crude remarks, Jadine threatens to report him to Valerian, after which she goes to find Valerian.
As another servant, Thérèse, does laundry, she thinks about Son, who had been in the house for several days prior to his discovery and whom Thérèse had been feeding. With Jadine gone to find Valerian, Son showers in her bathroom. When he is clean, he looks much more attractive. He finds Valerian before Jadine does, and he impresses Valerian with his knowledge of gardening and his sense of humor. Valerian tells Sydney to help Son get new clothes, and Gideon, another servant, and Thérèse take him shopping in a town near L’Arbe de la Croix. When Jadine sees the cleaned-up Son, she decides not to tell Valerian about his behavior in her bedroom. Instead, she invites Son on a picnic at the beach, and they talk a lot about their backgrounds. Despite their differences, they seem to connect on some level. On the way back from the picnic, their car runs out of gas, and Son leaves to retrieve gas from a pump at the pier. While Jadine waits, she decides to seek shelter from the sun and abandons the car. On her way to some nearby trees, she gets stuck in a swamp but manages to escape. Ondine is upset that Jadine and Son seem to be getting closer, but she does not intervene.
When Christmas arrives, Michael fails to show up, and the other guests get delayed because of bad weather. Margaret’s spirits sink, and she abandons her elaborate cooking projects and leaves Ondine to finish them. At Christmas dinner, Valerian upsets Ondine, Sydney, and Son when he announces that he fired Gideon and Thérèse for stealing apples. A heated argument breaks out. At the end of it, Ondine reveals that Margaret abused Michael when he was a boy. Valerian goes into shock, and Son and Jadine leave the table and go to bed together.
Soon after, Jadine and Son leave the island. They go to New York, having a great, carefree time as lovers. They live in a borrowed apartment, and neither of them has a permanent job, but they don’t seem to care very much about money. Meanwhile, back on the island, things are much more subdued, and Valerian refuses to let Margaret explain her actions to him. Ondine and Sydney worry that they will be fired.
When spring arrives, Jadine and Son visit his hometown of Eloe, Florida. The trip is a disaster for their relationship, because Jadine hates Eloe, and Son loves it. The many differences between Jadine and Son come to the surface, and their divisions tear them apart when they end up back in New York. They fight more and more frequently. After a particularly violent confrontation, Jadine leaves Son and New York behind. She intends to return to Paris, but first she stops at Isle des Chevaliers to retrieve her sealskin coat. Ondine is upset that Jadine seems to care more about the coat than about either Ondine or Sydney, but her anger does not detain Jadine, and ultimately she heads to Paris, telling Ondine and Sydney not to tell Son where she has gone. Soon after she departs, Son arrives in Queen of France, and Thérèse agrees to take him to Isle des Chevaliers by boat, so he can look for Jadine. But instead of piloting him to L’Arbe de la Croix as she had promised, Thérèse leaves Son on a foggy part of the island, and she suggests that he still has a choice. He can either keep searching for Jadine, or he can join the race of wild horsemen on the island, descendants of the first slaves brought there. The island opens to accommodate Son as he joins the horsemen.