The new Constitution of the United States of America, formally ratified on September 17, 1787, laid out a daring foundation for the new government. With his work in England mostly completed, moreover, Adams resigned from his overseas posting and returned home. On April 20, 1788, Adams and his wife boarded a ship in Portsmouth, England, and set sail for America. He returned to a hero's welcome: Celebratory booming cannon and a gala reception at the home of Massachusetts Governor John Hancock helped reassure Adams that his work had not gone unnoticed. He quietly returned to life on his Braintree farm, which the Adams family named "Peacefield," and hoped to spend some time recuperating from his years abroad. However, he found himself drawn to the new government that was forming from the Constitution. Too proud to ask for an office, he hinted that the presidency should go to Washington and that he hoped he would not end up too much lower. Despite his humility at Congress, he told neighbors that he would not accept the lowly post of senator from Massachusetts. Washington happily urged others to support Adams for vice president, expressing the general's desire to step down once the government got going and leave it in the hands of someone capable–like Adams. Adams won the vice presidency, but only with the support of some "apprehensive" states. Adams took this half-hearted endorsement as a personal affront and he was hurt by the seeming lack of support for his years in public service. Nonetheless, when he departed Boston on April 12, 1789, he was full of energy and excitement for his new job.

The vice presidency, however, was not everything he thought it would be. Adams was sworn in on April 20, 1789 and would be reelected in 1792. He found the office, which had been designed to provide a new head of state in the event of the death of the president, ill-suited for someone as opinionated and involved as he was. In fact, he chafed at having to spend his days in the shadow of the president, so much so that he would remark that the vice presidency was "the most insignificant office that ever the invention of man contrived."

What little power the vice president did have, he wielded in the U.S. Senate, where he presided as president of the Senate, the upper house of the newly formed Congress of the United States. He was only allowed to vote in the case of a tie but he did so twenty times–each time voting to support the policies of President Washington and supporting the expansion of the role of the new federal government. Adams voted to support American neutrality in a new war between Britain and France and to seek reprisals against Britain for interfering with U.S. shipping commerce. He also supported the financial measures of the ambitious secretary of the treasury, Alexander Hamilton.

One of Adams' earlier predictions–that despite all efforts to the contrary, political factions were inevitable–proved true right from the start of the new government. Despite President Washington's desire to serve as president for the whole nation, he found himself undercut by the increasing division of the government into political factions, the forerunners of today's political parties. His own Cabinet even developed the same divisions he sought to prevent.

The government began to separate into the Federalists and the Anti-Federalists; although there were degrees of moderation in each group, the general consensus was that the Federalists, led by Hamilton, supported the development of a strong federal government while the Anti-Federalists or Republicans (later still to become the Democratic-Republican Party), led by Secretary of State Thomas Jefferson, believed strongly in the autonomy of the individual states–the Republicans worried that a strong centralized government could lead to the same type of abuses that had led America to break away form Britain in the first place. Jefferson wanted every person (in this case, every white man) to play a role in his own governance.

Adams had begun his career long ago as a lawyer defending the rights of people and had written extensively on natural laws and rights. Now, he initially endorsed Jefferson's ideas of government by the masses. However, Adams had reservations. Adams recalled the days of mob rule in Boston prior to the beginning of the war and was just as worried of that as he was of a strong monarch. The chaos of the French Revolution only strengthened his resolve that a strong central government was the way to go. In 1791, Adams published a series of essays entitled, "Discourses on Davila," which commented on civil disorder, mainly the French Revolution. (The title of the piece came from the Italian historian Enrico Caterino Davila.) What Adams wanted was a "an independent executive authority, an independent senate, and an independent judiciary power, as well as an independent house of representatives." The best case was a combination of democracy and monarchy, he felt. Thus, despite his deep friendship with Jefferson–stretching back to when they wrote the Declaration of Independence–Adams found himself a Federalist. That being said, he did not agree with all of the faction's ideas: just as he hated mob rule, he was wary of Hamilton's belief that government should be controlled by the aristocracy.