It is little exaggeration to say that Grant failed at almost everything he tried to do while in the White House. Faced with Reconstruction, Grant tried to appease his supporters by pushing to give blacks the chance to vote. However, repeated and half-hearted attempts to enforce the Fourteenth Amendment and Fifteenth Amendment failed in the South, and Grant lacked the willpower to follow through. Even Congressional legislation failed to stop Ku Klux Klan violence in the South, as Grant could not make the laws stick. The Southern vote, not surprisingly, remained primarily white and allied with the Democratic Party against Grant.

By 1872, Grant had managed to alienate many of his supporters. They joined with the Southern Democrats to support Horace Greeley for the Presidency–one of the worst miscalculations of political history that century. Greeley, the fire-breathing New York abolitionist newspaper publisher, had been the man the Democrats loved to hate for more than a generation. His politics were much the same as Grant's on the issues of tariffs and civil service reform. Thanks to the Democrats' mistake of choosing Greeley–who managed to alienate even many of his own party–Grant won reelection in 1872, although his second term quickly proved that it would be no better than his first.

In 1873, the country was rocked by financial depressions, starting when a large bank, Jay Cooke and Company, filed for bankruptcy. Grant faced down increasingly pressure to allow inflation, vetoing legislation that would have allowed paper money again, and holding strong that gold money equaled sound money. Grant claimed that anything less might lead to financial disaster. His administration seemed to be greatly benefiting big businesses and companies at the expense of workers. The distrust of monopolies that developed at the turn of the century was finding its footing in Grant's government.

Furthermore, the scandals in Grant's administration only deepened. The entire administration looked like it tried to profit from the "salary grab" act of March 1873, which retroactively increased the pay of Congress and the President. Grant's Secretary of War became involved in graft on Army bases in the West, and the Speaker of the House received paybacks from a sketchy railroad deal. On the whole, Grant found himself way in over his head. His logical soldier's mind could not manage all the agendas and manipulations of those people around him. Members of his administration began to make claims that they were defending the American ideals that had been re-won on the battlefields of the Civil War–an expression that became known as "waving the bloody shirt." Throughout Grant's seven years as President, the shirt only waved faster and faster.

The campaign to replace Grant in 1876 only exacerbated everyone's fears, when Democrat Samuel J. Tilden won the popular vote but voting fraud in Florida and Louisiana allowed Rutherford B. Hayes to edge Tilden out in the Electoral College, 185 to 184. In an unspoken agreement, the Republicans pulled the federal troops out of the South in return for an undisputed transfer of power from the Democrats. One thing did help, however: whatever Grant's failings as a politician, everyone knew him to be a brilliant soldier. Talk of another civil war over the election results quickly died away with Grant still in the White House. Without incident, in March of 1877, Grant returned to private life.

Grant now set about trying to restore the image of the victorious four-star general over his current image of discredited corrupt President. He and Julia immediately set out for a tour of Europe. For the next two years, the couple traveled the world, visiting kings and queens, touring sites. Perhaps the best aspect of the trip abroad for Grant was the fact that he was treated with reverence as the man who won the Civil War. He ended the tour in San Francisco, where he had left in disgrace years before after resigning from the Army. The time away had done both Grant and his country good. People welcomed him back, and for once Grant seemed happy for a while.

By 1880, the pro-Grant party had been rebuilt. Hayes had had the audacity to initiate reforms throughout the government–something the party bosses in Florida and Louisiana who had rigged the election had definitely not bargained for. At the Republican convention that year, Grant supporters won a plurality to nominate the general for a third term, but could make no headway against the other candidates. On the thirty-sixth ballot James A. Garfield won the nomination.

Grant drifted to New York, arriving in the city almost penniless. He tried his hand at two business ventures, but both failed. Grant became worried about how he could provide for his family. When an article he wrote about the battle of Shiloh in 1884 sold surprisingly well, he began to write his memoirs. The writing soon became a race against death, as a lifetime of cigar smoking finally caught up with Grant in the form of throat cancer. Writing from his sickbed in Mount McGregor, New York, Grant formulated a masterpiece of war literature and commentary that stands today as one of the finest personal recollections of war ever published. Grant finished only a day or two before he died on July 23, 1885. He was buried in the mausoleum on the Hudson that bears his name.

In death, Grant succeeded at what he had failed at in life: providing a solid financial future for his family. Grant's Personal Memoirs, published by author Mark Twain, became one of the best-selling books of the time and raised more than $450,000 in royalties for Grant's family.