The film opens in 1951 at an inn located on the Northern California coast. Doris (Ellen Burstyn) is a 24-year-old housewife from Oakland, George (Alan Alda) a 27-year-old accountant from New Jersey. They meet at dinner, have an affair, and then agree to meet once a year to rekindle the sparks they experience at their first meeting, despite the fact both are happily married with six children between them.
Over the course of the next two dozen years, they develop an emotional intimacy deeper than what one would expect to find between two people meeting for a clandestine relationship just once a year. During the time they spend with each other, they discuss the births, deaths, including George's son Michael dying in Vietnam, which changes George politically, and marital problems each experiences at home, while they adapt themselves to the social changes affecting their lives.