Oerter began his Olympic career at Melbourne in 1956. He was not considered the favorite but he felt a rush during the competition and he unleashed a career best throw of 184'11" (56.36 meters). The throw was good enough to win the competition by more than 5'. It appeared Oerter's career would be over at the age of 20, however. In 1957, an automobile accident nearly killed him. He did recover in time to compete at the 1960 Summer Olympics at Rome. Oerter was the slight favorite over teammate and world record holder Rink Babka.

Babka was in the lead for the first four of the six rounds. He gave Oerter advice before his fifth throw and Oerter threw his discus 194'2" (59.18 m), setting an Olympic record. Babka was not able to beat Oerter's throw and finished with the silver. During the early 1960s, Oerter continued to have success. He set his first world record in 1962. In the process, he was the first to break 200 feet in the discus. He was considered a heavy favorite to win a third gold medal at Tokyo in 1964.

Injury seemed to have felled Oerter before the Games. He was bothered by a neck injury then he tore cartilage in his ribs shortly before the competition. Competing in great pain, Oerter set a new Olympic standard and won a third Olympic gold medal despite not being able to take his last throw due to the pain from his ribs. As before, he bettered his own record with a throw of 61.00 meters.

Oerter returned to the Olympics in 1968 at Mexico City but he had yielded the position of favorite to teammate Jay Silvester. Many felt that Oerter, at 32, was finished since Oerter had never thrown as far as Silvester did on his average throws. This was the Olympics, however. Oerter released another Olympic record throw of 64.78 meters on first throw. His record held and he became the first track and field athlete to win four consecutive gold medals.

Oerter retired from athletics after the 1968 Olympics. He did make an attempt to qualify for the American team in 1980 but he finished fourth. He nonetheless set his overall personal record of 69.46 meters (227'10¾") that year at the age of 43. When filming for a TV segment, he unofficially threw about 245 feet (74.67 meters), which would have set a still-standing world record.

Oerter Carried the Olympic Flame into the Stadium in the 1996 Olympic Games.

Trivia
Loaned his 1968 gold medal to the producers of the 1979 Susan Anton film Goldengirl. It was "ruined," Oerter said later.

When anouncing his retirement from track & field was quoted as saying "I only regret that I was never as good as Brian Oldfield, he is the best athlete of our time."