The song was written for the caretaker of the Northern California Broken Arrow Ranch, which Young purchased for $350,000 in 1970. The song compares a young man's life to an old man's and shows that the young man has, to some extent, the same needs as the old one. James Taylor played six-string banjo (tuned like a guitar) and sang on the song, and Linda Ronstadt also contributed vocals. In the movie Heart of Gold, Young introduces the song as follows:
About that time when I wrote (Heart of Gold), and I was touring, I had also -- just, you know, being a rich hippie for the first time -- I had purchased a ranch, and I still live there today. And there was a couple living on it that were the caretakers, an old gentleman named Louis Avila and his wife Clara. And there was this old blue Jeep there, and Louis took me for a ride in this blue Jeep. He gets me up there on the top side of the place, and there's this lake up there that fed all the pastures, and he says, "Well, tell me, how does a young man like yourself have enough money to buy a place like this?" And I said, "Well, just lucky, Louie, just real lucky." And he said, "Well, that's the darndest thing I ever heard." And I wrote this song for him.
He tells a similar story when introducing the song at a February 23, 1971 performance broadcast by the BBC (in which he says that he purchased the ranch from "two lawyers").
At former St. Louis Blues Brett Hull's jersey retirement ceremony, this song was played as his jersey rose to the rafters.
"Old Man" was released as a single in the spring of 1972, and reached no. 31 on the Billboard Hot 100 singles chart for the week ending June 3.
In 1996 the song was covered by Metal band Cyclone Tracy on their album "One Eyed."
The Finnish band, HIM has a rare track featuring the band covering the song, despite relatively poor quality.
The song was covered by American R&B/Soul singer N'Dea Davenport on her 1998 debut album, N'Dea Davenport.
In 2009, German Metal band Motorjesus covered the song on their 100.000 Volt Survivor EP. It was also released as a bonus track on their album Wheels Of Purgatory.
In 2010, Donna Loren covered the song on her album Love It Away.