The band debuted with three scattered gigs (all in Athens, Georgia) in 1984. The group played mostly cover tunes, though a few unreleased originals also made it into the mix. The first gig took place on Valentine's Day, 1984, and featured Bryan Cook (vocals and piano, a member of Athens bands Oh-OK and Time Toy), and R.E.M. members Bill Berry (drums) Peter Buck (guitar) and Mike Mills (bass). The follow-up gig took place two weeks later; added to the line-up was R.E.M. lead singer Michael Stipe on vocals and drums, and Warren Zevon on vocals. Zevon performed his hit "Werewolves of London", as well several other songs that were, at that point, unrecorded (including "Boom Boom Mancini" and "Trouble Waiting To Happen") The final 1984 gig took place in June, and featured the Cook/Berry/Buck/Mills line-up.
The group entered the studio as a quintet that summer, with the line-up of Berry/Buck/Cook/Mills/Zevon. They recorded two songs for release as a single, which were eventually released in 1986. The A-Side, "Gonna Have A Good Time Tonight" was a cover of an Easybeats tune; the B-side, "Narrator," was a Bill Berry composition that R.E.M. had played live, but never recorded. (The song's composition was credited to Berry/Buck/Mills/Stipe).
After a period of inactivity, Hindu Love Gods played one 1986 gig in Athens as a benefit for the family of the recently deceased D. Boon. Line-up for this performance was Berry, Buck, Cook, Mills and Stipe.
Buck, Mills and Berry later joined Zevon as his back-up band while recording Zevon's solo album Sentimental Hygiene (1987). During an all-night (and supposedly drunken) session in the midst of recording Zevon's album, the four recorded ten cover songs, mostly blues standards. Although originally not intended for publication, these recordings were finally released by Giant Records on the album Hindu Love Gods (1990), with the artist credit going to Hindu Love Gods. The song that received the most attention was a rock version of Prince's 1985 hit "Raspberry Beret", which reached No. 23 on the Modern Rockcharts.
During a concert at The Shadow in Kansas City in December 1990, Zevon commented that the album was "selling by the shitload," whereupon one of his backing band informed him that it was "selling like shit." The album peaked at No. 168 on Billboard's Top Album Charts.