Ruffin had many admirers among his fellow artists. "Nobody could sing like David Ruffin", said his close friend and colleague Martha Reeves (of Martha and the Vandellas fame). His contemporary, label-mate, and long-time acquaintance Marvin Gaye was particularly impressed with the virility of Ruffin's voice. Gaye said Ruffin's work "made me remember that when a lot of women listen to music, they want to feel the power of a real man.

Daryl Hall said, "His voice had a certain glorious anguish that spoke to people on many emotional levels". Ruffin himself said, "I don't know what kind of voice I have, I really don't...it's just about the feeling I get for the song".

Rod Stewart said: "['I Wish It Would Rain'] jumped out of the speakers and ravished my soul". Stewart would later become friends with Ruffin. "His voice was so powerful—like a foghorn on the Queen Mary", Stewart told Rolling Stone in 2005.

The cover art of Ruffin's last album, Gentleman Ruffin, was the inspiration for the art of rapper Wiz Khalifa's mixtape Kush & Orange Juice.

In 2013, David Ruffin was inducted twice into the Rhythm & Blues Music Hall of Fame at Cleveland State University. He is inducted as a solo artist and member of the Temptations. Ruffin is also inducted into the Mississippi Musicians Hall of Fame.

In June 2019, the city of Detroit unveiled "David Ruffin Avenue" as the secondary street name where he formerly lived at 17385 Parkside. The ceremony was hosted by the founder LaMont Robinson of the National Rhythm & Blues Hall of Fame. It was attended by Ruffin's family and friends including Martha Reeves and Mary Wilson.

On October 26, 2019, the city of Meridian ceremonially named a section of 8th Street near the Temple Theatre "David Ruffin Boulevard. Ruffin was born in nearby rural Whynot, but claimed Meridian as home. Ruffin's family was in attendance for the honor and the unveiling of his star on Mississippi's Arts + Entertainment Experience's walk of fame. During the celebrations, Jackson State University's "Sonic Boom of the South" marching band led a parade. Lamont Robinson, head of the National Rhythm & Blues Hall of Fame and also Ruffin's son-in-law, presented the idea to Meridian.

In 1998, NBC aired The Temptations, a four-hour television miniseries about the group's career, Ruffin was portrayed by Leon Robinson (who is usually credited as simply "Leon" as his professional name)

Grand Puba of Brand Nubian referenced the singer on the 1990 single "All for One", stating: "I hit a beat and swing a note as if my name was David Ruffin."

Rapper Machine Gun Kelly also referenced the singer in his 2013 song "See My Tears" stating: "Fought every temptation shit, I guess I'm David Ruffin, huh?" In 2019, rapper Lil Durk referenced David Ruffin on a song of the same name, off his mixtape “Love Songs 4 the Streets 2.” The song states “I know a killer who got inside that jam we call him David Ruffin, and they didn’t even use his statement so he sang for nothing.” The track references Ruffin’s singing abilities metaphorically, as “singing” is slang in many urban neighborhoods for snitching, or telling on someone to law enforcement in order to get a lighter sentence. But, the Feds didn’t end up using his statement, so he did all that for nothing. Little Brother sampled Ruffin's song Slow Dance for their song Slow It Down

The first song on Fall Out Boy's 2005 CD From Under the Cork Tree was to have been titled "My Name is David Ruffin...and These are The Temptations," but for legal reasons the name was changed to "Our Lawyer Made Us Change the Name of This Song So We Wouldn't Get Sued."

Hip-hop group Hotstylz used Ruffin's song and referred to his name on their single "Lookin' Boy".

NLE Choppa referenced Ruffin on Polo G's 2020 single, "Go Stupid" stating: "Givin' niggas temptation like he David Ruffin".

American rap rock group Gym Class Heroes references Ruffin in the 2008 song "Like Father, Like Son (Papa's Song)" from their album The Quilt stating: "I never understood temptation. But I guess we both got a little David Ruffin in us."

Ruffin is portrayed in the Broadway musical, Ain't Too Proud, by Ephraim Sykes.

Ruffin's performance at the 1969 Harlem Cultural Festival was featured in the award-winning documentary film Summer of Soul, directed by Questlove.