Born in New Orleans, Louisiana, Dorsey was a childhood friend of Fats Domino. He moved to Portland, Oregon when he was ten years old.[3] He served in the United States Navy in World War II and began a career in prizefighting. Boxing as a featherweight in Portland in the early 1950s, he fought under the name Kid Chocolate and was not successful, fighting only one time and being knocked out in the second round.[4] He returned to New Orleans in 1955, where he opened an auto repair business as well as singing in clubs at night.[2]
His first recording was "Rock Pretty Baby/Lonely Evening" on Cosimo Mattasa's Rex label, in 1958. This was followed by "Lottie Mo/Lover of Love", for the small Valiant label in late 1960 (picked up by ABC Paramount in 1961).[5] These efforts were unsuccessful, but around 1960 he was discovered by A&R man Marshall Sehorn, who secured him a contract with Fury Records, owned by Bobby Robinson.[6] After meeting songwriter and record producer Allen Toussaint at a party,[7] he recorded "Ya Ya", a song inspired by a group of children chanting nursery rhymes.[2] It went to number seven on the Billboard Hot 100 in 1961, sold over one million copies, and was awarded a gold disc.[8] Although the follow-up "Do-Re-Mi" also made the charts, later releases on Fury were not successful. Dorsey returned to running his repair business,[2] but also released singles on the Smash and Constellation labels in 1963 and 1964.[5]
He was approached again by Toussaint, and recorded Toussaint's song "Ride Your Pony" for the Amy label, a subsidiary of Bell Records. The song reached No. 7 on the R&B chart in late 1965, and he followed it up with "Get Out of My Life, Woman", "Working in the Coal Mine" – his biggest pop hit – and "Holy Cow", all of which made the pop charts in both the US and the UK. Dorsey toured internationally, and also recorded an album with Toussaint, The New Lee Dorsey in 1966.[2] In 1970 Dorsey and Toussaint collaborated on the album Yes We Can; the title song was Dorsey's last entry in the US singles chart. It was later a hit for the Pointer Sisters under the title, "Yes We Can Can". With declining sales, Dorsey returned to his auto repair business.[6]