Gray started playing rugby at Cambuslang RFC where he progressed from the mini section (at 7 years old) through to first XV regular at the age of 16. He also played for Calderglen High School in East Kilbride.[3]
In 2013, Gray was offered a full-time contract for three years by Glasgow Warriors[4] Domestically, the 2014–15 season saw Gray play a prominent role in Glasgow Warriors' maiden Pro12 title triumph.
At just 21 years of age, Gray was named captain of Glasgow Warriors in November 2015, taking over from recently retired mentor Alastair Kellock.[6] He became a co-captain with Henry Pyrgos for the 2016-17 season but following the arrival of Dave Rennie as Warriors Head Coach the club has reverted to a sole captaincy. For the 2017-18 season Rennie named Ryan Wilson as captain though Gray remains in the leadership group.[7]
In December 2017, it was announced that, after rejecting an offer from Bristol Rugby, Gray had extended his contract at the Warriors until May 2020.[8] It was announced in January 2020 that Gray would join Exeter Chiefs on a two-year contract ahead of the 2020–21 season, joining up with his Scotland teammates Stuart Hogg, Sam Skinner and Sam Hidalgo-Clyne.[9]
On 4 July 2024, Gray would sign for French side Bordeaux, after his early departure from Exeter, in the Top 14 competition from the 2024-25 season.[10]
Gray played in every match for Scotland during the 2015 6 Nations, attempting an astonishing 85 tackles in total and missing only one.[12] Subsequently, he was selected in Scotland's 31-man squad for the 2015 Rugby World Cup, playing four of their five matches in the run to the Quarter-Finals.