The Blades twice came from behind after Craig Morgan and Chris Llewellyn had scored for the home side but Gray's brace either side of Phil Jagielka's strike gave the visitors some much needed cheer after three straight league defeats.
The visitors threatened almost straight from kick-off when Tommy Black came close to playing Gray in but the Blades then had to withstand a lengthy spell of pressure from the home side, who created several good chances in the opening stages.
Hector Sam lobbed just over from Darren Ferguson's chip pass and then turned provider for Llewellyn, who shot low against the side netting.
And Wrexham, who have not reached the third round of this competition since 1981, took the lead after just five minutes, defender Morgan powerfully heading past Blades goalkeeper Paddy Kenny from Ferguson's corner.
The opening goal seemed to cool the home side's attacking ambition and midway through the first half, United, who were semi-finalists in this competition two years ago, were firmly on top.
Jonathan Forte was a constant menace and one break down the left found space for Gray, who looped a shot into the corner of the Wrexham goal with the aid of a deflection.
Neil Warnock's side looked to be running away with it, with Chris Morgan twice hitting the woodwork from Leigh Bromby crosses.
But on the stroke of half time Denis Smith's Wrexham regained the lead. A United corner was cleared by Andy Holt with Dean Bennett seizing the opportunity to break quickly down field.
The right-wing back's 60-yard run saw him slide the ball through to Llewellyn who stabbed the ball under Kenny despite a poor first touch.
But the visitors, whose league form has dipped alarmingly in recent weeks, showed no lack of confidence in getting themselves back into the match nine minutes after the interval, with Phil Jagielka rising to head Black's free kick firmly into the top corner of Matt Baker's goal.
And the evening got worse for Baker, who was making his first Wrexham start in the place of Andy Dibble, as his defensive mix up with the veteran Brian Carey saw United take the lead on 67 minutes.
A hopeful United through-ball saw the two Dragon's players leave it to each other and the grateful recipient, Alan Quinn, cut in from the left only to see his shot kept out by Craig Morgan's hand. However Gray made no mistake from close range to give his side an unassailable advantage.