Williams only found himself on the pitch after Richard Garcia was forced off the field with a hamstring injury but he completed a remarkable but fully deserved comeback for the visitors after Neil Danns' fourth goal in six games cancelled out Steve Kabba's early strike.
By resting half his outfield line-up, Blades boss Neil Warnock gave his in-form opponents real hope of an upset.
The confidence gained from 12 wins and only one defeat in their last 15 games saw Parkinson try to match his opponents man for man.
It was a risky strategy and one undone inside five minutes as Michael Tonge cut inside onto his left foot, and unleashed a shot Aidan Davison could only parry straight to Paul Ifill.
He smashed the rebound against the crossbar, but again it fell kindly, this time to Kabba, who kept his cool to net his 10th goal of the season and celebrate signing his new three-year contract in style.
The home side were denied a penalty when Karl Duguid tripped Kabba, referee Tony Leake deeming the full-back's tackle clean.
Colchester were close to an equaliser when Mark Yeates' shot was deflected to Richard Garcia, but the striker was unable to lob Barnes from close range.
Kabba thought he had his second moments later but he netted following a handball by Lilian Nalis, while Chris Iwelumo - who had scored in both the previous rounds - spurned a great chance, volleying Greg Halford's headed flick wide from six yards.
But the visitors soon had the equaliser their pressure merited, the Blades' inability to clear their lines presenting the ball to Danns who swept home his eighth of the season from close range.
It almost got worse when Wayne Brown blazed over following more uncertain defending, which continued right up to the half-time whistle as Colchester went in genuinely unfortunate not to be ahead.
A half-time downpour may have benefited their cause, however, with the playing surface visibly cutting up underfoot.
They survived another penalty claim when Nalis went down too easily for Leake's liking and lost Garcia to what looked a hamstring strain soon after, with Williams the replacement.
It did not seem to faze them, with the home side needing someone to take the game by the scruff of the neck.
Tonge was the first to put his hand up, a brilliant 50-yard run ending with a shot which Davison palmed just beyond the reach of the lurking Ifill.
Warnock introduced youngster Jonathan Forte in an attempt to crank up the pressure and even gave a long-awaited debut to Chinese veteran Hao Haidong.
But the move backfired when Forte was briefly forced to leave the field after suffering a blow to the face and Colchester took full advantage.
Danns forced a corner, which was passed to Yeates, whose whipped cross was glanced home by the substitute Williams, his first goal since January last year.
The Blades were looking a shadow of the side who took eventual winners Arsenal all the way to penalties last season, and Iwelumo almost put the tie out of their reach when he flashed another volley wide.
Armstrong came close to a late equaliser with a free-kick well held by Davison, who was looking to put one over the club he made two appearances for six years ago.
And he and his team-mates comfortably saw out the closing moments to become the first lower league side to knock the Blades out of the FA Cup since then Third Division Derby in 1986.