United had been poor all afternoon, but as the game moved into injury time, Morgan popped up to head Phil Jagielka's cross back across the face of goal and into the net after good work from Michael Tonge.
United had gone ahead against the run of play in the first half through Nick Montgomery, but Stoke looked worthy of a point after Mamady Sidibe had levelled early in the second half.
After a scrappy start, Luke Chadwick created the first chance for Stoke in the fifth minute, riding two challenges to cross for Paul Gallagher. His shot was charged down before Sambegou Bangoura saw his follow-up pushed round the post by Paddy Kenny. Chadwick then made another incisive run, cutting in from the right to test Kenny with a low bobbling shot which the United keeper could only parry.
But for all of Stoke's early pressure it was United who took the lead from a free-kick in the 18th minute. Stoke failed to clear David Unsworth's cross, allowing Morgan to play the ball back out to the edge of the area, where Montgomery picked his spot and hit a low shot through the crowd and into the bottom left-hand corner.
The goal brought United to life and Alan Quinn had already fired wide before Steve Kabba broke the offside trap to reach Danny Webber's pass, forcing Stoke goalkeeper Steve Simonsen off his line to block the shot. Despite taking the lead, United continued to look sloppy, making poor decisions in the final third and mis-placing several passes.
Things improved briefly in the second half when United were quickly out of the blocks, and Jagielka saw two shots charged down - one by Russell and a second by Dave Brammer - in the opening three minutes.
Unsworth then picked up his eighth yellow card of the season for a foul on Bangoura, prompting Neil Warnock to replace him with Rob Kozluk in the 55th minute.
But Stoke were level moments later. Bangoura cut in from the right to set up Chadwick. His shot was charged down, but the ball fell for Sidibe who rounded Kenny for a simple finish.
The visitors then had a penalty shout waved away in the 63rd minute as they continued to pressure United. Sidibe went down under the challenge of Leigh Bromby and despite a flag from the linesman, referee Lee Mason waved it away.
In the 69th minute, Neil Shipperley squandered a good chance for the hosts, blazing over after Quinn cut the ball back to find him unmarked on the edge of the area.
The home fans continued to grow frustrated as United wasted possession, and the hosts were almost punished in the 75th minute when Chadwick twice cut in to try his luck from the edge of the area, first firing over before Kenny got fingertips to the second.
With 10 minutes left, Shipperley prodded the ball through for Webber to set up Quinn, but his close-range shot was blocked by the sprawling Simonsen.
Stoke almost grabbed a late winner when Russell broke into the box in the 88th minute, but Kenny turned his effort over the bar with good save before Morgan snatched victory.