FA Cup Round-up: Spurs ease through as Newport stun Leeds
Elsewhere, Tottenham scored three goals in just eight second-half minutes to see off the challenge of League One side AFC Wimbledon while Arsenal took on Nottingham Forest.
In the early kick-off, on-loan forward Shawn McCoulsky headed in a dramatic 89th-minute winner for Welsh club Newport to seal a 2-1 victory against 1972 Cup winners Leeds and send the crowd at Rodney Parade wild.
Leeds had taken an early lead in the third-round tie when Gaetano Berardi's deflected long-range shot found the net.
But Newport, currently 11th in League Two -- the fourth tier of the English game -- levelled 14 minutes from time when Leeds defender Conor Shaughnessy sliced into his own net.
Infused with fresh belief in front of a passionate Welsh crowd, Newport pressed for a winner, and with a minute to go on-loan substitute McCoulsky rose highest at a corner to head in the decisive goal and send the stadium into raptures.
Leeds were then reduced to 10 men when substitute Samuel Saiz was immediately sent off over an off-the-ball incident.
"This is much better beating Championship teams like Leeds," said Newport manager Michael Flynn. "I don't want to be winning relegation battles, I want to be winning FA Cup ties and climbing the table.
"I thought we were the better team in the first half, the goal was unfortunate, but we had chances and kept going. I told them we don't have to win the game in the first 10 minutes of the second half and we got crosses into the box.
"It was an outstanding header. He has 11 goals for us this season and he has been superb for us. He was hungry and wanted it and I am delighted."
The Championship visitors had made nine changes from the side held by Nottingham Forest on New Year's Day, and manager Thomas Christiansen's decision backfired in a frenzied finale.
It is another miserable result in the FA Cup for Leeds, who crashed out in the fourth round last year to non-league Sutton United.
- Hart saves -
West Ham had goalkeeper Joe Hart to thank for keeping them in the Cup with a 0-0 draw after an uninspired performance against Shrewsbury Town of the third tier.
The England goalkeeper, returning to the club where he started his career, was handed a cap from the crowd as he battled against the slanting winter sun.
He made a fine save from Mat Sadler with about 10 minutes to go until the interval and repeated the trick against Alex Rodman to keep the score at 0-0.
The League One high-fliers continued to take the game to David Moyes's team in the second half but there were few clear-cut opportunities for either side.
Shrewsbury's Jon Nolan had a golden chance to seal a dramatic victory in the dying minutes of normal time but blazed over and the two sides will meet again at the London Stadium.
"The players are a little bit deflated almost," said Shrewsbury Town manager Paul Hurst.
"But we have a replay to look forward to," he added. "We know it will be tough. They will be at home, more comfortable in their own stadium."
At Wembley, Harry Kane broke the deadlock against AFC Wimbledon with just under half an hour to go before Kane again and Jan Vertonghen made it 3-0 to put the game beyond the reach of the visitors.
On Saturday, League Two Coventry City sent Premier League side Stoke City crashing out, prompting the sacking of Mark Hughes.