Goals from Aaron Mooy and Laurent Depoitre steered the 'Terriers' to a shock win at the John Smith's Stadium, despite a late goal from Marcus Rashford and whilst he knew it would be a test, Neville says that the result amazed him.
Speaking on 'The Gary Neville Podcast', the former United and England defender said: "I thought it would be tough for United because of the atmosphere but I also thought they'd come through it. I was stunned by the result.
"I was stunned also by Jose Mourinho's comments afterwards about the application of the players, saying it was the worst he's seen them up for the game.
"It was a big shock and bad goals to concede. Lindelof made a couple of poor mistakes. I think if the other team does something brilliant, you can recover from it but you don't give yourself a chance if you're making poor mistakes.
"When your goalkeeper, defenders are making poor mistakes, the anxiety runs not only into the team but into the crowd. It's a bad loss."
The former Valencia manager went on to stress the importance of getting back to winning ways when they face Spurs in the early kick off on Saturday, though he admits that it will be no mean feat.
"Saturday is now a huge game for United to get back on their bike after falling off," he explained.
"They're playing against a Tottenham side with great confidence.
"The one thing I would say is that probably the worst time I've seen Mauricio Pochettino's team was at Old Trafford last season so it will be interesting to see them, having gone to the Bernabeu and changed their fortunes around at Wembley.
"If Tottenham play anything like they did today, Manchester United will have to be at their best - they can't be below."