Mikel Arteta's side went in front through Granit Xhaka's first-half strike at St Mary's. But Armstrong levelled after the interval as Arsenal paid the price for failing to convert several chances to increase their advantage.
It was a rare slip from the Gunners, who had won nine of their 10 previous league games this season, with their only defeat coming at Manchester United. The end of Arsenal's eight-game winning run in all competitions leaves them two points clear of second-placed Manchester City with both having played 11 games.
While Arteta will be frustrated that Arsenal squandered the lead given to them by Swiss midfielder Xhaka, it is a tribute to his team's unexpected rise this season that failing to take maximum points ranks as a disappointment.
Arsenal have already proved they are a force to be reckoned with as they chase a first title since 2004 and they started in vibrant mood on the south coast. Bukayo Saka's cross was inches away from Gabriel Magalhaes before Gabriel Martinelli caused havoc in the Southampton defence with a direct run that ended with Xhaka's powerful drive repelled by Gavin Bazunu.
Xhaka turned provider moments later when he teed up Martin Odegaard for a low strike that whistled wide from the edge of the area. Arsenal were in complete control and Xhaka delivered the goal their pressure merited in the 11th minute.
It was a smooth move as Saka's backheel sent Ben White away down the right flank and his cross reached Xhaka, who crashed a blistering strike into the roof of the net from 12 yards. Arsenal had never lost when Xhaka scored, a sequence stretching to 17 games.
Gabriel Jesus was inches away from doubling Arsenal's lead when he shot into the side-netting from Odegaard's pass. Bazunu left Jesus frustrated again just before half-time as the Brazilian reached Odegaard's lofted pass for a volley that the Saints keeper blocked at point-blank range.
Mohamed Elyounoussi's last-ditch tackle stopped Jesus in his tracks after the Arsenal striker sprinted clear on goal. But, for all their territorial dominance, Arteta's team couldn't land the knockout blow and they were hit with a sucker punch in the 65th minute.
Elyounoussi slipped a clever reverse pass through the heart of the Arsenal defence and Armstrong timed his run perfectly to slot a composed finish past Aaron Ramsdale. With Southampton suddenly in the ascendancy, Joe Aribo tested Ramsdale with a low shot and Romain Perraud swivelled to drill wide.
Odegaard thought he had won it in the 79th minute when the Dane smashed home from Kieran Tierney's cross but the goal was disallowed because the ball had run out of play.