In an action-packed last 11 minutes, Kostic gave Frankfurt the lead before Egyptian striker Omar Marmoush levelled for Stuttgart, who had defender Waldemar Anton shown a straight red card.
"Filip brought lots of momentum when he came on - that's how we know him," Frankfurt captain Martin Hinteregger told DAZN.
Kostic, 28, skipped Frankfurt training before the transfer window closed a fortnight ago in order to force a move to Serie A side Lazio. After being left out of the squad for Frankfurt's previous league game, head coach Oliver Glasner brought Kostic on for the last half an hour.
The Serbia winger roared with delight after his superb shot flew into the net via the post to put the hosts ahead on 79 minutes, as he celebrated his first goal this season. Stuttgart centre-back Anton was then sent off for bringing down Frankfurt's Rafael Borre, who was through on goal.
The visitors equalised when Marmoush, on loan from Wolfsburg, fired into an empty net at the second attempt two minutes from time.
Frankfurt came within a whisker of grabbing their first win this season when Borre's shot smacked the underside of the crossbar and bounced away from goal.