Barcelona centre-back Samuel Umtiti was penalised for a rash lunge on Quintero's fellow substitute Jose Izquierdo as the South Americans received a morale boost ahead of the World Cup in Russia.
"The Colombians gave us a lesson in aggressiveness, whereas we were too quick to settle for what we had," France coach Didier Deschamps told 'TMC television'. "The top level is demanding and we didn't respond in the second half."
It was a remarkable turnaround after the hosts had got off to a flying start at the Stade de France.
They deservedly took the lead on 11 minutes and it was almost entirely down to a horrendous gaffe by Arsenal reserve goalkeeper David Ospina.
Lucas Digne timed his run down the left flank perfectly to gallop onto Thomas Lemar's inch-perfect through ball but the full-back's cross on the stretch was weak and straight at Ospina.
The Arsenal stopper inexplicably fumbled it, though, allowing Giroud to thrash a rising shot into the roof of the net as he fell backwards to score his 30th goal for France and open the scoring.
Lemar then doubled the lead with a beautifully-worked team goal for a vibrant 'Les Blues' but Luis Muriel gave the South Americans hope following a mistake by goalkeeper Hugo Lloris.
Muriel's inswinging cross from the left looked to be heading harmlessly into Lloris's arms. But the Tottenham goalkeeper was put off by Davinson Sanchez's attempts to divert the cross home from inside the six-yard box and dived the wrong way as the ball plopped almost apologetically past him.
Colombia soon equalised as N'Golo Kante lost possession in midfield and James Rodriguez scampered clear down the right before pulling back a cross for Radamel Falcao to convert from 15 yards.
Quintero then had the last word, just two minutes after entering the fray, and it would prove decisive for the visitors who go on to play Australia on Tuesday.
As for France, they will be hoping to pick themselves up against Russia on the same day, as the World Cup hosts fell 3-0 to Brazil in Moscow.