Goals by centre-back Robert Andrich, his second of the night, and winger Diaby in a five-minute burst late in the game swung the result for hosts Leverkusen.
While the Germans are guaranteed a last 16 place, the away defeat means Celtic will finish in third spot after second-placed Real Betis won 2-0 at Ferencvaros Budapest.
Andrich's early header for Leverkusen was cancelled out when Josip Juranovic equalised with a first-half penalty before Jota was left unmarked in the area to fire Celtic ahead for the first time just after the break.
Leverkusen centre-back Andrich equalised for the hosts eight minutes from time before Diaby's winner.
The result preserves Leverkusen's record of having never lost at home in the Europa League group stages, while Celtic must wait for a first win on German soil in European competition.
Leverkusen struck early when Andrich outleapt his marker Stephen Welsh at a corner and powered his header past Celtic's former England goalkeeper Joe Hart.
Celtic started applying pressure.
A great chance went begging when Japan striker Kyogo Furuhashi got in behind the defence and his speculative shot to the far post flew just wide of Jota.
The Scottish giants drew level five minutes from the break when Juranovic chipped in from the penalty spot after home goalkeeper Lukas Hradecky fouled Furuhashi went he went to punch clear a cross.
Leverkusen made a strong finish to the first-half as Florian Wirtz and Moussa Diaby both hit the woodwork, but it stayed 1-1 at the break.
Hart came to Celtic's rescue early in the second-half when he scrambled back to deny Jeremie Frimpong moments after blocking a Diaby shot.
At the other end, Jota, who is on loan from Benfica, fired Celtic in front when Furuhashi spotted the Portuguese unmarked at the edge of the area to blast his shot past Hradecky.
Leverkusen roared back as Andrich beat Hart for the second time five minutes before Diaby smashed in the winner after being picked out unmarked by a Florian Wirtz pass.
Standout matches today