The experienced midfielder found the net from the spot just before the break for the 'Reds' to reduce the arrears, but it proved only to be a consolation as the hosts held on to claim a vital win.
Remarkably, that ended a 39-game run when Milner had not been on the losing side when getting on the scoresheet.
That record goes back all the way to February 2010 and includes 41 goals for Liverpool, Manchester City, Aston Villa and England.
Milner was just 24 the last time he was both on the scoresheet and the losing side, netting for Villa as they lost to Manchester United in the 2010 League Cup final.
Now 32, Milner will be hoping for another lengthy run as he moves into the twilight of his career.