Serbia released a statement that read: "We have submitted seven video-recordings clearly showing Brych's tendency to make decisions against our national team.
"The footage reveals Brych's double standards in brandishing yellow cards, as he was quick to book our key players while he didn't do the same in similar situations at the other end."
The main complaint was the decision not to award Serbia a penalty after Aleksandar Mitrovic was impeded, an incident in which Brych did not consult VAR.
"The most glaring poor decision was not awarding Serbia a penalty for a foul on Mitrovic with the score 1-1, and awarding the spot-kick could have turned the match our way," added the statement.
"The whole world saw the penalty except Brych and it should have gone to VAR analysis.
"We are wondering why Serbia is the only team in the World Cup where the match official chose to ignore controversial situations at it raises the legitimate question whether the VAR is being used selectively."
Xherdan Shaqiri scored a late winner to secure a 2-1 victory for Switzerland which means Serbia face the daunting task of defeating Brazil to qualify for the Round of 16.