The 21-year-old converted a penalty in the 31st minute before Mike Havenaar and Davy Propper scored after the visitors had leveled in the 63rd minute.
The win took Peter Bosz’s men to the summit of the table temporarily since AZ Alkmaar will overtake them if they beat Feyenoord on Sunday.
Atsu who won the MOTM award on Saturday night is on loan at Vitesse from Chelsea.
His Dutch side have not lost a match since he made his debut against Heerenveen last month.
His club form should be good news for Ghana coach Kwesi Appiah as he leads his team to Cairo on November 19 to play Egypt in the final WC qualifying play-off.
Ghana won the first leg 6-1 in Kumasi, with Atsu being one of the scorers.