It was the Brazil midfielder's second goal for Arsenal — he scored against Sunderland last month — after joining from Newcastle in pre-season, and he celebrated by diving into the crowd.
It was the perfect way for Arsenal to celebrate coach Mikel Arteta signing a contract extension this week and bounce back from a hefty defeat at Brighton before the extended three-week break for internationals.
"This is the level for Arsenal, every week, every game, every minute, every action that you have to be at," Arteta said.
"Against Brighton (Guimaraes) wasn't at that level, and today he was much better, and he can still be much better, and that's what we have to demand."
Arsenal moved level on points with Manchester City at the top of the table ahead of City's tricky trip to Liverpool on Sunday.
It was a first defeat for Leeds, which was hoping to leapfrog Arsenal into second — courtesy of a better goal difference — with a win.
James Justin fired Leeds in front in the 55th minute but Riccardo Calafiori levelled just seven minutes later and then Declan Rice teed up Guimaraes to drive home his winner.
In the day's late match, Arsenal's winless London neighbours Tottenham snatched a deserved 1-1 draw at Manchester United despite with 10 men.
Rodrigo Bentancur was sent off in the 78th minute for headbutting Luke Shaw when they were already 1-0 down after Bryan Mbeumo broke the deadlock.
But Spurs levelled in stoppage time as terrible defending allowed Dominic Solanke to nod home Sandro Tonali's cross.
"The situation we were in with the red card ... we need this situation, we have not accepted the defeat," Tottenham coach Roberto de Zerbi told Sky Sports.
Jordan Henderson became Chelsea's oldest English goal-scorer as the 36-year-old netted a brace in a 5-1 win over Bournemouth.
Henderson scored after just 66 seconds and turned provider almost immediately, setting up Morgan Rogers for a deflected shot that put Chelsea 2-0 up after just 149 seconds.
Joao Pedro added another in the opening minute of the second half and Evanilson grabbed one back for Bournemouth.
But Pedro settled any nerves in Stamford Bridge with Chelsea's fourth before Henderson also doubled his tally with a superb free kick.
Brighton consolidated third place with a 2-0 win at Sunderland, despite playing most of the second half with 10 men after Pascal Struijk was shown a straight red card for denying an obvious goal-scoring opportunity.
Maxim De Cuyper scored moments before Struijk's dismissal and Jaouen Hadjam sealed the match in stoppage time.
Fulham appeared set for their first win in the Premier League this season but two very late goals — from Abdoul Ouattara and Dara O'Shea — saw Ipswich snatch a 2-1 victory.
Brentford remained unbeaten after drawing at Aston Villa 2-2.
24°C