Josh Hobbs returns to his ‘Measuring Transfer Success Through Minutes Played’ article to see how TransferLab’s Goal Difference Added metric can clarify the findings.
Last month, I published an article with Analytics FC looking at the idea of measuring transfer success by minutes played. The piece was inspired by hearing Liverpool’s Ian Graham give a talk at Statsbomb conference which considered the success of ‘big transfers’ (£10m or more) in the Premier League in terms of the percentage of minutes players managed after signing for their new clubs. The idea for my piece was that this could offer a heuristic for how successful last season’s ‘big transfers’ in the Premier League had been with some cases being extreme enough to suggest that a transfer could be considered an overall success or failure even after just one season.
This brought on another line of thinking: how else can we judge transfer success? Of course, minutes played and the percentage of minutes played can give a strong idea. But are there other ways?
One metric which we can look at is TransferLab’s ‘Goal Difference Added’ metric. Using our unique TransferLab algorithm–a Markov Chain model–we are able to take events that happen on the football pitch and ascribe them a numerical value which indicates how much an action improved that possession’s probability of ending in a goal (and just as importantly, how much it reduced the opposition team’s chance of scoring on the next possession). The algorithm calculates every action and can therefore determine the overall impact a player has through all their actions. The impact is presented to clubs in terms of ‘Goal Difference Added’ per 90.
I thought it would be interesting to attempt to use the same transfers from the last article and see whether their GD Added last season would make me reassess any of my thoughts which were previously based only on a percentage of minutes played and my own personal assessment of the performance of the players.
Here are the top ten performers for GD Added per 90 out of all 50 transfers from the Premier League last summer which came in at over £10m:
On the face of it, a couple of things stand out: firstly, in the original article, I mentioned players like Diogo Jota and Ferran Torres who it seemed wrong to rule as ‘unsuccessful’ given their contributions on the pitch. Here we can see that they were the 6th and 7th most valuable contributors in terms of GD added per 90 according to TransferLab. Although this list might not feel like the ‘most successful’ transfers from out of our sample, it does back up the feeling that it would be incorrect to put those players in the unsuccessful pile.
Then there are Rodrigo, Donny Van de Beek, Hakim Ziyech, Alex Telles and Robin Koch who also make the top ten despite paying less than 50% of the available minutes. This suggests that perhaps they haven’t played enough minutes for the metric to be truly reliable. In the case of Van de Beek, Ziyech and Telles, if their impact on teams was really that good, then surely they would be playing more minutes?
When it comes to Rodrigo and Koch, the two played around 40% of the available minutes each so the sample size isn’t tiny. But it is still likely they wouldn’t have such high performance in the metric if they had played a greater amount of minutes. That said, it does perhaps suggest that Leeds were unfortunate in the way their signings worked out last season, as both improved the team when they played, according to GD Added.
It should be noted though: Rodrigo has started all but one of Leeds’ games this season and currently is putting up 0.1 GD Added per 90, compared to 0.27 from last season. This suggests that something else might be at play in his very high performance from 20/21: perhaps a few substitute appearances in the latter part of the season where he came on with Leeds in the lead and the gamestate allowed him to put up inflated attacking numbers.
With all that in mind, here’s the same graph, only this time with those who didn’t play at least 50% of the available minutes filtered out:
This one feels a lot more like the list of transfers we might consider to be ‘successful’ compared to the previous list. There is a decent spread of attackers, midfielders and defenders and the majority of these players had big impacts on their respective teams.
However, it’s notable that there are no goalkeepers on the list. In metrics like this, it can be the case that some defensive players are undervalued because event data doesn’t capture everything they do which goes towards making their team better in a defensive sense. On top of this, attacking actions that directly contribute to scoring chances naturally score higher than defensive actions.
In the case of goalkeepers, they are primarily looking to stop goals and their actions in possession take place so far away from the opposition goal that they are unlikely to contribute much positively toward xG being generated for their own side. In that case, we can’t expect them to perform as well as forwards, but that doesn’t mean transfers like Emi Martinez to Aston Villa or Edouard Mendy to Chelsea aren’t successful.
To take this a little further, here is the list of the top performers when sorting for their percentile rank per position rather than just the raw value of GD Added per 90:
Interestingly, using this method bumped the Football Writers’ Player of the Year, Ruben Dias off the list of top performers as he ranked only in the 75th percentile for central defenders in the Premier League for GD Added. This put him 11th best performer in the data set.
This method did bring Aaron Ramsdale and Emi Martinez to the fore, which feels right as they certainly were very successful transfers and top performers in their positions. Meanwhile, Raphinha drops from the top performer to the 5th best as he ranked in the 88th percentile for wingers.
It could be queried how Gabriel ranked so highly, sitting in the 97th percentile for centre backs, but it’s clear that Arsenal were vastly improved when he played.
Reassessing Leeds United
In the last article, I focused briefly on Leeds as they stood out as spending a large amount on players with the majority of the money going on four players who didn’t manage to cross the threshold of 50% minutes. Leeds’ season was effectively carried by an outstanding transfer in Raphinha and a group of players stepping up into the Premier League who greatly overperformed their previous levels.
However, if we look at the way the players performed compared to their positional peers, it does reinforce the view that whilst the signings didn’t play the amount of minutes Victor Orta and Marcelo Bielsa would have liked, they were strong performers when they did play:
Considering the fact that Robin Koch and Diego Llorente both played predominantly as the right-sided centre back and they were rarely fit at the same time, between the two of them, Leeds had a strong player to call on in the position for the majority of the season.
The percentile rankings, coupled with the percentage of minutes played analysis in the previous article also emphasise how excellently the signing of Raphinha worked out for the Whites. Not only was he one of the best wingers in the league in terms of GD Added per 90, but also Leeds were able to call on him for over three-quarters of the season. This was crucial in their top ten finish in the league.
Although GD Added does show that Leeds signed good players in 20/21, questions remain around whether enough weight was put on the players’ injury records. Regardless of how well these new signings might have played, there is a sense that Leeds were lucky to have their existing players step up so well or they may well have struggled in their first season in the Premier League.
Interestingly, Leeds are suffering badly from injuries this season. Robin Koch has only played in the first game of the season and recently underwent undergoing surgery to correct a pelvic issue, and Diego Llorente has missed several games through recurring hamstring trouble. The rest of the team isn’t performing to the same levels as they did in 20/21 and Leeds subsequently find themselves 17th in the league at time of writing.
Looking Back at Aston Villa
In the previous article, I used Aston Villa as a model of success as all but one of their five ‘big transfers’ of 20/21 ended up playing well over 50% of the minutes available and were key players in Villa’s vastly improved season.
But simply looking at the percentile rankings of the players according to GD Added doesn’t quite tell the whole story:
Without the context of minutes played, you might presume that Sanson had been successful on this evidence. But his contribution in terms of minutes was so small that we can disregard his result here.
Matty Cash and Bertrand Traore both came in as a good amount above average for their position, whilst Ollie Watkins and Martinez featured in the top performers for percentile rankings earlier. When you consider that all these four players played for the majority of the time and they had a positive contribution on the field according to GD Added, these transfers look even better.
Another way of assessing transfer success is by comparing new players with previous players. To assess how good Villa’s transfers were last season, I looked at the players they replaced to get a sense of how much they improved Dean Smith’s team. I did this by taking the players in questions’ GD Added from the 19/20 season.
First, there was Tom Heaton, who was the first-choice goalkeeper in 19/20. His GD added was -0.3, which put him in the 69th percentile for Premier League goalkeepers. Martinez’s -0.05 and 93rd percentile ranking was a huge upgrade.
Villa’s main right back in 19/20 was Frederic Guilbert. He played 2041 minutes in the competition and added 0.06 GD per 90 according to TransferLab, ranking him 62nd percentile for fullbacks. Again, the signing of Cash was a big upgrade as he added 0.1 GD per 90 and ranked in the 70th percentile for fullbacks.
On the right-wing, Anwar El-Ghazi was first choice in 19/20, with his 0.06 GD Added per 90 only enough to rank him in the 51st percentile for wingers. Bertrand Traore’s 0.08 might seem like only a small upgrade but this ranked him in the 71st percentile for the position.
Finally, Wesley Moraes had been the main striker for the Villians in 19/20, although he did have his injury issues. His 0.03 GD added and percentile ranking of 52 was hugely out-performed by Watkins in 20/21 as the ex-Brentford striker added 0.13 GD per 90 and ranked in the 83rd percentile for strikers.
This confirms the view that Villa had a superb summer window in 20/21. Not only did they sign players that were able to contribute for the majority of the minutes on hand, they signed players significantly better than the predecessors in their positions.
A Little More Context For Bottom Three Failures
The previous article also featured a section that showed how the three relegated teams failed with almost all of their transfers over £10m in 20/21. Aaron Ramsdale was the exception to this rule but Grady Diangana, Karlan Grant, Rhian Brewster and Anthony Knockaert all made negligible impacts on their sides.
Looking at the bottom ten performers for percentile ranking of GD added, we can see why they weren’t given more minutes:
As you can see, Knockaert was deemed not good enough without even getting any Premier League minutes. Notice though how Brewster, Grant and Diangana simply weren’t performing well enough to force their managers to give them more minutes.
A Note on Edouard Mendy
One player who stood out in terms of his GD Added score from 20/21 was Edouard Mendy.
The Senegalese goalkeeper moved from Stade Rennais to Chelsea for a fee of just over £20m and came in to replace the struggling Kepa Arrizabalaga as number one. He played the vast majority of the games after signing, but according to his percentile ranking for GD Added, he was only in the 48th percentile for goalkeepers. Interestingly though, this season he is in the 93rd percentile for goalkeepers as his performances have been excellent.
As you can see from his TransferLab profile, he is outstanding for saves, parries and aerial wins:
It’s clear then, Mendy has come in and taken the vast majority of minutes since signing, and whilst he had a slow start to his career at Stamford Bridge in terms of making a positive contribution on the field, he is now key in helping his team win points. This just adds more evidence to the idea that this is considered a successful signing.
Conclusion: Gather As Much Information As Possible
Looking at the transfers that were previously assessed through only the percentage of minutes played, it’s clear that considering the GD Added metric is helpful for gaining increased information so as to properly judge the success of a transfer.
It would perhaps be important to also judge players specifically by the role they are expected to perform for their side. For example, if a player is signed to be a backup for their team’s first-choice starting eleven, it can’t be expected that they hit 50% of the minutes available. However, using GD Added, it’s possible to assess how effective the player has been on the field when they have been called upon.
Alongside this, it’s also helpful to assess key metrics needed for the player’s role in their team. For instance, a player might get a lot of chances in the team but fail to deliver on a certain aspect of what’s necessary for that player to be truly effective in the role.
At the most basic level, this could be an attacking midfielder who plays regularly for their team putting up an expected assists figure which puts them in the 20th percentile for the position, despite performing strongly in some other areas. If the player was brought in to increase creativity, this metric would clearly help show where something had gone wrong with how the signing had worked out. TransferLab profiles are obviously an example of how to use data to quantify transfer success.
As such, a combination of these three aspects of minutes played, GD Added and Key Performance Indicators for particular roles seems a more robust way of assessing the success of a transfer. Of course, this would be judged over the length of a player’s contract though, rather than the one year as we have focused on in these articles.
Header image copyright IMAGO / Sebastian Frej