RunRepeat, in partnership with The Professional Footballers' Association, has analyzed 2,073 statements from commentators in 80 soccer matches from the 2019/20 season. This is the first time a study of this kind has been conducted to understand if there is a difference in the way the soccer media talks about players of different skin color.
The comments discussed 643 unique players of various races and skin tones.
Key findings
The findings show bias from commentators who praised players with lighter skin tone as more intelligent, as being of higher quality, and harder working than players with darker skin tone.
Players with darker skin tone were significantly more likely to be reduced to their physical characteristics or athletic abilities -- namely pace and power -- than players with lighter skin tone players were.
In numbers:
- When commentators talk about intelligence:
- 62.60% of praise was aimed at players with lighter skin tone
- 63.33% of criticism was aimed at players with darker skin tone
- When commentators are talking about power they are 6.59 times more likely to be talking about a player with darker skin tone
- When commentators are talking about speed they are 3.38 times more likely to be talking about a player with darker skin tone
- When commentators talk about work ethic, 60.40% of praise is aimed at players with lighter skin tone
You may use material from this report as long as you refer to this original report. For special requests, or questions, email lead researcher Danny McLoughlin at danny@runrepeat.com.
Overview of methodology
- Selection of Games: Selected a total of 80 games from 4 of Europe’s top leagues (Italian Serie A, Spanish La Liga, French Ligue 1, and English Premier League) from the 2019/20 season.
- Transcription: Transcribed commentary from each game into a text document.
- Coding (Categorization of Comments): Recorded every adjectival phrase and manually assigned whether the comment was positive (praise) or negative (criticism), a category, and the associated player.
- Player Information and Identification of Race: Designated a player as “players with lighter skin tone” or “players with darker skin tone” based on skin tone attribute from Football Manager 2020 (an extensive database maintained by over 1,300 scouts).
- Analysis and Results: Analyzed the ratio of praise and criticism for each category to determine differences in how players of different skin tones are talked about.
For a full, detailed methodology including references and definition of categories please see the full methodology section at the bottom of this document.
What does bias look like?
If there was no bias in commentary, the distribution of comments towards players of different skin colors would be similar.
Players with lighter skin tone should receive the same proportion of comments about, for example, their intelligence or their work-ethic as players with darker skin tone.
The fact this is not the case across a large sample size indicates there is bias in the way the media discusses players based on the color of their skin.
Control category
The control category of “in-game events” was included to support the theory that there would be an even distribution of comments in the absence of bias.
There was less scope for bias in this category given it is “considered probably factual unless additional adjectives characterizing a player were included.” Eastman & Billings (2001).
Comments such as “That's a good ball in for Rafael at the back post.” or “But not the best ball from Toni Kroos.” were recorded as a positive or negative “In-game event.”
When commentators made purely factual statements (in-game events), both groups received the same proportion of comments and were praised or criticized the same amount.
The even distribution in this control category (where commentators are stating facts) is further indication that bias is present in the other categories (which are based on the opinion of the commentator) where the distribution and split of praise and criticism looks different.
Results
Table 1: Base Numbers
| Darker skin tone | Lighter skin tone | |||
| Sentiment | Negative | Positive | Negative | Positive |
| Quality | 13 | 50 | 12 | 162 |
| Appearance | 0 | 3 | 1 | 1 |
| Background | 8 | 28 | 17 | 66 |
| Form | 47 | 86 | 79 | 111 |
| Hard-Work | 3 | 21 | 8 | 61 |
| In-game event | 72 | 175 | 138 | 380 |
| Intelligence | 10 | 24 | 11 | 77 |
| Leadership + Mentality | 25 | 44 | 52 | 112 |
| Power | 0 | 31 | 0 | 9 |
| Speed | 0 | 53 | 11 | 19 |
| Versatility | 13 | 7 | 9 | 25 |
What was said about whom and how frequently?
Looking at the distribution of comments for each group allows us to look at how often commentators bring up a subject when talking about each group. For example, when talking about players with darker skin tone, they bring up form 18.65% of the time. When talking about players with lighter skin tone, they bring up leadership 12.05% of the time.
There is a significant difference in how often commentators discuss topics for our two groups.
Likelihood to comment on athletic abilities
These differences are most stark when commentators are discussing physical characteristics/athletic abilities -- speed and strength. Or put into a more recognizable football stereotype when referred to as “pace and power”.
Commentators are 6.59 times more likely to comment on the strength of players with darker skin tone than they are to comment on the strength of players with lighter skin tone.
Commentators are 3.38 times more likely to comment on the speed of players with darker skin tone than the speed of players with lighter skin tone.
Rada & Wulfemeyer (2005) stated:
“Portraying African Americans as naturally athletic or endowed with God-given athleticism exacerbates the stereotype by creating the impression of a lazy athlete, one who does not have to work at his craft.”
Likelihood to comment on in-game events
Returning to the control, the in-game event category shows that both groups of players are doing similar things on the pitch. So talking about them in different terms indicates a presence of bias.
Who was praised? Who was criticized? And for what?
This study comprises 643 unique players. 433 of these have been classified as “players with lighter skin tone” and 210 have been classified as “players with darker skin tone” based on the criteria listed in the methodology section at the bottom of the page.
In order to form an apples to apples comparison, we multiplied the number of comments about players with darker skin tone to equal the number of comments about players with lighter skin tone, allowing us to analyze who commentators criticize and praise per subject.
Ratio adjusted data
In the sample, there were a total of 2,073 statements. Of these, 1,361 were towards the group of players with lighter skin tone and 713 were towards the group of players with darker skin tone.
A multiplier of 1.9 was used resulting in 1,362 comments about players with darker skin tone and 1,361 comments about players with lighter skin tone:
Table 2: Ratio adjusted numbers
| Darker skin tone | Lighter skin tone | Darker skin tone | Lighter skin tone | |
| Sentiment | Negative | Negative | Positive | Positive |
| Quality | 25 | 12 | 96 | 162 |
| Appearance* | 0 | 1 | 6 | 1 |
| Background | 15 | 17 | 53 | 66 |
| Form | 90 | 79 | 164 | 111 |
| Hard-work | 6 | 8 | 40 | 61 |
| In-game event | 138 | 138 | 334 | 380 |
| Intelligence | 19 | 11 | 46 | 77 |
| Leadership + mentality | 48 | 52 | 84 | 112 |
| Power** | 0 | 0 | 59 | 9 |
| Speed** | 0 | 11 | 101 | 19 |
| Versatility | 25 | 9 | 13 | 24 |
*The sample size of the Appearance category is too small to make any definitive conclusions.
**The sample size of negative comments in the Power and Speed categories are too small to make any definitive conclusions on their own.
This allows a direct comparison to determine who commentators are talking about when they mention a particular topic in a positive or negative light.
Table 3: Ratio adjusted percentages for negative comments
| Darker skin tone | Lighter skin tone | |
| Quality | 64.57% | 32.43% |
| Appearance* | 0.00% | 100.00% |
| Background | 46.88% | 53.13% |
| Form | 53.25% | 46.75% |
| Hard-work | 42.86% | 57.14% |
| In-game event | 50.00% | 50.00% |
| Intelligence | 63.33% | 36.67% |
| Leadership + mentality | 48.00% | 52.00% |
| Power** | 0.00% | 0.00% |
| Speed** | 0.00% | 100.00% |
| Versatility | 73.53% | 26.47% |
*The sample size of the Appearance category is too small to make any definitive conclusions.
**The sample size of negative comments in the Power and Speed categories are too small to make any definitive conclusions on their own.
Table 4: Ratio adjusted percentages for positive comments
| Darker skin tone | Lighter skin tone | |
| Quality | 37.21% | 62.79% |
| Appearance* | 85.71% | 14.29% |
| Background | 44.54% | 55.46% |
| Form | 59.64% | 40.36% |
| Hard-work | 39.60% | 60.40% |
| In-game event | 46.78% | 53.22% |
| Intelligence | 37.40% | 62.60% |
| Leadership + mentality | 42.86% | 57.14% |
| Power** | 86.76% | 13.24% |
| Speed** | 84.17% | 15.83% |
| Versatility | 34.21% | 65.79% |
*The sample size of the Appearance category is too small to make any definitive conclusions.
Praise and criticism
When talking about intelligence, versatility and quality commentators praise players with lighter skin tone and criticize players with darker skin tone.
Taking intelligence as an example, 62.60% of praise was directed towards players with lighter skin tone.
This flips when criticizing players for their intelligence, where 63.33% of the criticism is directed at players with darker skin tone.
As put by Rada & Wulfemeyer (2005):
“One form of racial bias that researchers have consistently uncovered is the “brawn versus brains” descriptions directed towards players.”
And the paper goes onto mention:
“When announcers do refer to the White athletes’ physical accomplishment and/or attributes, they often highlight two factors: the players’ intellectual and cognitive prowess, and a strong, blue collar-like work ethic.”
The continuous praise of players with lighter skin tone for their skill level, leadership and cognitive abilities combined with the continuous criticism for players with darker skin tone is likely to influence the perception of the soccer watching public.
Praise of athletic abilities
It is rare for commentators to make negative statements about any player’s physical condition in Europe’s top leagues.
However, when praising players for their speed, 84.17% of praise is directed at players with darker skin tone. When praising players for their strength, 86.76% of comments are aimed at players with darker skin tone.
As quoted previously, reducing players with darker skin tone to their athletic abilities paints those players “as naturally athletic or endowed with God-given athleticism exacerbates the stereotype by creating the impression of a lazy athlete, one who does not have to work at his craft.”
PFA Equalities Executive, Jason Lee says:
“To address the real impact of structural racism, we have to acknowledge and address racial bias. This study shows an evident bias in how we describe the attributes of footballers based on their skin colour.
Commentators help shape the perception we hold of each player, deepening any racial bias already held by the viewer. It’s important to consider how far-reaching those perceptions can be and how they impact footballers even once they finish their playing career.
If a player has aspirations of becoming a coach/manager, is an unfair advantage given to players that commentators regularly refer to as intelligent and industrious, when those views appear to be a result of racial bias?”
Methodology
This study is built upon a common methodology to analyze bias within the sports media based on skin tone. In the past, a number of studies have been conducted covering bias in the Olympics, College Basketball, and American Football.
Selection of games
A total of 80 soccer matches - 20 from each of the English Premier League, Spanish La Liga, French Ligue 1, and Italian Serie A - were selected at random from the 2019/20 season.
To gather a sample of soccer matches, Rada’s (1996) method for collecting composite games was used. Segments (15 minutes) from each of the 80 games were taken at random. This was done by creating a list of games to be analyzed, then designating a segment to be watched, transcribed, and coded.
For Game 1, 0-14 minutes would be analyzed. For Game 2, 15-29 minutes would be analyzed and this would continue for 6 games. Game 7 would return to segment 1 of 0-14 minutes.
By taking a sample of each game it allows for analysis of a wider variety of teams, players and commentators, essentially creating a larger sample size from the same hours of coverage.
Some teams were involved in multiple games. This is because of the number of games plus the fact that more successful teams are scheduled for live TV
The reasons for using live match commentary were best put forward by Rada (1996):
“As events transpire in front of them at a machine-gun pace, announcers are caught up in the ‘heat of the battle.’ Having to inform and entertain in this environment, often without the time to choose words carefully, causes announcers to dredge up comments that reflect subconscious beliefs, images, attitudes, and values.”
Transcription
Once the games and segments had been selected, they were downloaded, watched and transcribed. The outcome of this stage was 80 transcripts (one for each match).
Coding (Categorization of comments)
As per Wonsek (1992), negative comments alone are not necessarily indicative of racial bias. Racial stereotypes can manifest themselves both through the presence of specific messages, or the absence thereof, depending on the context in which those messages do, or do not, occur.
As such, as per Rada & Wulfemeyer (2005), “analysis centered around what was said about whom and how frequently.”
This research employed many of the categories used in previous research but adapted for soccer as none of the studies on which this study is based focused on soccer. Please see the categories section for further details.
The coding process involved recording every adjectival descriptor and phrase/ sentence within each of the 80 transcripts. Each phrase was manually assigned by a coder as being positive (praise) or negative (criticism), was assigned a category and associated with the player, whom the comment was directed at.
Player information and identification of race
Player information was extracted from Football Squads. Information on each player’s skin tone was pulled from Football Manager 2020. In this game, each player has been designated a skin tone value from 1 to 20. 1 being the lightest and 20 being the darkest.
By using this, the researchers are not interpreting the race of players and the study is based purely on skin tone rather than ethnicity or a researcher’s interpretation of what race a person is.
A cut off point must be made to separate the two groups, so all player’s with a skin tone value of 1-11 were classified as “players with lighter skin tone.”
All player’s with a skin tone value of 12-20 were designated as “players with darker skin tone.”
It is important to note here that in trying to determine a player’s race, there always exists the possibility of misidentification. It is well beyond the scope of this research to engage in a discussion of the anthropological, biological, cultural, political, and sociological ramifications of race.
By focusing on skin tone alone, we are able to show a clear relationship between the color of a player’s skin, and the analysis that players receive from commentary teams.
Analysis and results
Once coding had been completed and player information had been collected, the end result was 2,073 positive and negative comments, sorted into 11 categories and associated with 643 unique players.
Given the sample size of 643 players, of which 433 were classified as players with lighter skin tone and 210 were classified as players with darker skin tone, it is reasonable to expect the distribution of comments to be within the margin of error for each category if there was an absence of bias from commentator’s based on the color of a player’s skin. The fact that there was a significant difference shows bias is present.
Categories
Below is an explanation of the 11 categories included in the study, along with a real example of a comment included in the study for each category:
-
In-game event: Factual comments about events occurring within the game. This category has minimal opinion and is simply positive or negative comment on events unfolding (e.g. “controlled well by Ekdal.”)
-
Power: Any mention of a player being strong or powerful (e.g. “He’s an important player in the Norwich City midfield. A strong, combative player.”)
-
Speed: Any mention of a player being a fast player (e.g. “His pace is frightening. He gets beyond people quite easily.”)
-
Leadership + mentality: Any mention of a player’s leadership ability, will to win or similar mental strength (e.g. “There's their talisman, Troy Deeney.”)
-
Intelligence: Any mention of a player’s in-game intelligence (e.g. “Very intelligent. Saw the goalkeeper off his line. Sometimes it doesn't have to be beautiful, it just needs to go in the back of the net.”)
-
Hard-work: Mention of a player’s work ethic or hard work (e.g. “Hwang is chasing that. He Worked so hard to get there.”)
-
Form: Praising or criticizing a player for how they are playing in the current game or in recent weeks (e.g. “He’s playing the best football of his career.”)
-
Versatility: Any mention of a player’s ability to play in various positions or perform multiple tasks for the team (e.g. “Players like Sergi Roberto are fitting in right, left and center just for the good of the team.”)
-
Appearance: Any mention of a player’s appearance. Mainly related to height. (e.g. “Header on by the giant Ajorque.”)
-
Background: Mention of a player’s history or information outside of the current match (e.g. “Pablo Zabaleta turned 35 last week, still looking good on it. The vastly experienced former Argentina International. 301 Premier League games now.”)
-
Quality: Any mention of a player’s quality, including position-specific or set-piece abilities or overall quality as a footballer (e.g. “He's a very good passer.”)
References
- Rada & Wulfemeyer (2005) - Color-Coded: Racial Descriptors in Television Coverage of Intercollegiate Sports
- Eastman & Billings (2001) - Biased Voices of Sports: Racial and Gender Stereotyping in College Basketball Announcing
- Rada (1996) - Color blind‐sided: Racial bias in network television's coverage of professional football games
About RunRepeat
RunRepeat is a team of sports enthusiasts who create the most objective and unbiased shoe reviews in the world. How we do it? 1. we buy all the shoes using our own money 2. we test the shoes ourselves - be it running, hiking, walking, training in the gym or playing basketball on the court 3. we get all the shoes to our lab where we weigh them, cut them in half, bend them, pump smoke in them... all of which ends with 20+ data points that accurately describe the shoe and how it performs. To learn more about our testing methodology, please check out the Testing Methodology page.
