Racial Bias in American football Commentary (Study): The Pace and Power Effect

RunRepeat, in partnership with The Professional Footballers' Association, has analysed 2,073 statements from commentators in 80 football matches from the 2019/20 season. This is the first time a study of this kind has been conducted to understand if there is a difference in the way the football media talks about players of different skin colour.

The comments discussed 643 unique players of various races and skin tones.

Key findings

The findings show bias from commentators who praised players with lighter skin tone as more intelligent, as being of higher quality, and harder working than players with darker skin tone.

Players with darker skin tone were significantly more likely to be reduced to their physical characteristics or athletic abilities -- namely pace and power -- than players with lighter skin tone players were.

In numbers:

When commentators talk about intelligence:
- 62.60% of praise was aimed at players with lighter skin tone
- 63.33% of criticism was aimed at players with darker skin tone
When commentators are talking about power they are 6.59 times more likely to be talking about a player with darker skin tone
When commentators are talking about speed they are 3.38 times more likely to be talking about a player with darker skin tone
When commentators talk about work ethic, 60.40% of praise is aimed at players with lighter skin tone

_{You may use material from this report as long as you refer to this original report. For special requests, or questions, email lead researcher Danny McLoughlin at danny@runrepeat.com.}

Overview of methodology

Selection of Games: Selected a total of 80 games from 4 of Europe’s top leagues (Italian Serie A, Spanish La Liga, French Ligue 1, and English Premier League) from the 2019/20 season.
Transcription: Transcribed commentary from each game into a text document.
Coding (Categorization of Comments): Recorded every adjectival phrase and manually assigned whether the comment was positive (praise) or negative (criticism), a category, and the associated player.
Player Information and Identification of Race: Designated a player as “players with lighter skin tone” or “players with darker skin tone” based on skin tone attribute from American football Manager 2020 (an extensive database maintained by over 1,300 scouts).
Analysis and Results: Analysed the ratio of praise and criticism for each category to determine differences in how players of different skin tones are talked about.

For a full, detailed methodology including references and definition of categories please see the full methodology section at the bottom of this document.

What does bias look like?

If there was no bias in commentary, the distribution of comments towards players of different skin colours would be similar.

Players with lighter skin tone should receive the same proportion of comments about, for example, their intelligence or their work-ethic as players with darker skin tone.

The fact this is not the case across a large sample size indicates there is bias in the way the media discusses players based on the colour of their skin.

Control category

The control category of “in-game events” was included to support the theory that there would be an even distribution of comments in the absence of bias.

There was less scope for bias in this category given it is “considered probably factual unless additional adjectives characterising a player were included.” Eastman & Billings (2001).

Comments such as “That's a good ball in for Rafael at the back post.” or “But not the best ball from Toni Kroos.” were recorded as a positive or negative “In-game event.”

When commentators made purely factual statements (in-game events), both groups received the same proportion of comments and were praised or criticised the same amount.

The even distribution in this control category (where commentators are stating facts) is further indication that bias is present in the other categories (which are based on the opinion of the commentator) where the distribution and split of praise and criticism looks different.

Results

Table 1: Base Numbers

	Darker skin tone		Lighter skin tone
Sentiment	Negative	Positive	Negative	Positive
Quality	13	50	12	162
Appearance	0	3	1	1
Background	8	28	17	66
Form	47	86	79	111
Hard-Work	3	21	8	61
In-game event	72	175	138	380
Intelligence	10	24	11	77
Leadership + Mentality	25	44	52	112
Power	0	31	0	9
Speed	0	53	11	19
Versatility	13	7	9	25

What was said about whom and how frequently?

Looking at the distribution of comments for each group allows us to look at how often commentators bring up a subject when talking about each group. For example, when talking about players with darker skin tone, they bring up form 18.65% of the time. When talking about players with lighter skin tone, they bring up leadership 12.05% of the time.

There is a significant difference in how often commentators discuss topics for our two groups.

Likelihood to comment on athletic abilities

These differences are most stark when commentators are discussing physical characteristics/athletic abilities -- speed and strength. Or put into a more recognisable American football stereotype when referred to as “pace and power”.

Commentators are 6.59 times more likely to comment on the strength of players with darker skin tone than they are to comment on the strength of players with lighter skin tone.

Commentators are 3.38 times more likely to comment on the speed of players with darker skin tone than the speed of players with lighter skin tone.

Rada & Wulfemeyer (2005) stated:

“Portraying African Americans as naturally athletic or endowed with God-given athleticism exacerbates the stereotype by creating the impression of a lazy athlete, one who does not have to work at his craft.”

Likelihood to comment on in-game events

Returning to the control, the in-game event category shows that both groups of players are doing similar things on the pitch. So talking about them in different terms indicates a presence of bias.

Who was praised? Who was criticised? And for what?

This study comprises 643 unique players. 433 of these have been classified as “players with lighter skin tone” and 210 have been classified as “players with darker skin tone” based on the criteria listed in the methodology section at the bottom of the page.

In order to form an apples to apples comparison, we multiplied the number of comments about players with darker skin tone to equal the number of comments about players with lighter skin tone, allowing us to analyse who commentators criticise and praise per subject.

Ratio adjusted data

In the sample, there were a total of 2,073 statements. Of these, 1,361 were towards the group of players with lighter skin tone and 713 were towards the group of players with darker skin tone.

A multiplier of 1.9 was used resulting in 1,362 comments about players with darker skin tone and 1,361 comments about players with lighter skin tone:

Table 2: Ratio adjusted numbers

	Darker skin tone	Lighter skin tone	Darker skin tone	Lighter skin tone
Sentiment	Negative	Negative	Positive	Positive
Quality	25	12	96	162
Appearance*	0	1	6	1
Background	15	17	53	66
Form	90	79	164	111
Hard-work	6	8	40	61
In-game event	138	138	334	380
Intelligence	19	11	46	77
Leadership + mentality	48	52	84	112
Power**	0	0	59	9
Speed**	0	11	101	19
Versatility	25	9	13	24

_{*The sample size of the Appearance category is too small to make any definitive conclusions.}
_{**The sample size of negative comments in the Power and Speed categories are too small to make any definitive conclusions on their own.}

This allows a direct comparison to determine who commentators are talking about when they mention a particular topic in a positive or negative light.

Table 3: Ratio adjusted percentages for negative comments

	Darker skin tone	Lighter skin tone
Quality	64.57%	32.43%
Appearance*	0.00%	100.00%
Background	46.88%	53.13%
Form	53.25%	46.75%
Hard-work	42.86%	57.14%
In-game event	50.00%	50.00%
Intelligence	63.33%	36.67%
Leadership + mentality	48.00%	52.00%
Power**	0.00%	0.00%
Speed**	0.00%	100.00%
Versatility	73.53%	26.47%

Table 4: Ratio adjusted percentages for positive comments

	Darker skin tone	Lighter skin tone
Quality	37.21%	62.79%
Appearance*	85.71%	14.29%
Background	44.54%	55.46%
Form	59.64%	40.36%
Hard-work	39.60%	60.40%
In-game event	46.78%	53.22%
Intelligence	37.40%	62.60%
Leadership + mentality	42.86%	57.14%
Power**	86.76%	13.24%
Speed**	84.17%	15.83%
Versatility	34.21%	65.79%

_{*The sample size of the Appearance category is too small to make any definitive conclusions.}

Praise and criticism

When talking about intelligence, versatility and quality commentators praise players with lighter skin tone and criticise players with darker skin tone.

Taking intelligence as an example, 62.60% of praise was directed towards players with lighter skin tone.

This flips when criticising players for their intelligence, where 63.33% of the criticism is directed at players with darker skin tone.

As put by Rada & Wulfemeyer (2005):

“One form of racial bias that researchers have consistently uncovered is the “brawn versus brains” descriptions directed towards players.”

And the paper goes onto mention:

“When announcers do refer to the White athletes’ physical accomplishment and/or attributes, they often highlight two factors: the players’ intellectual and cognitive prowess, and a strong, blue collar-like work ethic.”

The continuous praise of players with lighter skin tone for their skill level, leadership and cognitive abilities combined with the continuous criticism for players with darker skin tone is likely to influence the perception of the football watching public.

Praise of athletic abilities

It is rare for commentators to make negative statements about any player’s physical condition in Europe’s top leagues.

However, when praising players for their speed, 84.17% of praise is directed at players with darker skin tone. When praising players for their strength, 86.76% of comments are aimed at players with darker skin tone.

As quoted previously, reducing players with darker skin tone to their athletic abilities paints those players “as naturally athletic or endowed with God-given athleticism exacerbates the stereotype by creating the impression of a lazy athlete, one who does not have to work at his craft.”

PFA Equalities Executive, Jason Lee says:

“To address the real impact of structural racism, we have to acknowledge and address racial bias. This study shows an evident bias in how we describe the attributes of footballers based on their skin colour.

Commentators help shape the perception we hold of each player, deepening any racial bias already held by the viewer. It’s important to consider how far-reaching those perceptions can be and how they impact footballers even once they finish their playing career.

If a player has aspirations of becoming a coach/manager, is an unfair advantage given to players that commentators regularly refer to as intelligent and industrious, when those views appear to be a result of racial bias?”

Methodology

This study is built upon a common methodology to analyse bias within the sports media based on skin tone. In the past, a number of studies have been conducted covering bias in the Olympics, College Basketball, and American American football.

Selection of games

A total of 80 football matches - 20 from each of the English Premier League, Spanish La Liga, French Ligue 1, and Italian Serie A - were selected at random from the 2019/20 season.

To gather a sample of football matches, Rada’s (1996) method for collecting composite games was used. Segments (15 minutes) from each of the 80 games were taken at random. This was done by creating a list of games to be analysed, then designating a segment to be watched, transcribed, and coded.

For Game 1, 0-14 minutes would be analysed. For Game 2, 15-29 minutes would be analysed and this would continue for 6 games. Game 7 would return to segment 1 of 0-14 minutes.

By taking a sample of each game it allows for analysis of a wider variety of teams, players and commentators, essentially creating a larger sample size from the same hours of coverage.

Some teams were involved in multiple games. This is because of the number of games plus the fact that more successful teams are scheduled for live TV

The reasons for using live match commentary were best put forward by Rada (1996):

“As events transpire in front of them at a machine-gun pace, announcers are caught up in the ‘heat of the battle.’ Having to inform and entertain in this environment, often without the time to choose words carefully, causes announcers to dredge up comments that reflect subconscious beliefs, images, attitudes, and values.”

Transcription

Once the games and segments had been selected, they were downloaded, watched and transcribed. The outcome of this stage was 80 transcripts (one for each match).

Coding (Categorization of comments)

As per Wonsek (1992), negative comments alone are not necessarily indicative of racial bias. Racial stereotypes can manifest themselves both through the presence of specific messages, or the absence thereof, depending on the context in which those messages do, or do not, occur.

As such, as per Rada & Wulfemeyer (2005), “analysis centred around what was said about whom and how frequently.”

This research employed many of the categories used in previous research but adapted for football as none of the studies on which this study is based focused on football. Please see the categories section for further details.

The coding process involved recording every adjectival descriptor and phrase/ sentence within each of the 80 transcripts. Each phrase was manually assigned by a coder as being positive (praise) or negative (criticism), was assigned a category and associated with the player, whom the comment was directed at.

Player information and identification of race

Player information was extracted from American football Squads. Information on each player’s skin tone was pulled from American football Manager 2020. In this game, each player has been designated a skin tone value from 1 to 20. 1 being the lightest and 20 being the darkest.

By using this, the researchers are not interpreting the race of players and the study is based purely on skin tone rather than ethnicity or a researcher’s interpretation of what race a person is.

A cut off point must be made to separate the two groups, so all player’s with a skin tone value of 1-11 were classified as “players with lighter skin tone.”

All player’s with a skin tone value of 12-20 were designated as “players with darker skin tone.”

It is important to note here that in trying to determine a player’s race, there always exists the possibility of misidentification. It is well beyond the scope of this research to engage in a discussion of the anthropological, biological, cultural, political, and sociological ramifications of race.

By focusing on skin tone alone, we are able to show a clear relationship between the colour of a player’s skin, and the analysis that players receive from commentary teams.

Analysis and results

Once coding had been completed and player information had been collected, the end result was 2,073 positive and negative comments, sorted into 11 categories and associated with 643 unique players.

Given the sample size of 643 players, of which 433 were classified as players with lighter skin tone and 210 were classified as players with darker skin tone, it is reasonable to expect the distribution of comments to be within the margin of error for each category if there was an absence of bias from commentator’s based on the colour of a player’s skin. The fact that there was a significant difference shows bias is present.

References

Rada & Wulfemeyer (2005) - Colour-Coded: Racial Descriptors in Television Coverage of Intercollegiate Sports
Eastman & Billings (2001) - Biased Voices of Sports: Racial and Gender Stereotyping in College Basketball Announcing
Rada (1996) - Colour blindâsided: Racial bias in network television's coverage of professional American football games

About RunRepeat

RunRepeat is a team of sports enthusiasts who create the most objective and unbiased shoe reviews in the world. How we do it? 1. we buy all the shoes using our own money 2. we test the shoes ourselves - be it running, hiking, walking, training in the gym or playing basketball on the court 3. we get all the shoes to our lab where we weigh them, cut them in half, bend them, pump smoke in them... all of which ends with 20+ data points that accurately describe the shoe and how it performs. To learn more about our testing methodology, please check out the Testing Methodology page.