The Runscore

The Runscore is a 0-100 score that indicate how liked a running shoe is based on reviews and ratings from running shoe experts and users.

 

Runscore simplified

 

How the Runscore is calculated

The Runscore is a weighted average of user ratings and expert reviews adjusted for spam, shoes with few reviews, what version is reviewed and the credibility of the expert reviewer.
Runscore pipes

User ratings

Any runner can contribute with user ratings of shoes he/she has tried. To avoid that a running shoes with only a few (good) ratings reach the top of the ranking lists, adjustments are made:

  • 1-10 user ratings: subtract 15%
  • 11-25 user ratings: subtract 10%
  • 26-50 user ratings: subtract 5%
  • >50 user ratings = no adjustment

 

User rating distribution

Below is illustration how users rate running shoes. 71% give 5 stars.

 

user_rating_distribution

 

Number of user ratings

Below is illustrated how many ratings each running shoe has. 64% of all running shoes have 1 to 99 ratings. The most popular running shoes get +1000 reviews.

 

number_of_user_ratings

 

Expert reviews

To avoid that a running shoe with one (great) expert review easily reach the top of the ranking lists, adjustments are applied:

  • 1 expert review: subtract 15%
  • 2 expert reviews: subtract 10%
  • 3 expert reviews: subtract 5%
  • >3 expert reviews: no adjustment

If an expert scores a shoe as 8/10, it will be converted to 80/100. In the same way, a score of 3/5 becomes 60/100.

In case an expert does not attribute any numerical score, three RunRepeat.com team members independently interprets the reviews and suggests a 0-100 score. An average of the three interpretations is applied. Any expert can ask to adjust scores.

To be an expert one must publish reviews at other websites (typically his/her own). In addition we manually judge if the quaility is good enough.

How respected an expert is, is a subjective matter, which can be discussed for days. To keep things simple, and to avoid subjective judgement, we attribute each expert with a level from 1-5 depending on how many reviews the expert has published.

  • 1-2 reviews: level 1
  • 3-5 reviews: level 2
  • 6-14 reviews: level 3
  • 15-30 reviews: level 4
  • 31+ reviews: level 5

The higher the level the more the reviews weight.

  • Level 1: 100%
  • Level 2: 200%
  • Level 3: 300%
  • Level 4: 400%
  • Level 5: 500%

 

Expert level distribution

Below is illustrated how most experts are level 1 experts and only 2% of all experts are level 5. Though, the 2% level 5 experts account for more than 35% of all reviews.

 

experts

 

Expert review distribution

The average expert review is 85/100. Below you see the percentage distribution of all expert reviews.

 

expert_review_distribution

 

The final calculation

The 0-100 Runscore is finally calculated by weighting the user ratings and expert reviews.

  • User ratings: 67%
  • Expert reviews: 33%

The color of the Runscore depends on it’s rating:

  • 80-100: Green
  • 60-79: Yellow
  • 00-59: Red

 

Runscore distribution

80% of all running shoes have a Runscore between 73 and 90 with an average of 81. Running shoe with a Runscore above 81 is better rated than the average running shoe. Below is illustrated the distribution of the Runscores in percentages.

 

runscore_distribution

 

Potential flaws in the Runscore

The Runscore is not a universal truth of how good running shoes are. Here are a few potential flaws:

  • Brands have a special interest in high Runscores on any of their own running shoes and low Runscores on all other brands. They might try to manipulate the Runscore by adding many user ratings. We are aware of this, and this has in fact been a problem (the brands will not be mentioned). Our way to avoid the problem is by making each user rating IP restricted. Each user can only give 1 rating to each shoe. In addition we have a spam-detecter that tries to detect if certain patterns appear, which we obivously will not explain how works here.
  • Some experts are biased since they get free samples (and cash) from brands.
  • Experts who do not give a numerical score will be judged by three independent RunRepeat members. We try to be as neutral as possible, but we cannot guarantee that some members will attribute some expert reviews with a slightly more positive score because they themselves are personally biased with own experiences of that specific running shoe.
  • An expert might have done 18 reviews without knowing more about running shoes that an expert who published 2 reviews, though the first’ reviews will be weighted higher.

Keep in mind that choosing the right running shoe is individual, and that running shoes with a low Runscore still have some lovers, which might be you. We suggest that you use the Runscore with a grain of salt.