Evaluation of London 2012 Olympic medal tally forecasts: Who has won?

Gerard Kuper and Elmer Sterken

August 2012

 

At the London 2012 Summer Games 962 medals have been awarded to athletes from 85 countries. With 302 medal events and 53 events awarding two bronze medals the total number of medals at the Olympic Games is 659. However in some events athletes have tied:

       In the menís high jump three bronze medals are awarded.

       In the 200m freestyle swimming for men two silver medals, and no bronze medals, are awarded.

       In the menís keirin track cycling event two bronze medals are awarded.

This makes a total of 302 gold medals, 303 silver medals and 357 bronze medals.

 

Medal winning is concentrated: just as in 2004 and 2008 the Top-8 from the medal tally won about 50% of the medals total. In this note we compare six forecasts, and focus on the Top-31 since Poland and Azerbaijan are tied on place 30. These 31 countries take 83% of all medals.

 

Forecasts differ

There are two types of medal tally forecasts: expertís opinions and model-based forecasts. Examples of the former are USA Today and Sports Illustrated. These journals name each gold, silver and bronze medal winner from which we compile the medal table based on the number of gold medals. Johnson, Williams (using the method of Bernard and Busse), and Kuper and Sterken all use econometric models to generate forecasts. Barra bases his predictions solely on World Championship results.

 

Johnson and Williams predict the number of gold medals and the total number of medals per country. The others predict the number of gold, silver and bronze medals won per country. The predictions also differ with respect to the number of countries considered. Johnson and Williams have the smallest numbers of countries (33 and 65 countries, respectively). Kuper and Sterken have 126 countries, with 76 countries winning at least one medal. USA Today has 79 medal winning countries, and Sports Illustrated has 82 medal winning countries which is very close to the actual number of countries that have won medals in the London 2012 Olympic Games.

 

Before we analyse the performance of each forecast, we should note that Johnson has a smaller sample of countries (only 33) and misses 1 Top-10 country (Korea) and nine Top-31 countries. Williams only publishes a list of countries winning at least six medals, and a table of countries winning at least four gold medals. Out of 20 countries winning at least four gold medals she identifies only 13. Out of 37 countries winning at least six medals she identifies only 28. In this respect Kuper and Sterken are closer to the realisations with 19 countries expected to win at least four gold medals and 36 counties expected to win at least six medals.

 

Comparing forecasts

There are various ways to compare the forecasts. Obviously, one way is to calculate rank correlation coefficients between the various ranking of countries and the actual ranking. There are some complications with the publications of Johnson and Williams as discussed above. For Williams we have to rank the countries on place 14 and below based on the number of total medals won. In the case of Johnson all non-listed countries are tied on place 34 with 0 medals. This will affect the Kendall rank correlation coefficients for these forecasts.

 

Table 1 presents Kendall rank correlation coefficients for the Top-31 since Poland and Azerbaijan are tied on place 30. Johnson has the smallest rank correlation coefficient. However, with a standard error of 0.127 the rank correlation coefficients are not significantly different.

 

Table 1 Ė Kendall rank correlation coefficients for the Top-31 countries

 

Sports Illustrated ††††††††††††††††† 0.525

USA Today††††††††††††††††††††††††††† 0.576

Kuper & Sterken †††††††††††††††††† 0.525

Williams††††††††††††††††††††††††††††††† 0.557

Johnson††††††††††††††††††††††††††††††††† 0.372

Barra††††††††††††††††††††††††††††††††††††† 0.525

 

Because of the issues discussed above with some of the publications a better way to analyze the medal forecasts is to compute the distance of the forecasted medal tally to the actual results. So we compute the Mean Absolute Errors and the Mean Squared Errors for gold, silver, bronze, and the total number of medals. The results are in Table 2. Here one can see that the experts of Sports Illustrated perform best. USA Today and Kuper and Sterken show very similar results. The forecasts of the other model-based forecasts are worse.

 

Table 2 Ė Mean Absolute Errors (MAE) and Mean Squared Errors (MSE) for the Top-31 countries

 

MAE

Gold

Silver

Bronze

Total

Kuper and Sterken

2.935

2.645

2.387

4.355

Sports Illustrated

2.742

2.065

1.935

3.387

USA Today

3.032

2.581

2.806

3.903

Johnson

3.419

7.645

Williams

3.267

4.516

Barra

3.677

2.581

2.677

4.484

MSE

Gold

Silver

Bronze

Total

Kuper and Sterken

16.161

15.871

9.290

28.935

Sports Illustrated

12.226

7.355

7.355

16.548

USA Today

16.258

12.387

11.903

36.032

Johnson

22.710

106.290

Williams

24.538

35.516

Barra

20.516

15.161

12.484

44.032

 

 

Conclusion

Sports Illustrated outperforms the other forecasts of Olympic medal tallies. Kuper and Sterken use a mix of econometric and World Cup results and clearly outperforms the other model-based forecasts. Their forecast performance is similar to USA Today but much better that the other forecasts.

 

 


Some more facts

On August 11, 2012, Gabon won its first ever Olympic medal when Anthony Obame took silver in Taekwondo (men's +80kg weight class).

 

Guatemala won their first Olympic medal at the 2012 Summer Olympics on August 4, 2012, when Erick Barrondo took second place in the men's 20km walk.

 

One day after the closing ceremonies the Belarusian shot putter Nadzeya Ostapchuk has been stripped of her Olympic gold medal after failing a doping test. New Zealand's Valerie Adams will be awarded the gold, Russia's Evgeniia Kolodko will take the silver and China's Lijiao Gong will win the bronze. Ostapchuk was tested twice, once on 5 August and again the following day after she had finished first. The IOC said both samples indicated the presence of metenolone, which is classified as an anabolic agent.

 

Significance test

The sampling distribution of the Kendall rank correlation coefficient converges towards a normal distribution (the convergence is satisfactory for values of N larger than 10), with a mean of 0 and a variance equal to [2(2N+5)] / [9N(N-1)].