Evaluation of Beijing 2008 Olympic medal tally forecasts: Who has won?

Gerard Kuper and Elmer Sterken

September 2008

 

 

During the Beijing Summer Games 957 medals have been awarded to sportsmen form 87 countries. Medal winning is concentrated: just as in 2004 the top-8 from the medal tally won 50 per cent of the medals total (75 percent is won by 23 countries. In this note we are interested in the quality of medal tally predictions. We compare 6 forecasts. There are two types of medal tally forecasts: experts opinions and model based forecasts. Examples of the former relating to the Athens Summer Games are published in USA Today (UT) and Sports Illustrated (SI). Johnson and Ali (JA), Bernard and Busse (BB), and Kuper and Sterken (KS) all used econometric models to generate forecasts. Barra (Ba) uses world championship results.

 

JA and BB predict the number of gold medals and the total number of medals per country. The others predict the number of gold, silver and bronze medals won per country. The various predictions also differ with respect to the number of countries considered. JA and BB has the smallest numbers of countries (36 and 33 countries, respectively). KS has 80 countries, UT has 82 and SI has 83 which is very close to the actual number of countries that have won medals in the Beijing 2008 Olympic Games.

 

There are various ways to compare the forecasts. Obviously, one way is to calculate rank coefficients of the various ranking of countries and the actual ranking. The rankings are based on the number of gold medals. The table below presents Kendall rank coefficients for the Top 30 to give JA and BB a fair chance to compete with the other studies. Nevertheless, the experts win and KS definitely outperforms the other econometric model-based forecasts and performs similar to the expert forecast.

 

Table 1 Kendall rank coefficients for the top 30 countries

 

 

 

 

Kendall rank coefficient

JOHNSON & ALI

 

0.211 (6)

BERNARD & BUSSE

 

0.411 (5)

BARRA

 

 

0.591 (4)

SPORTS ILLUSTRATED

0.669 (2)

USA TODAY

 

0.674 (1)

KUPER & STERKEN

 

0.669 (2)

 

 

Another way to analyze the medal forecasts is to compute the distance of the forecasted medal tally to the actual results. So we compute the Mean squared errors for Gold, Silver, Bronze, and the Total number of medals. The results are in Table 2. Here one can see that the expert forecasts by Sports Illustrated and USA Today perform best. Of the model forecasts the forecasts by KS are in the same ballpark.

 

 

 

 

 

 

Table 2 Mean squared errors for all countries and the top 30 countries

 

 

 

 

 

ALL COUNTRIES

 

 

 

 

Gold

Silver

Bronse

Total

JOHNSON & ALI

 

7.435 (6)

 

 

54.336 (6)

BERNARD & BUSSE

 

5.176 (5)

 

 

23.473 (5)

BARRA

 

 

4.985 (4)

4.450 (4)

8.031 (4)

16.702 (3)

SPORTS ILLUSTRATED

3.771 (3)

4.313 (3)

5.229 (1)

10.824 (2)

USA TODAY

 

2.611 (1)

2.405 (1)

6.771 (3)

8.916 (1)

KUPER & STERKEN

 

3.221 (2)

2.603 (2)

6.237 (2)

18.748 (4)

 

 

 

 

 

TOP 30

 

 

 

 

Gold

Silver

Bronse

Total

JOHNSON & ALI

 

29.200 (6)

 

 

184.533 (6)

BERNARD & BUSSE

 

19.800 (4)

 

 

79.467 (5)

BARRA

 

 

20.433 (5)

16.900 (4)

30.333 (4)

61.400 (3)

SPORTS ILLUSTRATED

15.267 (3)

15.667 (3)

17.000 (1)

39.067 (2)

USA TODAY

 

10.200 (1)

7.233 (1)

23.433 (2)

32.133 (1)

KUPER & STERKEN

 

10.967 (2)

8.033 (2)

23.433 (2)

67.633 (4)

 

 

Conclusions

 

Expert forecasts outperform econometric forecasts of Olympic medal tallies. Expert forecasts are often based on within-Olympic period results (e.g. in World Cups). Kuper and Sterken use a mix of econometric and World Cup results and clearly outperforms the other econometric studies.