Evaluation of Vancouver 2010 Olympic medal tally forecasts: Who has won?

Gerard Kuper and Elmer Sterken

March 2010

During the Vancouver Winter Games 258 medals have been awarded to sportsmen form 26 countries: 86 Gold, 87 Silver and 85 Bronze medals. Two silver medals were awarded to the Men's 20 km Individual Biathlon event. Just as in 2006 medal winning is concentrated: The top-5 from the medal tally won 50 per cent of the medals total (72 percent is won by 10 countries.

In this note we are interested in the quality of medal tally predictions. We compare 6 forecasts. There are two types of medal tally forecasts: experts opinions and model based forecasts. Examples of the former are The Associated Press (AP), USA Today (UT) and Sports Illustrated (SI). These experts pick the winners of gold, silver and bronze medals for all 86 events. Johnson (JO) and Kuper and Sterken (KS) used econometric models to generate forecasts. Barra (BA) uses world championship results.

JO predicts the number of gold medals and the total number of medals for countries he thinks will win 10 medals or more. KS predict the number of gold, silver and bronze medals for all countries that won at least one medal in earlier editions of the Winter Games. The various predictions also differ with respect to the number of countries considered. JO has the smallest numbers of countries (13). The others are very close to the actual number of countries that have won medals in the Vancouver 2010 Olympic Games: UT and AP have 24 countries, KS and SI have 25 and BA has 26. The latter which is very

 There are various ways to compare the forecasts. Obviously, one way is to calculate rank coefficients of the various ranking of countries and the actual ranking. The rankings are based on the number of gold medals. The table below presents Kendall rank coefficients for all medal winning countries. The experts win and Kuper and Sterken definitely outperforms the other econometric model-based forecast and Barra.

 Table 1 – Kendall rank coefficients for the Top-10 and all 26 medal winning countries

Top-10	Kendall
Kuper&Sterken	0.444
The Associated Press	0.533
Barra	0.400
Johnson	0.533
Sports Illustrated	0.667
USA Today	0.444

All countries	Kendall
Kuper&Sterken	0.492
The Associated Press	0.532
Barra	0.474
Johnson	0.055
Sports Illustrated	0.603
USA Today	0.566

Another way to analyze the medal forecasts is to compute the distance of the forecasted medal tally to the actual results. So we compute the Mean squared errors for Gold, Silver, Bronze, and the Total number of medals. The results are in Table 2. Here one can see that the expert forecasts by Sports Illustrated, USA Today and The Associated Press perform best. The forecasts by Kuper and Sterken beat Barra and Johnson.

Table 2 – Mean squared errors for the Top-10 and all 26 medal winning countries

Top-10	Gold	Silver	Bronze	Total
Kuper&Sterken	4.50	6.90	6.20	22.00
The Associated Press	3.80	9.20	8.70	18.90
Barra	12.00	15.30	10.50	13.20
Johnson	15.00			51.10
Sports Illustrated	3.20	7.30	8.50	22.00
USA Today	3.90	6.00	12.90	28.40

All countries	Gold	Silver	Bronze	Total
Kuper&Sterken	3.32	3.08	3.34	15.32
The Associated Press	1.84	3.29	2.82	6.68
Barra	4.05	5.45	3.87	6.21
Johnson	5.55			29.53
Sports Illustrated	1.16	2.55	3.61	7.05
USA Today	1.26	2.03	4.45	8.84

Conclusions

Expert forecasts outperform econometric forecasts of Olympic medal tallies. Expert forecasts are often based on within-Olympic period results (e.g. in World Cups). Kuper and Sterken use a mix of econometric and World Championship results and clearly outperforms the other econometric study.