Evaluation
of London 2012 Olympic medal tally forecasts: Who has won?
Gerard
Kuper and Elmer Sterken
August
2012
At the London 2012 Summer Games 962
medals have been awarded to athletes from 85 countries. With 302 medal events and
53 events awarding two bronze medals the total number of medals at the Olympic
Games is 659. However in some events athletes have tied:
· In the men’s high jump three bronze medals
are awarded.
· In the 200m freestyle swimming for
men two silver medals, and no bronze medals, are awarded.
· In the men’s keirin
track cycling event two bronze medals are awarded.
This makes a total of 302 gold
medals, 303 silver medals and 357 bronze medals.
Medal winning is concentrated: just
as in 2004 and 2008 the Top-8 from the medal tally won about 50% of the medals
total. In this note we compare six forecasts, and focus on the Top-31 since
Poland and Azerbaijan are tied on place 30. These 31 countries take 83% of all
medals.
Forecasts differ
There are two types of medal tally
forecasts: expert’s opinions and model-based forecasts. Examples of the former
are USA Today and Sports Illustrated. These journals name each gold, silver and
bronze medal winner from which we compile the medal table based on the number
of gold medals. Johnson, Williams (using the method of Bernard and Busse), and
Kuper and Sterken all use econometric models to generate forecasts. Barra bases his predictions solely on World Championship
results.
Johnson and Williams predict the
number of gold medals and the total number of medals per country. The others
predict the number of gold, silver and bronze medals won per country. The
predictions also differ with respect to the number of countries considered. Johnson
and Williams have the smallest numbers of countries (33 and 65 countries,
respectively). Kuper and Sterken have 126 countries, with 76 countries winning
at least one medal. USA Today has 79 medal winning countries, and Sports
Illustrated has 82 medal winning countries which is very close to the actual
number of countries that have won medals in the London 2012 Olympic Games.
Before we analyse the performance of
each forecast, we should note that Johnson has a smaller sample of countries (only 33) and misses 1 Top-10
country (Korea) and nine Top-31 countries. Williams only publishes a list of
countries winning at least six medals, and a table of countries winning at
least four gold medals. Out of 20 countries winning at least four gold medals
she identifies only 13. Out of 37 countries winning at least six medals she
identifies only 28. In this respect Kuper and Sterken
are closer to the realisations with 19 countries expected to win at least four gold
medals and 36 counties expected to win at least six medals.
Comparing forecasts
There are various ways to compare
the forecasts. Obviously, one way is to calculate rank correlation coefficients
between the various ranking of countries and the actual ranking. There are some
complications with the publications of Johnson and Williams as discussed above.
For Williams we have to rank the countries on place 14 and below based on the
number of total medals won. In the case of Johnson all non-listed countries are
tied on place 34 with 0 medals. This will affect the Kendall rank correlation coefficients
for these forecasts.
Table 1 presents
Table 1 –
Sports Illustrated 0.525
USA Today 0.576
Kuper & Sterken 0.525
Williams 0.557
Johnson 0.372
Barra 0.525
Because of the issues
discussed above with some of the publications a better way to analyze the medal
forecasts is to compute the distance of the forecasted medal tally to the
actual results. So we compute the Mean Absolute Errors and the Mean Squared
Errors for gold, silver, bronze, and the total number of medals. The results
are in Table 2. Here one can see that the experts of Sports Illustrated perform
best. USA Today and Kuper and Sterken show very similar results. The forecasts
of the other model-based forecasts are worse.
Table 2 – Mean Absolute Errors (MAE)
and Mean Squared Errors (MSE) for the Top-31 countries
MAE |
Gold |
Silver |
Bronze |
Total |
Kuper and Sterken |
2.935 |
2.645 |
2.387 |
4.355 |
Sports Illustrated |
2.742 |
2.065 |
1.935 |
3.387 |
USA Today |
3.032 |
2.581 |
2.806 |
3.903 |
Johnson |
3.419 |
7.645 |
||
Williams |
3.267 |
4.516 |
||
Barra |
3.677 |
2.581 |
2.677 |
4.484 |
MSE |
Gold |
Silver |
Bronze |
Total |
Kuper and Sterken |
16.161 |
15.871 |
9.290 |
28.935 |
Sports Illustrated |
12.226 |
7.355 |
7.355 |
16.548 |
USA Today |
16.258 |
12.387 |
11.903 |
36.032 |
Johnson |
22.710 |
106.290 |
||
Williams |
24.538 |
35.516 |
||
Barra |
20.516 |
15.161 |
12.484 |
44.032 |
Conclusion
Sports Illustrated outperforms
the other forecasts of Olympic medal tallies. Kuper and Sterken use a mix of
econometric and World Cup results and clearly outperforms the other model-based
forecasts. Their forecast performance is similar to USA Today but much better
that the other forecasts.
Some more facts
On August 11, 2012, Gabon won its
first ever Olympic medal when Anthony Obame took
silver in Taekwondo (men's +80kg weight class).
Guatemala won their first Olympic
medal at the 2012 Summer Olympics on August 4, 2012, when Erick Barrondo took second place in the men's 20km walk.
One day after the closing ceremonies
the Belarusian shot putter Nadzeya Ostapchuk has been stripped of her Olympic gold medal after
failing a doping test. New Zealand's Valerie Adams will be awarded the gold, Russia's
Evgeniia Kolodko will take
the silver and China's Lijiao Gong will win the
bronze. Ostapchuk was tested twice, once on 5 August
and again the following day after she had finished first. The IOC said both
samples indicated the presence of metenolone, which
is classified as an anabolic agent.
Significance test
The sampling distribution
of the Kendall rank correlation coefficient converges towards a normal
distribution (the convergence is satisfactory for values of N larger
than 10), with a mean of 0 and a variance equal to [2(2N+5)] / [9N(N-1)].