Is there a regional bias in the elo rating ?

An interesting question popped up in the discussions around my World Cup simulations and more specific the somewhat lower than expected predicted performance of Senegal compared to home nation Qatar in group A. 

Amir commented:

“Elo indeed has a regional problem – since there are not many matches between teams from different confederations, it takes a long time for ELO to adjust to the actual average per confederation.

If the difference are getting smaller (which is something worth testing), it means that Elo will indeed under-estimate CAF.”

Apart from the difficulty to determine the “actual average per confederation”, I personally have the impression that elo ratings are not regionally biased, because there isn’t anything in the elo calculation that’s influenced by the team’s confederation. But maybe, indeed the number of intraconfederational versus interconfederational matches a team playes could be the exogenous factor. I decided to give it a go and research this.

• elo-ratings of both teams in a match are adjusted according to the difference in elo-ratings between both teams before the match with the home team, having home field advantage, receiving an extra 100 rating points. The rating points one team wins are lost by the other in all circumstances: it is a zero-sum system with a constant number of points in the system, unless a completely new team is introduced to the ranking;
• in case a new team is introduced in the system a starting points number is estimated for the team. Within some 30 matches the points total for the new team is finetuned against the points of their opponents in those matches and their points total will, from that point on, reflect their ‘true’ strength;
• every match of the National Team of each country counts, at least: to the discretion of the maintainer of the elo-system. For instance the matches in the African Nations Championships where countries are only allowed to select players from their own domestic league, are excluded. On the other hand, matches of Senegal in the last COSAFA Cup, where they played as a guest without their European-based stars, were included;
• 5 different types of matches with corresponding weights are applied:
• 60 – World Cup finals (WC);
• 50 – continental championship finals (EURO, Copa America, Gold Cup and the Nations Cups of AFC. CAF and OFC) and major intercontinental tournaments (the former Confederations Cup);
• 40 – WC and continental qualifiers and major tournaments like f.i. the Arab Cup, COSAFA Cup, the regional championships in Asia (East, South-East, South, West), Gulf Cup etc.;
• 30 – all other tournaments;
• 20 – friendly matches;
• also the goal difference in a match has an effect on the exchanged rating points;
• see http://www.eloratings.net/about for all the details.

I selected from my match database all matches played since January 1st 1980 which are included in the elo rating calculation and determined the match type and confederation of each participating team. There were in that time frame three confederation changes:

• Israel switched per 1/1/1994 from AFC to UEFA;
• Kazakhstan switched per 1/1/2002 from AFC to UEFA;
• Australia switched per 1/1/2006 from OFC to AFC.

The distribution over the match types in this sample:

Here are tables with the intraconfederational (own confed) versus interconfederational (other confed) matches per confederation and match type. Note that per confederation the matches are counted with at least one team from that confederation involved. This means that the line with the totals shows not the summed number of matches per confederation against other confederations but only half of it:

Observations:

• in general 85% of all matches is intraconfederational;
• CONMEBOL with their 10 member NT’s plays the most against teams from other confederations (56%);
• CAF with their 54 member NT’s plays the least against teams from other confederations (16%);
• in World Cup matches (60) the interconfederational percentage is logically (because partly forced in the group stage) the highest: 74%;
• continental finals (50) and qualifiers matches (40) are by nature the match types with the lowest percentage interconfederational matches (resp. 12% and 1%);
• the ‘friendly’ match types (30 and 20) show higher percentages interconfederational matches again with resp. 25% and 31%.

I determined for each year from 1980 until 2022 the compete elo-ranking on January 1st, so position and rating points per team. No longer existing teams were, where applicable, renamed to their current name: West-Germany became Germany, Burma became Myanmar etc. Otherwise no longer existing countries (f.i. East-Germany) and also non-FIFA members are left out of this analysis.

With the dissolution of the Soviet-Union, Czechoslovakia and Yugoslavia a lot of new European countries were created in the beginning of the nineties. I followed the customary line of reasoning in this and considered respectively Russia, Czech Republic and Serbia the rightful successors of these former states.

In order to be able to analyze trends in average elo ratings for the top-x teams per confederation I determined per confederation a ranking of teams based on the average elo ranking position of each team over all 43 year rankings:

In a graph:

Below are the graphs with the average elo rating over the top-x teams in each confederation.

For the top-10 teams:

For the top-20 teams:

For the top-30 teams:

Observations:

• UEFA is in all top-x categories the confederation with the highest average rating over all the analyzed years;
• the top-10 teams average rating for AFC, CAF and CONMEBOL lies close together over all the years;
• when we take more teams in consideration CAF and AFC drop a little in average rating (logically) compared to CONMEBOL. You can see that AFC climbs towards the CAF average rating for the top 20 teams during the eighties and nineties;
• for the top-30 teams you can see that CAF has an consistent higher average rating than AFC;
• for the top-10 teams CONCACAF is already lower in average rating than AFC, CAF and CONMEBOL and drops even further away when more than 10 teams are considered.

Another research aspect is the trend over the years in rating difference between for instance UEFA and the other confederations. I’ve taken UEFA as reference because you can see above that their average rating over the years is nearly constant.

I’ve put a linear trend line in each graph to easily see if the rating difference is growing of shrinking over the years. For these graphs I consider the average ratings for the top-30 teams so it’s only relevant for CAF, AFC and CONCACAF:

Observations:

• the rating difference for top-30 teams between UEFA and AFC is definitely diminishing over the years;
• for CAF it is slightly growing;
• for CONCACAF it is rapidly growing.

Conclusions: