I Got 99 Problems, But Statistical Triangulation Ain’t One
I often have conversations with people about Internet penetration rates.
The point I’d like to make is that these statistics are complicated and it is hard to get at the “right” number. That’s why we try to triangulate — look at different sources of data to see if things seem right. We also should always assess the credibility of the source of the data.
For this example I chose mobile phone penetration.
Data sources:
ITU is the UN’s official statistics and these numbers come from the governments themselves who usually get the numbers from the telecommunications companies. These companies count number of SIM cards sold and it is not unusual for people to have multiple SIM cards. This is data to be highly skeptical about.
Caucasus Barometer, Gallup, and EBRD are surveys taken face-to-face in households. All use different sampling techniques and are collected by different organizations. None are perfect, but they’re as good as we’ve got. Of the three, I trust Gallup the least.
Noteworthy:
All of these were collected at different times of the year.
Margin of error varies in all of these.
A ~4-6 point difference is within the margin of error and shouldn’t be looked at with too much suspicion.
So what do we see?
– Look at the huge difference between the ITU and the Caucasus Barometer in all three countries in 2004.
– 2006 is a little better, but Georgia’s a little too far off to be left to chance.
– 2007 is a little questionable in both Armenia and Azerbaijan.
– 2008 is really off in Armenia and not great in Azerbaijan.
– 2009 isn’t bad.
– 2010 is all over the place. My thought is that by the time you’d are at more than three-quarters of households having phones, the ones that don’t are also probably the ones that are less likely be be surveyed – the poorest of the poor, for example.
– The 2011 difference between the CB and the ITU is likely due to SIM counting. While a household may own a phone, they may have a lot of SIMs too.
