Another Index. Armenia is #95. Meaningful?

The Legatum Institute, a London-based think tank, released its annual “Prosperity Index” and, as reported by PanArmenian, Armenia came it at #95.

Credit: rifreedom.org
Credit: rifreedom.org

After looking through the data, I can say that it is varied enough that it presents what I’ll call a picture of general understanding. That is, the use of reports like these is less what rank a country achieves and more about giving the reader an idea of where the country falls on a wide spectrum. In the case of Armenia, the prognosis looks to be that it has a lot of room for improvement.

Otherwise, the “index industry” (c) William Bairamian, is in full swing. Every think tank and their mother (do think tanks have mothers? the jury’s out but I’m going with yes) is publishing an index of something: happiness, corruption, poverty, water, sex, environment. Some are useful, some are an exercise in finding data no one else has yet manipulated and putting it in digestible form to keep up with the think tank Joneses. The Legatum report falls somewhere in the middle.

They aggregate dozens of pieces of data, including polling data from Gallup, so the ranking gives a more holistic picture than something like “X is the happiest country in the world and Y is the unhappiest based on our evaluation of data about the average distance a citizen must travel to the nearest lollipop stand.”

However, strongly lacking is any non-quantitative, country-specific analysis. The whole report is neatly based on an evaluation of numbers but an overreliance on numbers doesn’t tell the whole story. They do recognize this, though insufficiently, in my opinion. Their section on Prosperity Index Anomalies grants the inconsistencies that the data may belie; it deserves more attention.

Here are a few of the issues, in order of importance:

1) Lack of qualitative analysis

This is a major problem with these indices: everything is reduced to data. Culture, for example, has no bearing on the evaluation of the results, probably because it cannot be dependably quantified.

To make the point, here is a scenario:

You’re Armenian, you go to someone’s house and they ask you if you want something to eat. Almost invariably, you will refuse the first, maybe, thousand times they ask.

This cultural peculiarity in answering questions is not represented in polling data although it may well significantly skew responses in one direction. And when a significant part of your conclusion is based upon polling data, this can have a huge impact when that information is whittled down to a number. 95, perhaps?

2) Autocratic countries

As noted in their Anomalies section, it’s hard to conceive that people in an autocratic country might answer questions fully truthfully. A conversation might go like this:

Pollster: “Do you feel you’re able to freely express yourself?”

Citizen who believes he might be asked a similar question by undercover police to determine who isn’t saying the right things so they can get them to say the right things…in an empty room…with no cameras: “Obviously! We’re freer than hell! I mean, not just hell, everywhere! Long live our nation! Can I go home now?”

Pollster: “Of course you can. I’m just a pollster. I can’t prevent you from going home.”

Citizen: “Sure. Last time a guy told me that, I woke up without any clothes and pain in my, er, uh, my face! Because I was smiling so much! Have a nice day!”

Credit: times.co.uk
Credit: times.co.uk

Conversely, and curiously, when a country has greater freedom of expression, citizens may use that freedom to comment on the country’s lack of freedom more freely.

3) Unrecognized countries

This is an issue when, for example, the people of Artsakh aren’t polled. Comprising a sizable part of the population of Armenia, about 7-9%, it may have had an impact on the results, which Legatum seems to acknowledge.

Similarly, if they did the survey 25 years ago and asked Armenians living in Artsakh, then under Azerbaijan SSR, “How do you feel people of other ethnicities are treated in this country?”, I’m going to guess that would’ve affected their score a wee bit.

4) “Data Lag”

Not all the data is current. Some data points are from 2012 or, even 2011. Kind of a big deal for an annual prosperity index.

It’s not perfect. Nowhere close to it. But it has some uses. I think it’s better to explore the data that’s conveniently summed up in one place and beautifully presented rather than getting hung up on the ranking which, in a defiance of the laws of mathematics*, is less than the sum of its parts.

Full Prosperity Index Report.

* My knowledge of the laws of mathematics is only surpassed by, well, my knowledge in pretty much everything else.

By: William Bairamian

[twitter-follow screen_name=’bairamian’]

Armenia Does More With $1 In Education Than U.S., Norway

The Huffington Post reported today on a study that compares the math and science skills of students in American states and countries around the world.

United States Math Scores

According to the infographic above, Armenia appears to have received the ignominious distinction of being among the worst. The Anniston Star, that I’m going to go out on the limb and guess less Americans know than even Armenia, noted the “bad news” and seemed a bit disgruntled about being in the same category as Armenia.

Ever conscious of the possibility of the Armenian community to collectively bury its face in its hands from shame or call for the heads of everyone in the Armenian government, I wanted to take one step beyond the basic infographic above and lend some context.

However, I would like to note that there is no excuse for not pursuing excellence and I like to think I’m fully aware of the government’s shortcomings. I just happen to think that the best approach is not always bludgeoning each other to a pulp and then expecting that things will get better.

1) The comparison is based on the results of two different tests.

This is relevant because it seems the two tests are being compared in similarities in length and type of question and not on qualitative similarities. Nevertheless, this isn’t so much a problem of how the countries in the world (except the U.S.) compare to each other as it is with how U.S. states compare with those countries. 

2) Armenia scored 467 in mathematics. But is that bad? Compared to Japan, which scored 570, yes. But how about compared to some others? Well, here are some countries Armenia scored better than in mathematics:

Bahrain: 409

Iran: 415

Chile: 416

Georgia: 431

Lebanon: 449

Turkey: 452

Romania (EU): 458

UAE: 456

And although it didn’t score higher, Armenia was just below that loser of all losers in the international community, Norway, who racked up a whopping 475 points.

Oh, and Norway spends $4,595 of public money per capita on education. That’s $4,396 more than Armenia, which spends $199 per capita. Which brings me to the next point:

3) Nothing to brag about but Armenia spends significantly lower per student than Alabama and pretty much everyone else.

Per capita (not per student) public spending on education:

(I used the percentage spent as reported by the NCES study, country GDP/capita as reported by the World Bank on Wikipedia and state GDP/capita on Wikipedia.)

Armenia: $199

Alabama: $1,090

California: $2,076

Massachusetts: $2,324

United States: $2,998

Norway: $4,595

South Korea (highest score in mathematics): $1,231

Turkey: $734

Georgia: $177

UAE: $420

So, per dollar spent, Armenia has a much higher return in terms of test points.

Which leads me to the following conclusion:

4) Armenia may be the most efficient educator of children in the world.

Or darn close, according to this study.

To make the point (pun!), Armenia spends about 43 cents per point on the mathematics exam.

South Korea which scored highest in mathematics, spends $2 per point received.

Alabama, ranked the same as Armenia, spends $2.34 on education for every point on the mathematics test.

The United States, with an average score of 507, spent $5.91 per point.

Of course, relativity here does not count for much because if a student performs better on the test it’s better than if they didn’t perform well on the test. South Korean students are the best at mathematics and if it took spending $2 per point on education, so be it.

But when you look at Alabama spending almost two dollars more than Armenia and still getting the same score, it suggests that unlike Alabama, Armenia is doing something right, even if it’s not everything.

(Alabama, looks like you need to go to the emergency room because I just burned you.)

It is further encouraging to know that given the apparent ability of the education system of Armenia, a still-developing economy, to score as well as it did given scarily low spending on education, there is a good likelihood that things will get better before getting worse.

5) Ah, but what of the science, you say? After all, the tests were in mathematics AND science. Well, Armenia scored 437 in science which is admittedly among the lowest scores. But, being the optimist that I am, there is a silver lining:

Georgia: 420

Maybe if Georgia spent some money on the Armenians in Javakhk, it could up its scores the next time test time comes around.

And although the Anniston Star (from that place in Alabama) suggests that politicians from low-performing states should be sent to high-performing states, maybe they should send them to Armenia instead. It has Armenians just like Massachusetts does – and the food is better.

The full NCES report with all the data is here.

By: William Bairamian

[twitter-follow screen_name=’bairamian’]