Calculating Average Happiness is Actually Pretty Simple

A Few Robustness Checks

Some background- myself and Alexander de Vries, whose blog I heartily recommend, wrote this together.

Suppose you - for whatever reason (policy, curiosity, search for retirement spots) - wish to know whether Denmark or France has happier citizens on average. Thanks to the tireless efforts of sociologists and government bureaucrats, you have access to a (probably) representative sample of answers to the question “Taking all things together, how happy would you say you are?”, from citizens of both countries.¹

Respondents have been asked to answer on a 0-10 scale, where 0 is labeled “extremely unhappy” and 10 is “extremely happy”. How do you calculate?

The naïve answer is: For each country, sum up the answers and divide by the number of answers. In general, that’s how a psychologist would approach this problem. Economists, though, are more sceptical. They worry that while a 9 is greater than an 8 on the happiness scale, and an 8 is greater than a 7, the distance between them may not be constant. Maybe as scores get higher, the gaps are getting larger. Or maybe the opposite is true, and the chasms of misery between a 1 and a 2 are vastly larger than the jump from 8 to 9. Essentially, the psychologists believe that respondents treat happiness scales as pretty much cardinal, while economists say all we can know for sure is that the responses are ordinal.

The economists’ ordinal interpretation is a problem for us, because if true, it means the naïve approach outlined above might not work. If the distance between two points on the scale isn’t always the same, it’s at least somewhat inaccurate to sum up an 8 and a 5 in the process of calculating the mean.

There have been various attempts to deal with this problem in the literature by trying to prove cardinality, e.g. as many authors point out, happiness scores tend to correlate linearly with important variables. Consider, for example, this paper [widely mocked at the time it was published due to a perception it didn’t do anything which hadn’t already been done] which illustrates that various sentiments correlate linearly with various behaviors: https://pnas.org/doi/full/10.1073/pnas.2210412119…

But suppose there are some deviations from linearity in the relationship between happiness and happiness scores? How much wiggle room do we have here? If happiness scores and happiness aren’t perfectly correlated, are we in real danger of drastically underestimating or overestimating the rank of a country relative to other countries, or the rank of a group relative to other groups? Could this lead to misinformed policy from a utilitarian perspective?

Philosophy Bear and I set out to test this question for a set of most countries in Europe.²

The short answer is that the scale is extremely robust to violations of the assumption of cardinality. After having computed the ordinary means, we then computed the means of the happiness scores squared and looked at the correlation between those two metrics. We found that the correlation was r > 0.99; weighting higher scores more hardly change the results at all!

But what if it works the other way? What if the gap between a 0 and a 1 is huge, but the gap between a 5 and a 6 is smaller, and the gap between a 9 and a 10 is minimal? To simulate this, we took everyone’s happiness score, subtracted 10 from it and squared it. We called this the misery score. Hence a 0 equals 100 points of misery, whereas a 9 represents 1 point of misery and a 10 no points. Now the correlation fell slightly… to r > 0.98. So despite imposing pretty significant deviations from linearity, we once again found very little difference. Negative utilitarians can rest assured that the naïve method of calculating average happiness won’t ignore those who are suffering the most.

[Our changes also had virtually no effect on the rank order of countries - the Spearman rank correlations were similar in both cases to the Pearson correlations; our transformations moved some countries by a place or two, but that’s it.]

But let’s indulge in a bit of scientific paranoia: what if squaring is not enough? What if the real relationship between happiness and happiness scores is even more left-skewed or right-skewed? We tried cubing the scores - a transformation we take to be vastly beyond the plausible range of values - one that would indicate either that there is more distance between 8 and 10 than between 0 and 8, or that there is more distance between 2 and 0 than between 10 and 2. The correlation remained strong at r >.91 for straight happiness cubed scores, and r > 0.95 for misery cubed scores.

In short, it seems pretty clear that even if happiness scales were very non-linear in either of the ways we considered, it empirically wouldn’t impede the use of a “naïve” arithmetic-mean method of calculating a country’s average happiness score- at least as far as rankings go. There may be other psychologically plausible transformations that do result in more significant changes, but if so let them be brought forth.

[Note: We have only calculated this for European countries, but due to the strength of the correlation we expect it to apply in other high-income nations and probably also in middle-income nations. Poor countries could well have a very different happiness distribution conditional on mean happiness, due to high variance in extreme conditions such as war, famine, natural disasters, etc. For example, we might expect the misery index to be less strongly correlated with mean happiness in poor countries. Anyone who has the survey data is encouraged to explore the application of these results to other countries.]

The sad truth about happiness?

Finally, we should say a word on Bond & Lang (2019), titled “The Sad Truth About Happiness Scales”. Bond and Lang object to the practice of making assumptions about average happiness using Likert scales because these scales top out at 10 and bottom out at 0. What if, for example, the vast majority of variation in the scale is between the degrees of ecstasy different 10’s feel? I (Philosophy Bear here with this take) would urge ignoring it, because it’s silly. Yes, it’s possible mathematically, but mathematical possibilities by themselves do not prove plausibility. It’s just a priori implausible that there is more variance in 10 then there is between 0 and 9, or vice versa for 0. At one point Bond and Lang say: “In the Online empirical appendix we further show that nearly every result can be reversed b a lognormal transformation that is no more skewed than the wealth distribution of the United States… To be clear, we are not proposing that satisfying this minimal criterion would make a result convincingly robust. It is plausible that happiness is more skewed than wealth is.”

To this, I would say stop being silly now, please. Let’s have a look at how skewed the United States wealth distribution is: 

There are good theoretical reasons to think happiness cannot be this skewed. Happiness is an evolved mechanism intended to regulate our behavior towards evolutionary fitness. Why on earth would we evolve in such a way as to allow some people to be millions of times happier than others? All human inquiry, but perhaps especially the human sciences, begins in media res: we assume that we know something about how people work. It’s reasonable to entertain the possibility of deviations in cardinality happiness from happiness scores, but common sense can and should inform how much deviation we think is plausible.

---

Footnotes

1

And also the rest of Europe.

2

We chose Europe because it has the highest ratio of government-bureaucrats-to-population, and therefore some pretty damn good survey data collection: https://ess-search.nsd.no/en/study/bdc7c350-1029-4cb3-9d5e-53f668b8fa74