However, my problem is really one of abuse. I don't think metrics are totally meaningless or useless. Rather, it is the mindless use of metrics, with a disregard for their limitations, that is a problem.
This post is not about metrics, jobs, and funding. I have probably already written too many posts on that. Rather, I want to give two examples where I have found some multi-dimensional metrics helpful, when considering issues relating to public policy and development, particularly in the Majority World.
The case is that of the HDI (Human Development Index). Prior to its introduction people tended to use GDP (Gross Domestic Product) as a measure of how a country was performing and where it ranked in the world. In contrast, the HDI is a composite metric, factoring in income per capita, life expectancy, and education. The map below gives a sense of how the HDI varies around the world.
There is a lot one can learn from just the map. Sub-saharan Africa is the worst as a region. Even though India now has a middle class of several hundred million people, it is still comparable to some African countries.
Whenever I need to know something about a country, I look at the HDI. The fact that Australia often ranks in the top 3 tells me what a privileged environment I live in. Unfortunately, too many Australians really don't know or appreciate this.
I recently met a medical doctor from Niger [which I knew nothing about it]. He told me that Niger is ranked 182 out of 182 countries! This quickly gave me a sense of some of the challenges he faces.
Obviously, like any metric it has limitations. For example, some people prefer the IHDI (Inequality-adjusted HDI). The USA ranks 25th on the HDI.
The second example of a multi-dimensional metric concerns broader issues than human development, that is "human flourishing". This often means quite different things to different people. Last year there was a nice paper in PNAS that argues why this is important for both public policy, but also research in medicine and social sciences.
On the promotion of human flourishing
Tyler J. VanderWeele
The abstract gives an excellent summary.
Many empirical studies throughout the social and biomedical sciences focus only on very narrow outcomes such as income, or a single specific disease state, or a measure of positive effect. Human well-being or flourishing, however, consists in a much broader range of states and outcomes, certainly including mental and physical health, but also encompassing happiness and life satisfaction, meaning and purpose, character and virtue, and close social relationships. The empirical literature from longitudinal, experimental, and quasi-experimental studies is reviewed in an attempt to identify major determinants of human flourishing, broadly conceived. Measures of human flourishing are proposed. Discussion is given to the implications of a broader conception of human flourishing, and of the research reviewed, for policy, and for future research in the biomedical and social sciences.Broadly, when trying to describe and understand complex systems one should search for some measures of the properties of the system. Given the systems are complex one may need several measures. These will never be complete or perfect. But, provided one uses them with the appropriate caution this is a good thing.
1-dimensional measures are surely problematic, but a multi-dimensional measure requires individuals to consider what weight should be used for each dimension (more generally, one might want to use a nonlinear function of n variables). An appointment might hang on whether number of articles in Nature or of articles in Science counts more heavily for a majority of people on the committee. In that case, people might factor their weights for each dimension to obtain the result they want for reasons along dimensions that are not included in the allowed list. It seems that a formal method that cannot be easily gamed but that still allows some freedom to weight dimensions differently for different cases would have to be quite elaborate.
ReplyDelete