A case for (and against) multi-dimensional measures

I am a vocal critic of the use of metrics to evaluate individuals, single scientific papers, journals, sub-fields, institutions, ....
However, my problem is really one of abuse. I don't think metrics are totally meaningless or useless. Rather, it is the mindless use of metrics, with a disregard for their limitations, that is a problem.

This post is not about metrics, jobs, and funding. I have probably already written too many posts on that. Rather, I want to give two examples where I have found some multi-dimensional metrics helpful, when considering issues relating to public policy and development, particularly in the Majority World.

The case is that of the HDI (Human Development Index). Prior to its introduction people tended to use GDP (Gross Domestic Product) as a measure of how a country was performing and where it ranked in the world. In contrast, the HDI is a composite metric, factoring in income per capita, life expectancy, and education. The map below gives a sense of how the HDI varies around the world.

There is a lot one can learn from just the map.  Sub-saharan Africa is the worst as a region. Even though India now has a middle class of several hundred million people, it is still comparable to some African countries.

Whenever I need to know something about a country, I look at the HDI. The fact that Australia often ranks in the top 3 tells me what a privileged environment I live in. Unfortunately, too many Australians really don't know or appreciate this.

I recently met a medical doctor from Niger [which I knew nothing about it]. He told me that Niger is ranked 182 out of 182 countries! This quickly gave me a sense of some of the challenges he faces.

Obviously, like any metric it has limitations. For example, some people prefer the IHDI (Inequality-adjusted HDI). The USA ranks 25th on the HDI.

The second example of a multi-dimensional metric concerns broader issues than human development, that is "human flourishing". This often means quite different things to different people. Last year there was a nice paper in PNAS that argues why this is important for both public policy, but also research in medicine and social sciences.

On the promotion of human flourishing 
Tyler J. VanderWeele

The abstract gives an excellent summary.

Many empirical studies throughout the social and biomedical sciences focus only on very narrow outcomes such as income, or a single specific disease state, or a measure of positive effect. Human well-being or flourishing, however, consists in a much broader range of states and outcomes, certainly including mental and physical health, but also encompassing happiness and life satisfaction, meaning and purpose, character and virtue, and close social relationships. The empirical literature from longitudinal, experimental, and quasi-experimental studies is reviewed in an attempt to identify major determinants of human flourishing, broadly conceived. Measures of human flourishing are proposed. Discussion is given to the implications of a broader conception of human flourishing, and of the research reviewed, for policy, and for future research in the biomedical and social sciences.

Broadly, when trying to describe and understand complex systems one should search for some measures of the properties of the system. Given the systems are complex one may need several measures. These will never be complete or perfect. But, provided one uses them with the appropriate caution this is a good thing.

How much background material do beginning graduate students need to master?

I am working with a graduate student beginning research and she has asked this important question. I don't think there is a simple universal answer.

Background material includes review articles of a field, details of an experimental technique or computer code, details of derivations, seminal articles on the topic, ....

At the UQ condensed matter theory group meeting, we had a brief discussion about the question.
Answers from students, both beginning and advanced, were helpful. It also underscored how important the question is because students really do struggle with this issue. One shared how he developed some mental health problems because at the beginning of his Ph.D. he was too obsessive about understanding all the details. The question and discussion underscored to me how we need to have more discussions of this nature.

Beginning research is a difficult transition for most graduate students. When they were undergraduates they often could understand all the details and work through all the derivations.
(They are unlike a significant fraction of undergraduates who just don't seem to realise that the details DO matter.)
However, the painful reality is that what was possible for a gifted and motivated undergraduate is simply not possible for most Ph.D. research.
Research fields are so vast and have so much foundational material a student simply does not have the time to check everything and understand everything in full.
The question is painfully relevant in Australia because Ph.D. students do not do coursework (or a Masters degree) and the government continues to reduce the number of years of funding.
Furthermore, the "publish or perish" culture puts pressure on students and advisors to be cranking out papers, which means there is pressure for students not to ``waste time'' on slow and deep learning of background material.

Like many things in life, I think answers to the question require some balance and need to allow for differences in personality, learning styles, personal goals, and nature of the research topic.

Here are some composite pictures to illustrate the extremes and the associated problems and potential.

Sanjay loves to understand and master details. He is also interested in the big foundational questions the research might address. When he reads an article he likes to work through all the details of the mathematical derivations. He would prefer to write his own computer code so he really knows what is going on. He has a large stack of papers on his desk, waiting to be read, consisting of many of the papers related to his research topic. After a year he is still learning background material. However, in his third year, he has a big breakthrough because he realises that one a key assumption/derivation in the field is wrong in certain cases. He not only corrects it but opens up a new avenue of research.

Priya just wants to get on with research and is not a detail oriented person. Following her advisors request she reads a few background articles superficially and dives into research. However, she does not really grasp the big picture or understand the limitations of the technique she is using. Consequently, she wastes a lot of time making mistakes, producing dubious results, and getting help for things she should have worked out for herself. However, this approach actually suits her learning style and she does eventually learn the essential things she needs to know and understand what is going on. Furthermore, because she has "dived in'' early, by the end of her Ph.D. she has produced several nice papers.

What do you think?
It would be good to hear from beginning graduate students, advanced graduate students, and faculty advisors.
What did you do? What do you wish you had done?

Universality, probability, and the growth of rough surfaces

On Friday there was a nice UQ Maths and Physics Colloquium, Beyond the Gaussian Universality Class, given by Ivan Corwin,
The talk was a very nice example of synergy between fundamental physics and maths research.
There are interesting connections with simple one-dimensional models for surface growth, the Kardan-Parisi-Zhang equation, the KPZ universality class, traffic models, random matrix theory, directed polymers in random media, ....

Some hypotheses about universities

In the next month, I have been asked to give a talk and to write an article about universities in two different forums. What are universities for? How do they promote human flourishing?

Before I get too carried away I thought I would float a few ideas/claims/hypotheses that will be central to my argument that there needs to be a greater debate about fundamental issues and about the history of universities.

Some of the claims are interconnected.
In future posts, I may expand on some of these claims.

Universities are currently having a crisis of identity, mission, and purpose.

This crisis arises because there is a multitude of competing and conflicting visions from a range of "stakeholders".

This crisis and the degree of conflict is far greater and deeper than those faced by other institutions: government, schools, hospitals, business, charities, ...

Over time universities have been one of the most successful human institutions for promoting human flourishing (broadly defined) in many different ways: training leaders, science that produces useful technology, enriching cultural life, ...

Universities are a victim of their success.
Their current struggles follow the natural evolution of successful and growing organisations. 
Success increases the size, complexity, and cost of the organisation. Success also attracts "hangers on" who want a piece of the action: money, power, and social status. They do not have the same vision as the founders and first few generations of builders of the organisation. Their focus is more on their own agenda and interests than on the "common good" that the organisation originally sought to serve. This is a large part of the origin of the competing and conflicting visions.
Success also leads to the iron triangle of cost, access, and quality.

Here I expand on the first few claims.

Universities are currently having a crisis of identity, mission, and purpose.
I have written about this before. I think it is nicely illustrated by the diverse voices that claim this.

Universities on the Defensive 
Hunter Rawlings, a former President of Cornell, and currently the President of the Association of American Universities, a consortium of 60 of the leading North American universities.

The Slow Death of the University
Terry Eagleton, a distinguished literary critic, former Oxford Professor, and Marxist.

Higher Education is Drowning in BS,
by Christian Smith, a sociologist at the University of Notre Dame.



The crisis is not just about research universities in Western countries but also smaller teaching institutions and large state universities in the Majority World. Last year I was involved in a range of consultations run by a global NGO, identifying big issues in universities, and it was surprising how often this question, ``What is a university for?" came up.

This crisis arises because there is a multitude of competing and conflicting visions from a diverse range of "stakeholders".
The latter includes faculty, administrators, students, parents, alumni, future employers, politicians, taxpayers, funding agencies, donors, ...
The conflicting visions include neoliberalism, job training, a finishing school for the privileged, nation building, sectarian religious, social critique, scholarship, social transformation....

This crisis and the degree of conflict is far greater and deeper than those faced by other institutions.
One might argue that many public institutions (government, schools, hospitals, business, charities, ...) are currently in turmoil. However, most of these conflicts are about funding, governance, internal codes of conduct, and how to respond to rapid social and technological change. They are not conflicts about the primary purpose of the institution. One might also claim that as societies have become more multi-cultural, more politically divided, and complex this crisis just reflects, the many competing voices in the public square at large.
However, I would contend that universities are one of the most contested public institutions in society today. There are comparable debates about funding and access to health care. However, everyone agrees on the mission of hospitals: to help cure sick people. There is little debate about the goal, about the methodologies, or about the relative importance of different sub-fields of medicine. In contrast, in universities, there is significant contention about what the actual primary mission is and of the relative importance of different academic disciplines. No one proposes a large hospital without a pathology or oncology department. However, there are people who think humanities now have little to contribute to universities.

What do you think?
Which of my claims do you disagree with?

Bad metallic behaviour in ultracold atoms

There is a nice paper
Bad metallic transport in a cold atom Fermi-Hubbard system 
Peter T. Brown, Debayan Mitra, Elmer Guardado-Sanchez, Reza Nourafkan, Alexis Reymbaut, Simon Bergeron, A.-M. S. Tremblay, Jure Kokalj, David A. Huse, Peter Schaus, and Waseem S. Bakr

The paper represents a significant experimental advance in using ultracold atoms to investigate questions directly relevant to strongly correlated electron systems. In this case, the system Hamiltonian can be tuned to be a Hubbard model on a square lattice, such that the model parameters, U and t, and the doping, n are known.
One limitation is that current experiments can only be performed down to the lowest temperature of T/t =0.3. [For comparison, for cuprates this is of the order of 1000 K!].
Using imaging techniques the authors are able to directly extract the density (charge) diffusion constant D and the density susceptibility, chi, shown below. The experimental data are red dots. The blue curve is the result of calculations based on the Finite Temperature Lanczos Method (FTLM). Green dots the results of Dynamical Mean-Field theory. Gamma is the density relaxation rate.

Both the experiment and the ability to make such a detailed comparison with concrete theoretical calculations is a significant and exciting achievement.

The dashed curve in the upper panel is the value of the diffusion constant associated with the Mott-Ioffe-Regel limit below which one expects bad metal behaviour.
Aside: one should always keep in mind that for the MIR limit, different authors use different criteria, leading to different factors of pi, sqrt(pi), ...

Using the Nernst relation, sigma = chi * D,  the data above gives the conductivity (sigma) and resistivity (rho), shown below.

The blue and green curves correspond to the predictions, of FTLM and DMFT, respectively. The dashed grey line is the Mott-Ioffe-Regel limit.

One comment I have concerns an additional comparison that the authors could make. Based on heuristic arguments and results from AdS-CFT, Hartnoll conjectured a lower bound for the diffusion constant, hv_F^2/T.
Previously, Nandan Pakhira and I showed that this bound was significantly violated in the bad metallic regime, as described by DMFT.

There is also a commentary on the paper by Ehud Altman at the Journal Club of Condensed Matter.
I thank Matt Davis for bringing the paper to my attention.