Ranking respect: Can teacher status be compared across countries?

By Katherine Aleynikova

The more the teachers in a country are respected, the higher the academic results are amongst their students, or so the Global Teacher Status Index 2018 claims to prove. As instinctively plausible as the argument is, and as easily agreeable (who would argue against respecting teachers?), I have to wonder how exactly is such a complex, culturally-variant idea compared meaningfully across many different contexts?

Published in December by the Varkey Foundation, the University of Sussex and the National Institute of Economic and Social Research, the GTS Index report is typical of a trend in educational policy discourse. It produces a single measure to represent a multifaceted concept, and then ranks 35 countries by their respective indices. This is done in the spirit of international competitiveness and prioritisation of measurable outcomes, prevalent in educational policy discourse largely due to the work of international organisations in producing global rankings of educational systems. The most prominent of such international comparative tests – PISA produced by the OECD – is used in the GTS Index as a measure of student achievement, which in itself is indicative of the report fitting in the trend. In global educational policy discourse, there is a strong belief in quantitative data as a superior source of truth, as well as a perceived necessity of comparison and competition for progress. While quantitative data has its obvious advantages, such as a wide scope for generalisability, it can have its pitfalls, too – and the GTS Index is a clear example of that.

The report is built upon questionable methodology. Based on an online survey of the general public, the GTS index measures teacher respect relative to 14 other professions, such as doctor, librarian and social worker. The selection of other professions used in the index is arbitrary, influenced by a number of assumptions made by the researchers, such as their sweeping judgement on “how similar or dissimilar the work might be”. The researchers also do not take into account any potential variation in respect for other professions across countries, which would skew the results for the status of teachers.

The imperfections of the index are obvious from the discrepancies in the report’s own results. For example, Russia comes very highly in the overall teacher status ranking – it is fourth out of all countries surveyed. At the same time, ranked by parental encouragement – how likely a person is to encourage their child to become a teacher – Russia is bottom of the list. If both measures are devised to provide an insight into respect of teachers in the country, why is there such a large discrepancy?

Putting the imperfections of the index itself aside, the report’s claim of causality between GTS and student performance is unsound, since the report is only able to show a correlation. Causality could potentially lie the other way around – teachers could be more respected if their students score highly in the international comparative tests, providing the public with evidence that teachers are doing a good job. Or, there might be no causal links between the two measures, if both are caused by some other unaccounted factor.

But most importantly, nowhere in the report did I find a discussion of what respect actually means. It is, however, crucial to understand what the concept stands for, firstly, in order to avoid comparing apples with oranges, and secondly, to provide meaningful policy suggestions. Surely, what actually matters for student success is not the respect ranking by the general public in itself, but rather the way these attitudes translate into students’ experiences inside and outside of the classroom?

In the design of the index, it is assumed that cultural factors are what influences the levels of respect in different countries, respect being the same in its nature but varying in extent. In reality, the concept of respect itself can have different meanings across cultures. For example, coming from a Russian cultural background, I see that in school context respect is often conflated with fear, and demonstrated by students through behaviours like obedience, silence in the classroom or not voicing disagreement. In a different cultural context, respect can mean something else. A student can respect a teacher as a role model, as an opponent in debate, or as a partner in the process of learning. Respect can be understood as mutual or as a hierarchical concept that a student has to pay to their teacher and not the other way around. Respect can be viewed as a reward for exceptional work, in which case only the best teachers would be seen as worthy of respect or, on the contrary, respect can be seen as a basic virtue that everyone is entitled to. Students may also respect the time and effort that teachers put into their work, and show that through diligence, for example submitting homework on time.

It is clear that multiple conceptions of respect may exist at the same time, and some of them may even be mutually exclusive. This does not necessarily imply that it is impossible to compare respect across contexts. However, any quantitative data needs to be supported by careful qualitative analysis, defining precisely what respect means for the purpose of a particular study, and how this idea translates into educational practice. Otherwise, numbers risk being meaningless, undermining evidence-based policymaking.

Why is it then that the authors of the GTS index report decided to use solely quantitative methods for what is fundamentally a qualitative question? In my opinion, this is precisely because of the tendency of unquestioning trust in numerical data. The authors set out with an intention to prove their initial belief regarding the importance of respect. Clearly, they felt that the most convincing way to do so would be to produce a numerical index and to rank a large number of countries accordingly. However, their findings do not prove their hypothesis, since there is no substantive understanding of what respect behind the numbers.

While the desire to learn from other countries in educational discourse is promising, it is not enough to suggest that countries should imitate those who score highly in international comparative tests. Instead, the questions we should be asking are: what is it about the classroom practices, and day-to-day teachers’ experiences, that makes some systems better at achieving certain goals than others? What are the priorities of a given educational system, and do they align with what is required to ace a particular test? These objectives are hard enough to determine within one national context, let alone for international comparisons.

This is not to say that measurable indicators are not useful – they are essential for informed policymaking, planning or any sort of generalisable conclusions. However, we need to question what the numbers actually stand for if we are to trust the conclusions.