On the equal validity of methods in Global Health research

Active Ebola case finding in Sinje, Liberia: The temperature of a resident is checked. Photo by UNMEER

Active Ebola case finding in Sinje, Liberia: The temperature of a resident is checked. Photo by UNMEER

By Felix Stein

Students and investigators of Global Health may frequently feel a little lost as to which methods to employ in their research. Should they track the finances of major Global Health organisations or measure treatment success wherever this money is used? Should participant observation be the method of choice or will quantitative analyses yield more relevant results? Should they conduct clinical trials to assess the latest advances in medical research, or may it be preferable to leave the laboratory to observe medical practice in applied settings? These questions are all the more difficult to answer since the field of Global Health itself is highly heterodox, combining several areas of study, various objects of interest and greatly divergent epistemic traditions. Here are two examples.

On the one hand, the term “global” in Global Health points to the fact that this area of study can be considered part of the wider discipline of Development Studies. The history of Global Health “remains predominantly about flows of goods, services, and strategies along well-trod north-south pathways”[i] and most of its publications are in one way or another concerned with healthcare provisions for people living in poverty[ii]. As far as this is the case, Global Health can be considered the medical arm of the wider sets of political, economic and personal relations that the field of Development Studies has traditionally been concerned with. Unfortunately, Development Studies is not known for providing students and researchers with a clearly defined research method. Here, neoclassical economists and quantitatively minded sociologists use surveys and “big data” to learn about equity and growth in the world’s poorest countries, historians conduct archival research that shows how colonialism continues to matter, anthropologists engage in participant observation to make sense of people’s lived experiences, and philosophers rely on a priori deductions to decide whether and how altruism and self-interest, national sovereignty and international cooperation may or may not be justified.

On the other hand, the “health” part of Global Health emphasizes that the field is essentially concerned with medical practice. Many of its institutions, relations and activities are derivatives of medical practice in that they purport to fund, coordinate or otherwise enable it. This is most obvious with organisations that have “health” inscribed into their core, such as the WHO, Gavi and The Global Fund. Yet even institutions that were not originally founded to operate in the Global Health business, such as the World Bank, assert that they work “to ensure all people receive quality, affordable care without suffering financial hardship”. Medical practice introduces a series of wholly different epistemic traditions into the Global Health mix. Here randomised controlled trials (RCTs), often carried out in clinical settings have been established as one of the dominant research methods. In RCTs, research subjects are randomly split into two groups, of which one receives an intervention that is being tested, while the other (known as the comparison or control group) receives an alternative treatment or a placebo. The two groups are then observed to see if there are significant differences in treatment response[iii]. These differences come to be seen as indicators of both treatment effectiveness and of the validity of the research method itself.[iv]

Some students, researchers and healthcare practitioners may be tempted to establish a hierarchy between the different research methods that coexist in Global Health. They may consider certain forms of sense-making to be “gold standards” that clearly trump others in that they produce actual knowledge, rather than mere opinions.[v] This approach is frequently based on an understanding of the scientific method as outlined by Karl Popper. Popper distinguished science from pseudo-science by arguing that the former consisted of theories that could stand up to tests[vi]. According to Popper, scientific theories are empirically refutable, while empirically irrefutable theories are by definition unscientific[vii]. As a result, scientific practice may be seen to consist in empirical testing that falsifies or refutes existing theories. This understanding of science puts RCTs capable of falsifying previously established research hypotheses at the top of our methodological toolkit. Quantitative, non-experimental research may be seen as a close second (its systematicity somewhat alleviates the fact that it is not replicable), while qualitative social research lies at the bottom of the pile.

I would like to challenge this hierarchical understanding of research methods. On a theoretical level, the work by Thomas Kuhn has long shown that scientific work is marked not just by one, but by a series of characteristics. Kuhn suggests that science has to be (1) accurate, in that it agrees with existing experiments and observations and (2) consistent (both internally and with existing acceptable theories). It also has to be (3) broad in scope, to extend beyond a handful of particular observations, (4) simple, in the sense that it orders phenomena under study, and (5) fruitful, in so far as it leads to new research findings[viii]. Taken together, these characteristics do not provide a scientist with hard and fast criteria of what to do or what to think. Instead, they are imprecise and at times they stand in conflict with one another, presenting scientists with several equally valid interpretive choices. As a result, theory choice and the interpretation of scientific facts includes subjective factors, such as scientists’ previous experiences, cultural norms, political prerogatives and various economic limitations.

On a practical level, the equal validity of experimental, quantitative and qualitative methods in the study of Global Health becomes even more obvious because each method comes with its own particular blind spots. Activities in controlled settings, such as in laboratories or clinics systematically entail a series of uncertainties, the most striking of which is whether or not results will remain consistent in non-controlled environments. Quantitative research remains besieged by questions of meaningfulness, i.e. doubts about how relevant the analysts’ categories actually are for the lives of the people under study. In contrast, qualitative studies tell us little about whether and how their detailed descriptions of the lives of just a few people apply to the lives of many others.

In light of these considerations, establishing general hierarchies of research methods seems ill-advised. Researchers should not evaluate their choice of method in the abstract, but do so with reference to the research questions they need to address. It is in focusing on the problem at hand that students and researchers of Global Health can most successfully determine the methods that might be most appropriate to solve it. Thus, the most interesting question to ask may not be which research method is the best in general but how our theoretical convictions and our recognition of the problems that we consider worthy of research relate to one another.

 

References

[i] Packard, Randall M. A History of Global Health : Interventions into the Lives of Other Peoples.  Baltimore: Johns Hopkins University Press, 2016, p.11.

[ii] Clinton, Chelsea, and Devi Sridhar. Governing Global Health: Who Runs the World and Why?  Oxford: Oxford University Press, 2017.

[iii] Kendall, J M. "Designing a Research Project: Randomised Controlled Trials and Their Principles." Emergency Medicine Journal 20, no. 2 (2003): 164-68.

[iv] Kaptchuk, Ted J. "The Double-Blind, Randomized, Placebo-Controlled Trial: Gold Standard or Golden Calf?". Journal of Clinical Epidemiology 54, no. 6 (6// 2001): 541-49.

[v] Sackett, David L, William M C Rosenberg, J A Muir Gray, R Brian Haynes, and W Scott Richardson. "Evidence Based Medicine: What It Is and What It Isn't." BMJ 312, no. 7023 (1996): 71-72.

[vi] Popper, Karl. The Logic of Scientific Discovery.  London and New York: Routledge, 2002 [1935], p.17.

[vii] Popper, Karl. "Science: Conjectures and Refutations." In Philosophy of Science: An Anthology, edited by Marc Lange. London and New York: Routledge, 2009.

[viii] Kuhn, Thomas S. "Objectivity, Value Judgment and Theory Choice." In Machette Lecture. Furman University, 1973.