Interest in the philosophy of measurement has seen a resurgence of interest over the last decade and this interest is marked by its attention to measurement practices (Tal 2013). While much of this interest has been within the physical sciences, e.g. (Chang 2004; Van Fraassen 2010; Tal 2011), increasingly philosophers within the social and medical sciences have begun to enter this discussion (Alexandrova 2012; Angner 2009; McClimans 2010). In this symposium we address two questions that animate the discussion of measurement across the sciences: 1) can current measures of quality of life or well being live up to the epistemic standards of measurement in the physical sciences? And 2) is there an account of measurement that can unify the sciences?
In our first paper, A Lay of the Land: Nomothetic and Idiographic Approaches to Quality of Life Measurement John Browne (Professor of Epidemiology and Public Health, University College Cork) foregrounds the two main conceptual approaches to quality of life measurement and discusses the methodological and practical difficulties they face. In our second paper, Epistemic and Ethical Problems with Nomothetic and Idiographic Quality of Life Measures Leah McClimans (Associate Professor of Philosophy, University of South Carolina) evaluates these two approaches to quality of life measurement and finds them wanting. She argues firstly that quality of life measures are not as different from measures in the physical sciences as some social scientists have suggested. Nonetheless both nomothetic and idiographic measures are epistemically (and ethically) unsound and likely to remain so. Finally Laura Cupples (graduate student, University of South Carolina) examines one possible unifying account of the epistemology of measurement: Eran Tal’s model-based account. She argues that this account does not extend unproblematically to cover nomothetic quality of life measures. Tal’s models epistemically support measures primarily through theory articulation, but, as McClimans has argued, quality of life measures lack a solid theoretical ground. Cupples suggests that this limitation means that there is no theory to articulate. Thus, if models epistemically support quality of life measures, it must be in some other fashion.
There are two main conceptual approaches to quality of life assessment: the nomothetic, where the individual’s perception of his or her quality of life is filtered through the lens of a standardised model of ‘the good life’; and the idiographic, where quality of life is constructed from individual evaluations of personally salient aspects of life. The dominant method is nomothetic. This is criticised for a number of reasons. First, many of the original quality of life tools such as the EuroQol or SF-36 were designed with little input from patients. Second, individual definitions of quality of life are posited to be highly heterogeneous and idiosyncratic, meaning that very few patients fit the ‘average’ definition. Third, the experience of completing and interpreting nomothetic measures has been described as artificial and lacking face validity by many patients and clinicians. Fourth, the most popular quality of life measures in current use were developed using classical test theory methods and can only be applied at the group level. The idiographic tradition in the social sciences assumes that for many important phenomena, including quality of life, individuals cannot be described using general rules because of the complexity of each life history. These methods were applied most intensively by psychologists in the 1960s working within the phenomenological tradition (e.g. George Kelly, Carl Rogers). The idiographic approach was adopted by quality of life researchers in the early 1990s and led to the development of a number of individualised measures such as the Schedule for the Evaluation of Individualised Quality of Life (SEIQoL). These measures allow each respondent to individually define the domains and weights to be assessed within the questionnaire. The track record of individualised measures will be reviewed in this presentation. The evidence to date suggests that although individualised measures have a strong surface appeal they are generally not useful within the confines of research paradigms that are nomothetic in nature (e.g. comparative effectiveness research). A more useful role of individualised measures lies within research contexts which are defined as idiographic at the outset, such as the formulation and monitoring of individualised care plans.
Quality of life measures within in the nomothetic tradition are popular with health policy makers in large part because of their ability to function as quantitative measuring instruments while also providing the patients’ point of view. From a development perspective this attraction requires that these measures are epistemically and ethically sound. This double burden has proven difficult to achieve and these instruments have received significant criticism, mostly from those who develop and work with them. For instance, in 1995 the Lancet ran an editorial cautioning the use of these measures as end points in clinical trials, in 1997 Sonia Hunt’s editorial in Quality of Life Research argued that they are misleading and probably unethical; more recently in 2007 Jeremy Hobart and colleagues argued in Lancet Neurology that almost all current measures are invalid. In my own work I have argued that they are invalid and difficult to interpret at least in part because they do not accurately represent the patients’ point of view.
In this paper I ask why nomothetic quality of life measures face these challenges. One explanation that researchers commonly invoke is that quality of life measures lack a ‘gold standard’ and are thus more difficult to measure than physical properties such as blood pressure. In what follows I examine and reject this explanation and offer a different one: the problems that quality of life measures encounter arise because quality of life lacks a theory that provides a representation of the measurement interaction, i.e. the relationship between the quality of life construct and its instruments. I further argue that the development of such a theory is in principle problematic given the idiosyncratic way that individuals find quality in their lives, particularly during times of significant change, e.g. an unexpected diagnosis, sudden loss of physical functioning. To the extent that nomothetic measures seek to quantify quality of life in these contexts they fail to be epistemically and ethically sound.
If nomothetic quality of life measures fail in this way, what of measures within the idiographic tradition? Individualized measures such as the SEIQoL are the epistemic equivalent to bioethics’ emphasis on individual autonomy. Both are problematic primarily because we can be wrong in our assessments of ourselves, i.e. we can be wrong about what we think we want and we can be wrong about the quality of our life. Epistemically and ethically sound measures cannot take individual assessments at face value. I thus conclude that neither the nomothetic nor the idiographic tradition supply us with quality of life measures that meet our demands
Eran Tal has developed a model-based account of the epistemology of measurement. While Tal’s work focuses on the measurement of time, he has suggested that this account might also apply to other measures as well, providing a unifying account of measurement across the physical and social sciences. I argue in this paper that his account does not extend unproblematically to measures in the social or medical sciences. As a case study, I examine nomothetic quality of life measures. Does the epistemic support models offer for these measures mirror that of Tal’s physical measures, or are models playing a different role in these measures?
According to Tal’s model-based account, “a necessary precondition for the possibility of measuring is the specification of an abstract and idealized model of the measurement process” (Tal 2012). That is, our claims about measure validity and our judgments about measurement accuracy only become meaningful in reference to some model of the measurement process under consideration. Similarly, we can only meaningfully compare measurement outcomes when we have a model to contextualize those outcomes (Tal 2012).
Tal explains that he takes models to be abstract representations of local phenomena that are constructed based on theoretical, statistical, and pragmatic assumptions about those phenomena. He argues that models can function as mediators between abstract theory and concrete phenomena. They can also serve as instruments that help predict and explain the behavior of target systems (Tal 2012). This account suggests that, for Tal, models provide epistemic support for measurement primarily through theory articulation, i.e. taking scientific theory in concert with statistical and pragmatic assumptions and applying that theory locally.
However, many philosophers as well as thoughtful researchers and clinicians have complained that quality of life measures lack solid theoretical grounding. There is no generally agreed upon account of what well-being entails or how quality of life varies with life circumstances or our adaptation to those circumstances. There is also little theoretical grounding for our assumptions about how respondents understand and interact with these survey instruments, i.e., how these measures tap into the phenomenon in question. Leah McClimans has argued that because respondents have varying conceptions of quality of life, they often interpret the questions posed by these survey instruments in unexpected and inconsistent ways (McClimans 2010).
It is clear that if models provide epistemic support for quality of life measures, it is not because they are mediating between abstract theory and concrete phenomena as Tal argues they do in physical measures. There is no well-developed or widely agreed theory of quality of life to serve as a target for mediation. Given this state of affairs, we should ask what it might mean to give a model-based account of quality of life measures and if it is still possible for models to provide epistemic support for claims about the validity, accuracy, and comparability of these measures.