Technik und Umwelt
Institut für Technikfolgenabschätzung und Systemanalyse (ITAS)
|
|
Forschungszentrum Karlsruhe Technik und Umwelt Institut für Technikfolgenabschätzung und Systemanalyse (ITAS) |
|
TA-DATENBANK-NACHRICHTEN |
In this article, a novel interdisciplinary approach to technology evaluation is presented in the field of health technology assessment. The case of pediatric cochlear implantation (CI) - a device for profoundly deaf people - has aroused much controversy in the Netherlands because CI were regarded as a serious drawback in the history of deaf emancipation and should therefore not be reimbursed by health insurance institutions. The article is based on a theoretical inquiry into the societal context of health technology assessment which led to the development of an interactive "participatory" evaluation procedure for pediatric cochlear implantation. By this procedure, criteria of success and acceptability of CI could be formulated and concrete policy recommendations were made which were widely endorsed and thus legitimised.
It seems as if the questions technology development enforces us to consider are becoming more and more complex and multifaceted. One has to take only a superficial glance at developments in the field of traffic, agriculture, health care, and environmental issues to be able to confirm that technology development is not only for the engineer, the physician, and others who develop and apply technology, but also for the politician, the ethicist, the sociologist, the citizen, and still others. In response, interdisciplinary approaches towards evaluation and managing technology have developed. As the problems of technology development have outgrown the highly specialized areas of separate disciplines, clearly, the time has come for such approaches. However, they come with their own dilemmas. This article adopts an evaluator's perspective in the field of health care, and seeks to identify some of the dilemmas evaluation researchers face when they explicitly place technology in its societal context. We draw from a theoretical inquiry into the societal context of health technology assessment and our experience with an interactive evaluation of pediatric cochlear implantation (CI).
A cochlear implant, or "bionic ear", is a device that can provide profoundly deaf people with a sound sensation. Basically, sound is received by a microphone, transduced into electric signals, optimized by a so-called "speech-processor", and then transmitted to the auditory nerve. The last step requires an electrode to be inserted in the cochlea, that is, it requires surgery. In the past, the technology, especially in children, has aroused controversy, as the technology is valued differently depending on whether deafness is viewed as a pathological disorder to be eradicated, or as a feature of a specific cultural minority. Those who adhere to the latter view, sometimes regard cochlear implants as a threat to the deaf community and a serious drawback in the history of deaf emancipation. In the Netherlands, however, as well as in most other countries, the "medical perspective" on deafness as something to be corrected has been dominant and has lain at the basis of many assessments. To summarize these assessments: cochlear implants do work fairly well in the sense that many recipients are able to use sound in a meaningful way. However, in 1996 the Dutch Minister of Health, confronted with protests that were based on the "cultural perspective", concluded that, despite the assessments, she could not decide on including the technology in the reimbursement package.
There are several important lessons we have learnt from this paradigmatic case.
Social constructivism vs. positivism
First, the cochlear implant case confirms what has been increasingly recognized since the 1980s, namely that health technology, like technology in general, is a "social construct" (Rip et al. 1995). The philosophy of social constructivism holds that a technology is not to be viewed as an artefact that can be defined apart from its societal context in terms of causal relations between a technology and its effects. Rather, technology should be regarded as the product of a society and its cultural heritage, reflecting the problem definitions, concerns, norms, and values that dominate within. For example, in-vitro-fertilisation is not simply a reproduction technique, but could emerge only in a society in which specific views prevail on the human body, the right to motherhood, and so forth. Likewise, a cochlear implant for a child is more than a hearing aid, as it incorporates specific views on deafness and the position of deaf children in society. Its success and acceptability depend on the way it is embedded in a broader context of rehabilitation, family advisory services, special education, and so forth. This has major consequences for the evaluation of health technology, or health technology assessment.
However, particularly in health technology assessment, the social constructivist philosophy has not yet been acknowledged (Reuzel and Van der Wilt 2000). Instead, a positivist theory still underlies health technology assessment, in which health technology is isolated from its context in order to test presumed causal inferences in a scientific experiment (Alderson 1998). The question is whether this is still adequate. Is it not true that the problems encountered in contemporary society are too large to be tackled within such a paradigm? For example, could one assess the value of IVF by counting the number of successful pregnancies? Or recommend that CI be included in the reimbursement package on the basis of the speech recognition skills of deaf children? No, that is to short-circuit matters too easily, if the question reads how to manage technology in its societal context. On the other hand, evaluating technology in its societal context requires that a broad range of aspects be considered. Most often, such endeavour is beyond the scope and competence of individual researchers. Interdisciplinary research is then warranted, but what kinds of expertise are needed? How to organize such interdisciplinary research?
Supporting decision-making vs. accommodating value pluralism
And then again, to combine expertise and organize interdisciplinary research is not automatically to accommodate value pluralism. The cochlear implant case is but one example that teaches us that technology is valued differently, depending on the perspectives persons involved hold on this technology. When, in an evaluation, this fact is neglected and the evaluation is based on one particular perspective only, its validity and usefulness may be impoverished, and a serious controversy may arise. This is rather poorly acknowledged in the field of health technology assessment. Which raises the question how health technology assessment relates to the aspirations of those who aim to provide decision-makers with comprehensive information on the value of a technology (Henshall et al. 1997). It appears that, currently, researchers in the field of health technology assessment are poorly equipped to accommodate value pluralism.
However, accommodating value pluralism seems to be at odds with vigorous decision-making. Underlying the previous paragraph is the assumption that the societal context of a technology, and particularly the life world of persons involved in the application of the technology, has primacy over the political realm of decision-makers and the professional realm of physicians, researchers, ethicists, and others conventionally involved in technology assessment. Therefore, an important aspect of valuing technology is to identify the value for whom it concerns directly. Consequently, evaluation and decision-making should be rooted in the experience of whom it concerns. This is not to say that stakeholder valuing is exclusive, exhaustive, or irrefutable, but that, in decision-making, stakeholder valuing should be acknowledged and considered as a primary source of legitimacy. This can be presented as an improvement of decision-making based on evaluation research. However, it is a problem, too. For obviously there is a strong tension between a genuine respect for different perspectives, or value pluralism, and the wish to make policy decisions on the basis of a widely endorsed agreement. Clearly, different perspectives on what is the most justified thing to do, will not allow for decisions readily being made. For example, departing from different perspectives on deafness and the value of CI, advocates and adversaries of CI hold different views on what constitutes appropriate care for deaf children. Which left the Minister of Health with an inability to determine the most justified course of action. How can this tension be eased? How can we make vigorous decisions when necessary and still maintain value pluralism?
The expert vs. the lay person
It is questionable whether assembling an interdisciplinary expert team can solve this problem, as it is difficult even for experts to acknowledge and represent the valuing of all those involved in the technology. Participatory and interactive approaches to evaluation research can be viewed as a response to this difficulty. Persons involved, including those called "lay persons", having expert knowledge of their own life world, are asked to forward whatever relevant knowledge and experience they have. Put somewhat simplified, it is said: let persons involved interact and deliberate until they agree on the conditions of a technology being acceptable and successful. But if things were only that simple! Is the strong emphasis on interaction between persons involved not making matters even more complex than they were already? Is it not invoking a lot of opportunism, emotions and irrationalities, and power play that are certain to destroy the last hope of decision-making being based on some kind of agreement at all? How to deal with the opinions of persons involved, without distancing oneself from their realm for reasons of irrationality and emotionality? Moreover, what is, in a stakeholder world featuring different perspectives and empowered voices, the role of evaluators, decision-makers, and ethicists by trade? Under what conditions are the results of interactive evaluation legitimate? When can these results be considered to be a fair achievement?
We have addressed the difficulties presented above in a project entitled "The societal context of health technology assessment", which has been completed with a thesis on "Health technology assessment and interactive evaluation: different perspectives" (Reuzel 2001). On the basis of our theoretical inquiries we have developed an interactive evaluation procedure, which has been employed in the case of pediatric cochlear implantation in the Netherlands. First, we decided that the evaluation of cochlear implantation should be "participatory", with persons involved participating, not just as research subjects, but as active agents who determine the evaluation's course and outcomes.
There are several reasons for performing participatory research:
Thus, it is acknowledged that a technology is not just an artefact or device, but is interwoven with its societal context.
These reasons for doing participatory research, however, might be qualified as strategic. It is a way of embracing a constructivist paradigm, but the problems attached to value pluralism and the problems of the expert vs. the lay person cannot be resolved by participation alone. Therefore, we decided to take a step further and start an interactive evaluation. Interactive evaluation distinguishes itself from participatory evaluation in that participants are not only consulted, but interact with each other. The interactive evaluation of cochlear implantation was based on active interaction between 51 stakeholders, including otorhinolaryngologists, parents of deaf children, decision-makers, teachers, pedagogues, representatives of deaf organisations, etc. In view of the controversy over cochlear implants, we have chosen to facilitate interaction between these participants through three cycles of individual interviews, in which participants discussed their claims, issues, and concerns with the evaluator, this evaluator acting as a messenger and moderator who transmitted the various arguments from one participant to the other. In order to prevent power-differences from disturbing the process, the participants remained anonymous to each other. The goal of the assessment was threefold: (1) to identify criteria of success and acceptability, (2) to achieve agreement about regulations to be implemented to meet these criteria, and (3) to formulate additional research questions in case of lacking information. The interactive design has several additional advantages:
These aspects of interactive research can be qualified not only as a strategic, but also as a moral improvement.
The bottom line is that the participants entered a debate, a process of "vicarious learning" (Guba and Lincoln 1989), or, as Jennings puts it, "a discourse of counsel and consensus" (Jennings 1993). As such, interactive evaluation does not only allude to social constructivism, but also provides an answer to the other two dilemma's. First, interactive evaluation is suitable for passing the razor's edge between accommodating value pluralism and decision-making. For it seeks to find agreement over policy recommendations, which, very importantly, does not require that the different perspectives are aligned to each other. That is, one may agree to policy recommendations with respect to a technology, without necessarily adopting the same perspective on this technology.
Second, the process of vicarious learning may provide an answer to the difficulties represented by stakeholder participation. In fact, this is a process of continuously scrutinising the arguments of the participants, which renders the process more rational in the sense that spurious and poorly evidenced arguments are corrected. Very importantly, this is not done by giving authoritative privileges to so-called "experts", but by stimulating the participants to learn and deepen their insight into the subject matter. (cf. Reuzel, forthcoming)
Where does it all lead to? The evaluation of pediatric cochlear implantation in the Netherlands has been successful and the results are encouraging. The case has been dealt with fairly comprehensively. Within one year it has been possible to identify the criteria of success and acceptability of pediatric cochlear implantation. Moreover, it has been possible to formulate eighteen concrete policy recommendations on family advisory services, the distribution of implant centres, information and counselling, protocols, and further research. Very importantly, they are widely endorsed, and therefore legitimized.
We conclude that participants have successfully overcome - not eradicated! - their different valuing. Moreover, we believe that the process of vicarious learning, which is the core feature of interactive evaluation, is by large responsible for the success. Indeed, analysis of the interviews shows that mutual understanding has been improved, and that some of the policy recommendations are syntheses of different perspectives. This leaves us fairly optimistic about the potential of interactive evaluation to ameliorate the dilemmas of evaluation in its societal context.
Alderson, P., 1998:
The importance of theories in health care. BMJ, 317: 1007-1010
Henshall, C.H.; Oortwijn, W.J.; Stevens, A.; Granados, A.; Banta, H.D. (eds.), 1997:
Priority setting for health technology assessment: theoretical considerations and practical approaches (a paper produced by the Priority Setting Sub-group of the Eur-assess Project). Int. J. Technol. Assess. Health Care, 13(2): 144-185
Jennings, B., 1993:
Norms of argument in health policy. In: Fischer, F.; Forester, J. (eds.): The argumentative turn in policy analysis and planning. Durham: Duke University Press
Reuzel, R.P.B., 2001:
Health technology assessment and interactive evaluation: different perspectives. Thesis: University of Nijmegen, Nijmegen
Reuzel, R.P.B., forthcoming:
To assess rationality before anything else: a remark on the legitimacy of rational technology assessment. In: Decker, M. (ed.): Interdisciplinarity in technology assessment: implementations and their chances and limits. Wissenschaftsethik und Technikfolgenbeurteilung, Band 11, Berlin: Springer Verlag
Reuzel, R.P.B.; Van der Wilt, G.J., 2000:
Health technology assessment and evaluation: back to basics? Evaluation, 6(4): 383-398.
Rip, A.; Misa, Th.J.; Schot, J., 1995:
Managing technology in society: the approach of constructive technology assessment. London: Pinter.
Rob P.B. Reuzel
University Medical Centre Nijmegen
Dept. of Medical Technology Assessment (253)
P.O. Box 91 01
6500 HB Nijmegen, The Netherlands
Tel.: +31 / 24 / 361 66 59
E-Mail:
rreuzel@baserv.uci.kun.nl
Redaktion der TA-Datenbank-Nachrichten