Introduction
Environmental and health risk communication researchers and practitioners are apt to benefit from greater cross-fertilization between fields. This paper demonstrates how research on risk perceptions, decision making and environmental risk communication can contribute to vaccine risk and safety communication. Breadth of the review is preserved at the cost of some depth -- to convey the rich variety of empirical findings available to guide risk communication.
Uncertainty about vaccine risks and benefits spurs the need for vaccine risk communication.[1] This uncertainty stems from several sources, including lack of data, disagreement about its interpretation and lack of biologically plausible theories to explain effects some attribute to vaccination. Of greatest concern are rare adverse events, including high-pitched, inconsolable screaming and death. As with many technological and environmental risks, highly improbable and sometimes poorly understood adverse events are at the heart of an adversarial situation where existing communications appear inadequate.
Part of this inadequacy may be due to contention about what communications should convey. Those facing risk seem likely to want information to help establish causality and responsibility. Ultimately, this information can also contribute to development of safer vaccines. Health experts likely want information corroborating the improbability of adverse effects and the magnitude of vaccination benefits. Such information seems essential to achieve high immunization rates. While the benefits of vaccination are demonstrable, the near-eradication of some diseases means that parents faced with vaccination decisions may be unfamiliar with them,[2] and have little basis for evaluating benefits.
Citizens have the right to be informed and participate in decisions affecting their quality of life and health. Such decisions are more difficult than those involving known risks. Development of theory-based and empirically researched risk communication to support decisions is a vital component of public health risk management.
Faced with the need to communicate about vaccine risks, government agencies and private organizations encounter a host of problems in succinctly addressing public concerns while conveying meaningful technical information. In addition, the public health objective of increasing immunization rates may seem difficult to reconcile with increased efforts to communicate vaccine risks. Insights from research on vaccine risk perceptions, decision making, and environmental risk communication can help.
After a brief description of what is known about such matters, the paper concludes with an illustrative research agenda for vaccine risk communication.
Evidence for misconceptions along these lines can be found in recent surveys of physicians, parents and adult patients. For example, presented with a set of vaccination scenarios, physicians chose not to immunize in over two-thirds of cases for which the American Academy of Pediatrics or the Advisory Committee on Immunization Practices recommends it, because the physicians judged as contraindications conditions not so recognized by those bodies, or they judged the risks of vaccines as outweighing the benefits.[4] Another recent study of immunization[5] found that nearly one third of parents had concerns about a single injection, while more physicians than parents had concerns about giving very young children three injections. Consistent with these findings, a study by Askew et al. found that 30% of private physicians interviewed agreed with the statement that "giv ing more than one injection at a time increases the likelihood of side effects (even more so than the combined risk of side effects if the injections were given separately)."[6] In another study, the most common (34.3%) reason for elderly patients' previously failing to received an influenza vaccination was fear of side effects and shots.[7]
Two small studies have explored more generally how parents think about vaccines. One ongoing study of the mental models of African American parents in Pittsburgh has found some suggestive differences between the mental models of parents whose children receive DTP3 (diphtheria, tetanus, pertussis) immunization in a more timely fashion and those whose children are immunized later.[8] Those whose children are more often immunized on time appear more likely to mention direct exposure to the disease and a weak immune system as risk factors; more likely to say that shots strengthen the immune system, and that a series of shots is needed to get full protection from pertussis; and less likely to say that poor health is a risk factor or that shots act as medicine. In a much earlier, similarly open-ended study of 34 mothers carried out in England, knowledge of the effects of whooping cough was very limited, including not knowing that it could be fatal.[9] Some appeared unaware of vaccine risks; most were unfamiliar with contraindications for vaccination and with the concept of herd immunity. This study also looked at sources of information and whether mothers wanted more information before making decisions. A majority did, but without much consensus regarding that information; risk statistics, after-effects of vaccination, and effects of the disease were all mentioned.
Studies have revealed other influences on vaccination decisions. For example, an "omission" bias has been identified in hypothetical vaccination decisions,[10] as well as in other decision contexts. This bias is essentially a perception that actions are riskier than inactions (omissions of actions), and that vaccination, because it involves taking an action, is riskier than disease, even if the expected mortality and morbidity rates are lower with the vaccine. These results are consistent with the general observation made by some groups that vaccination of a child is a "physical intrusion into a healthy body,"[11] and with physicians' preferences to "do no harm." Free-loading (relying on herd immunity and choosing not to vaccinate) and altruism(vaccinating to protect others) may play some role in vaccine decisions, but Meszaros et al. have found that "bandwagoning" (doing what everybody else seems to do) appears to be a much stronger influence than either.[12]
In other countries, common circumstances seem to have led to increases in anti-vaccine movements and disease outbreaks from falling immunization rates. These include (1) salient cases of neurological vaccination reactions, (2) outspoken and articulate advocates of non-vaccination, and (3) apparent decreases in advocacy of vaccines by medical practitioners concerned about vaccine safety and efficacy.[13] Parents are likely to follow physician recommendations,[14] or other trusted sources, highlighting the critical role of providers in determining immunization rates.
If these findings are reliable, a sizable, perhaps increasing, minority of patients and providers either hold what experts might call misperceptions, or they differ radically from experts in their decision-making processes and beliefs about immunization.
Perceptions of risks depend fundamentally on circumstances: Who is at risk, what the options for risk management are, who can and does control the risk, the nature of the hazardous process and how these are portrayed. Judgments of risk acceptability[18] depend on the probability of an adverse outcome and its magnitude or severity, but not these alone. For example, people tend to avoid ambiguity if they can; a risk judged more certain (e.g., the risk from a known disease) may be viewed as less risky than an equivalent or smaller risk that is perceived as more ambiguous (e.g., the risk from a new, unfamiliar vaccine).[19] While people care what the numbers are (and quantification may well serve to sharpen rather than resolve conflict),[20] they also care how risks are managed and whether they have a say in the risk management process.[21] Many argue that risk is socially or culturally constructed.[22]
Perceptions are also shaped by cognitive short-cuts that people take when they are processing information. These short-cuts, called heuristics, can lead to predictable biases, as in the case of the "omission" bias described above. So, for example, people's subjective estimates of risks are often compressed, overestimating rare causes of death and underestimating common ones.[23] This may in part be due to the use of "availability," using how easily something comes to mind as an indicator of prevalence.[24] Very unlikely but catastrophic events are memorable and may easily come to mind.[25] Another relevant heuristic is representativeness, in which people judge probabilities according to similarity of (sometimes irrelevant) circumstances, ignoring prior probabilities[26] -- e.g., basing judgment of risk on the similarity between two children, one of whom experienced a serious adverse event after vaccination, could result in a highly subjective estimate of whether the other would experience a similar event, neglecting to take into account that such events occur very seldom. Here representativeness, or forgetting to take base rates into account, could potentially lead to serious misestimates of the likelihood of an adverse event. This heuristic may be related to people's preference for stories over statistics: The more specific the story, the easier to envision, and the more likely it seems. In fact, the more details there are to match, the less likely a given case will match another. For example, it is more likely that a child will have a fever than it is that a child will both have a fever and an allergy to gelatin used in a vaccine.
The National Research Council (NRC)[27] identified three kinds of goals for risk communications: advocacy, education and decision-making partnership. An advocacy goal would be to enforce or encourage a behavior or belief. It can be argued that in so doing one is attempting to persuade the public to follow expert advice. In education, the goal is to inform. This subsumes a category of education that could be called decision support in which the goal is to give the public enough information to enable them to make decisions according to their own values. The third kind of goal is to establish or foster a decision-making partnership. This requires that the public be involved actively in risk management and decision-making, including structuring the problem and selecting management options. The first two have been more commonly adopted than the third.
Advocacy. Often, risk communi cation seeks to encourage people to change behavior, for example, to vaccinate one's children; explicit advocacy goals can be found in most public health messages. However, when there is conflict, advocacy can be perceived as persuasion not necessarily in the interests of risk communication recipients. Arguably, and perhaps for good reason, to many health practitioners, "risk communication" about vaccines means persuading the public that the risk is vanishingly small and should be ignored. Sheila Jasanoff has suggested that: "risk communication is often a code [word] for brainwashing by experts or industry."[28]
Public education. One type of goal is simply to inform people about the risk. Unless such goals are further refined, the kind of information communicated can range from technical to arcane and may not help the recipient. For example, simply knowing that AIDS comes from a virus will not necessarily help one protect oneself
A more specific goal is to enable people to make informed risk decisions. The kind of information that one disseminates in this context should be geared toward risk control decisions. Information of the sort provided in "6 common misconceptions"[29] can support decision-making about vaccination if communication appropriately targets the audience and is construed as trustworthy and relevant.[30] People are likely to face two kinds of decisions about most risks: Those about their own and their family's exposures, where they have considerable individual control, and those in democratic government, where they have limited individual control, but can contribute to the debate. This is clearly true for vaccination decisions; people decide whether they or their children should be vaccinated or whether to become active in organizations such as Dissatisfied Parents Together (DPT) with a public policy agenda.
Medical and participatory decision making. Participation in decision making is the third kind of goal identified by the NRC. While public input and participation may be a goal of health agencies, their activities and efforts don't always correspond.[31] Recent research has highlighted the importance of trust and credibility in risk communication.[32] Government agencies and officials may be less trusted than many other sources, including media.[33] However, surveys show that people tend to trust physicians more than other sources of expert advice.[34] Nevertheless, participation by the public in medical decision-making is likely to continue to increase. Where patients are required to give informed consent,[35] or decide for themselves how to use over-the-counter treatments,[36] participation is inevitable. When used responsibly, public participation can be a mechanism for establishing the kind of partnerships critical to success of many risk management problem-solving endeavors.
At the 1995 American Association for the Advancement of Science meeting, Stephen Pauker of the New England Medical Center and Tufts University offered a continuum of models for medical decision making from the doctor's perspective. These ranged from MDMG "Me doctor Me God" (the classic model in which doctors make all decisions unilaterally) and its successor informed consent to various forms of increasingly shared decision making (including the use of in-depth videos about the nature and outcomes of specific medical interventions), the use of decision analysis (in which doctors actively elicit the patient's values and support multi-attribute decision making), to the extreme in which doctors provide only guidelines and the doctor plays a small, if any, role. These models exhibit the range of goals of risk communication from least to most participative. One could argue that public policy goals in a democracy should be more rather than less participative, recognizing that participative decision-making about immunization policy, for example, does not mean that stakeholders will necessarily agree on how best to manage vaccine risks.
Ethical issues. Health risk communicators cannot ignore the policy and ethical implications of messages. While exposure to vaccine risks may be avoided by choosing not to vaccinate, this entails increased risk to individuals and the community. Messages regarding mandated vaccination may be perceived differently from those about voluntary vaccination; infant vaccination of may be perceived differently from vaccination of older children or adults.[37]
Several ethical principles have been identified as important in decision-making research on environmental and medical risks, including the omission bias and the "do-no-harm" bias.[38] The last causes reluctance to harm some to help others, even wh en the harm is less than that from not acting. Such attempts to be fair, even to unidentifiable groups, can result in unjustifiable judgments.[39]
As mentioned, altruism, free riding and bandwagoning all had a role in vaccination decisions in one study, but bandwagoning appeared stronger than altruism or free riding.[40] While it might seem reasonable to appeal to altruism in vaccine communications (to not infect others or to eradicate the disease), another recent study also indicates that such appeals may be ineffective. Of a sample of 55, one person cited community protection as a reason for preferring OPV over injected polio vaccine (IPV), but none cited this as the only reason.[41]
In the context of immunization, it has been argued that Hardin's notion of the tragedy of the commons[42] is the appropriate lens for discerning the quandaries faced: Not to immunize may be optimal for an individual if there is herd immunity, but this will lead to aggregate failure of herd immunity. As indicated, some evidence suggests that the "do-no-harm" bias may lead physicians to recommend vaccine deferral or omission for individual patients if their risk is salient.
Informed consent is perhaps the most widely used paradigm to address individual liberties in medical decision-making. Some have argued that informed consent can be achieved with value-of-information and decision analyses.[43] Others argue that a decision-theoretic approach is inappropriate.[44] In this debate, knowledge and expertise are key, but a focus on some kinds of expertise (e.g., health risk assessment) may lead to an impoverished context in which other expertise and experience are ignored in defining "informed."
Health risk communication designers have an ethical obligation to account for available resources and constraints on their potential audiences. For example, if local policies are inconsistent with national health goals, they may hinder citizens' effective decision-making.[45] If parents cannot find an accessible and affordable way to vaccinate their children, they may not, regardless of what communications about vaccines have reached them.
Research on outbreaks attributable to the failure of immunization programs in other countries or in sub-populations opposed to immunization for philosophical or religious reasons[47] illustrates that several audience segmentation strategies may be appropriate for vaccine risk communication. First, any one of several stages in health decision processes may characterize a particular audience.[48] Second, there are multiple reasons for non-vaccination decisions.[49] Third, as described above, in addition to parents and adult patients, some health care providers may be concerned about effectiveness or vaccination risks. Decisions providers and policy makers face, e.g., how to frame the decision for a patient or parent, differ in many regards from those faced by parents. Communications appropriate for some subset of parents may not effectively address concerns of others -- or of providers. Fourth, patterns of community health risk communications may influence decisions, e.g., when the media amplifies some kinds of information and dampens or suppresses others,[50] or when there is a tendency for media to link responsible agents and adverse effects, rather than discuss possible risk reduction solutions.[51] Finally, the most common approach to audience segmentation is demographic;[52] such an approach may enable communicators to target those with a common language and culture.
Determinants of behavior. Several theories of health-protective behavior may be used to predict the effects of risk communication, including the health belief model,[55] the theory of reasoned action,[56] protection motivation theory,[57] social learning theory[58] and subjective expected utility theory.[59] One comprehensive model, the extended parallel process, builds on these and holds that people are unlikely to undertake a risk control measure unless they feel that they can effectively control the risk (i.e., response- and self-efficacy) and it is personally relevant and serious.[60]
Both contextual and individual attributes determine specific health behaviors, such as vaccination. Vaccination rates may vary by socio-economic status or race/ethnicity.[61] As illustrated above, it is also likely that most parents follow the guidance provided by their pediatrician. If so, either the parent's decision agrees with the physician's advice (e.g, the benefits of vaccination outweigh the risks), or the parent has decided to follow the pediatrician's advice, without going through an independent decision process regarding any specific vaccination. When the parent perceives a risk or potential risk from a vaccine and makes an independent decision, a risk behavior or decision model may be applicable. These are also precisely those cases where the parent is likely to be most motivated and engaged in the issues surrounding vaccination, and where the parent may be seeking further information about vaccines.
While there is little overlap between the extensive literature on threat and fear appeals and the literature on risk perception and communication,[62] there are obvious overlaps in their implications for health risk communications. Psychometric research shows that perceptions of risk acceptability are related to judgments of how dreadful, catastrophic, and unfamiliar the risk is;[63] these dimensions appear related to threat and fear. Controllability of and exposure to the risk are also judgments that are predictive of risk acceptability. Together, these bear a family resemblance to the notions of response efficacy, which is defined as how effective responses to the risk will be, and self-efficacy, which has to do with judgments of one's own ability to control the risk, and may well be related to locus of control.
In sum, risk communication recipients need appropriate mental models (i.e., a basic understanding of the underlying hazardous process, including exposure, effects, and mitigation processes), the belief that the risk is a serious threat to them, as well as the resources to control the risk and the belief that they can act effectively to reduce the risk, in order for them to consider acting.
A four-step approach to risk communication based on people's mental models of risk processes is outlined in Bostrom, Fischhoff and Morgan.[65] This mental models approach provides a way of discovering what people know and using it to develop risk communication, as described in the following.
Mental models interview. Members of the intended audience are interviewed to assess their mental models of the hazardous process. The first part of the interview is completely non-directive, in that it does not use preconceived response scales, and so lets the respondent structure the response. The interview opens with: "Tell me what you know about (e.g., immunization) and any risks it poses." The interview becomes progressively more structured, but is still open-ended. Prompts follow for exposure processes, effects processes, risk assessment and management, risk comparisons, and personal risk. Effects processes include the nature of effects and uncertainty about effects. Risk assessment and management includes the sources of the respondents information about the risk, as well as all aspects of testing and reducing risk. Risk comparisons might be requested. The directive portion of the interview has varied in the studies done to date. Photograph sorting tasks, definitions of key terms, and hypothetical decision-making and prediction tasks have been used.[66]
Expert decision models. Results of interviews are coded into an expert decision model. The expert decision model for several studies to date has been the representation of an influence diagram, is a directed network showing the probabilistic dependencies between events in a process. For more details, see the summarized influence diagram in the paper by Morgan et al.,[67] or the more extended diagram in Bostrom et al.[68] The role of experts is discussed below.
Diagnostic knowledge test. A test was developed based on the results of the interviews and information from the "basic" level of the expert influence diagram, which is hierarchical. The interview process is so time and resource consuming that it would otherwise be exceedingly difficult to sample a large group of respondents. The test results are analyzed to provide profiles of what sets of beliefs people have and how specifically they think about the risk, as well as what people on average do or don't know about the risk.
Designing risk communication. As discussed, prior knowledge and cognitive attributes, (e.g., heuristics, attentional and memory limitations) affect how new information is processed. Risk communication design can be divided into two tasks: content design, and formatting. For a risk communication to be effective, both of these should take into account how people process information. The content of a risk communication should, as stated above, address recipients' mental models, and include the basic facts about how to identify the risk, and about exposure, effects and mitigation processes. The format should highlight and summarize key information, and provide information in a usable format. That is, the format should be compatible with the structures of the decisions people face. Because risk control decisions are made in several stages, communications should be targeted carefully so that the recipient finds the information appropriate for the decision stage he or she faces. Risk communication design is discussed more extensively by Atman et al.[69]
Many other frameworks have been used to find out what people know in order to incorporate it into message design,[70] under the general rubric of formative research or audience-centered approaches. Several elements distinguish a mental models approach from others, including the formalization of an expert decision model and the formal analysis of the open-ended interviews.
Although obvious, it should be emphasized that expertise in immunology, for example, does not confer the expert with expertise in risk communication, even that regarding immunization.
As discussed, prior knowledge and cognitive limitations, (e.g., heuristics, memory) affect how new information is processed. There is evidence that even simple numerical estimates -- of population, for example -- rely on both domain-specific knowledge and heuristics such as availability.[78] The metrics and mapping framework proposed by Brown and Siegler[79] provides a model for how domain-specific knowledge, such as that elicited in mental models research, and heuristics, such as those investigated by Baron and others,[80] relate to the perception and communication of numerical risk estimates. This framework rests on three tenets: (1) There is no one "real-world estimation process." Rather, people use both heuristic strategies and knowledge-based strategies to generate quantitative estimates. (2) These strategies may be used alone or in combination. (3) When multiple strategies are used, results derived from competing processes are weighed according to their predictive value. In addition, this research has led to a decomposition of estimation tasks into independent metric and mapping components. The metric component focuses on the statistical properties of the quantitative dimension of interest; the mapping component is concerned with how items are ordered along this dimension. Both are relevant to risk perceptions, given that mapping is the basis for comparative risk assessment within a domain (e.g., which diseases are riskier than others), and metrics the basis for overall risk assessment and comparison to risks in other domains (e.g., how likely is someone who has had chickenpox to get shingles later in life). Numerical estimates of risks are based on specific circumstances (e.g., a reported death rate for the U.S. population in a given year). In choosing which numbers to present, risk communication designers should consider which numbers are likely to be relevant to -- and perceived as relevant by -- the communication recipients.
Most risk communications include some statement of probability based on some exposure or dose. In this context, it can be tempting to use a verbal probability alone to simplify the presentation for the reader. However, the communicator should beware. Studies show that the interpretation of verbal probabilities depends on the context.[81] "Likely" in "likely to get AIDS" is unlikely to be interpreted the same as "likely" in "likely to catch a cold." Specific information about or frequencies of exposure may be understood differently depending on whether exposure estimates are represented cumulatively or for single exposures. There is evidence that people do not cumulate estimates of single exposures at a high enough rate.[82] However, larger numbers tend to have larger effects, all else held constant. Thus, cu mulative estimates of risk are likely to lead to higher perceived risk than estimates of single shot risks.[83]
As mentioned above, numbers may well serve to sharpen rather than resolve conflict.[84] When verbal quantifiers are used instead of numbers, communication recipients may perceive that the communicator is avoiding being specific, or intentionally obscuring the facts ("weasel-wording" or "government double-talk"). Withholding information has been a documented source of contention and conflict for many environmental risks.[85]
Many risk communications include comparisons. For example, in the "6 Common Misconceptions" brochure the antigens in vaccines are compared to antigens that result from the bacteria introduced by food into a body.[86] In making comparisons, the communicator should bear in mind that risk is multidimensional. Which dimensions are being compared? A simple comparison of probabilities may imply a comparison on other dimensions, such as voluntariness, in which case the reader may find the comparison unacceptable or uninformative. Studies of risk comparisons to date have found that even the best experts' predictions of responses to risk comparisons may be wrong.[87] In the few studies that have been done of graphical risk comparisons, it has been shown, for example, that risk ladders communicate differences more effectively that pie charts,[88] and that choice of a range for a risk ladder can be critical, because the location of the risk on the ladder (high or low) has an effect independent of the information conveyed about the absolute and relative size of the risk.[89]
Given the state of the science of risk assessment, it is important to convey the uncertainties that exist in estimates of risk, although this is difficult to do. There are different kinds and sources of uncertainty, such as a lack of scientific agreement on interpretation of data, or a lack of data.[90] There are also different techniques for conveying uncertainty, including graphs, diagrams, verbal presentations, and the presentation of alternative numerical estimates. Representing the uncertainty in information can lead to changes in readers' perceptions and decision making, including (1) focusing on the upper bound of the risk rather than the midpoint;[91] (2) de-emphasizing the uncertain information and focusing on other, potentially less relevant information when making decisions[92] and (3) possibly attributing the uncertainty to incompetence on the part of the source of the information.[93]
Good risk communication necessarily simplifies and summarizes, but this must be done cautiously. Risk communication researchers have suggested including where to go for more information, or including glossaries and appendices with more information.[94]
A basic distinction can be made between message-based evaluation methods, such as content or structural analysis of text, or audience-based methods that allow analysis of the risk communication recipients' reactions.[96] Surveys are perhaps the most common audience-based method, followed by focus groups. Less common techniques include think-aloud protocols,[97] in which a person is asked to say out loud everything she or he thinks while reading a risk communication, or making a risk-related decision. Audience-based empirical evaluation can contribute to process goals as well.
Measures appropriate for outcomes such as reduced risk may be entirely inadequate for assessing whether a risk communication has corrected errors or filled gaps in a communication recipient's mental model of the hazardous process, but may be helpful in assessing how much decision support the communication has provided. Reduced risk can be evaluated using self-reports of changed behaviors, observed behavioral changes, or changed outcomes (e.g., changes in vaccination rates or the incidence of adverse events, as assessed via surveys of parents). While self-reports are easiest to collec t, they can be misleading and very inaccurate. Extensive research on how well people remember frequencies of various behaviors has produced guidelines for improving accuracy, but has also shown how erroneous self-reports can be.[98] In light of this, the relatively high costs of observing behaviors or measuring changed outcomes look less objectionable. However, for some risks simple assessment of changed behaviors may be inappropriate due to variations in the relative costs and benefits of those behaviors for individuals in differing circumstances.
Changes in mental models can be evaluated using mental models interviews,[99] or, more efficiently, using questionnaires based on mental models interviews.[100] Only those misconceptions and misplaced foci captured in the design phase are likely to be represented in subsequent research results. Experts and non-experts tend to conceptualize processes quite differently.[101] But lay mental models tend to share common traits due to common cultural and physical experiences.[102] For this reason, small mental models studies with members of the target audience are likely to provide a reasonable empirical basis for questionnaire design. A potential obstacle is that any study of mental models entails the use of an expert standard -- that is, a characterization of the risk that represents the consensus of experts about the relevant risky processes and potential risk control interventions. It follows that expert models formulated from several perspectives may be appropriate, and should be considered carefully.
Improved decision making can be measured a number of ways, including using metrics for reduced risk or improved mental models. Hypothetical decisions based on scenarios may also be useful in assessing whether a risk communication supports and improves individual decision making (e.g., choosing between various schedules of IPV and OPV). Decision making processes on a larger scale can be assessed to some extent using metrics for process goals. Metrics related to process goals include measures of attitudes and values -- usually on Likert-type scales,[103] credibility and trust,[104] and rates of public participation in policy decisions. Many attributes of risk communication processes are likely, however, to be missed by such measures. Political and cultural factors can be assessed,[105] but are unlikely to be changed by a single risk communication, or even a comprehensive risk communication program. Public participation in policy decisions may not be measured adequately by participation rates, which cannot reflect the degree of responsiveness and mutual respect engendered by specific risk communications. When values are not shared, negotiation and conflict resolution become critical components of the risk communication process, and should be evaluated as well.
To summarize, in determining how to evaluate a risk communication or risk communication program, the evaluator must first determine or define risk communication objectives, then select a mix of evaluation metrics and methods. Defining the objectives in terms of relevant metrics accomplishes the first half of this selection process. However, multiple selection criteria for evaluation metrics and methods should be considered. Cost-effectiveness of an evaluation method should be weighed against other criteria, such as timeliness, and process effects. Evaluation should take into consideration that the effectiveness of risk communication may vary depending not only on the characteristics of the risk the communication addresses and how that risk is addressed, but also on such factors as the context in which the communication occurs, the size of the target audience(s), the diversity of stakeholders, and the degree to which stakeholders share values vis-á-vis risk-related behaviors and outcomes.
Risk communication evaluation is likely to be most informative and useful when integrated into a comprehensive risk management strategy. Two approaches that explicitly incorporate communication and evaluation into the risk management process are the mental models approach,[106] and the behavioral intervention planning guide.[107] Even when resources are limited, risk communications should be evaluated empirically; experience and research show that common sense and expert opinion cannot yet adequately replace empirical evaluation.[108]
Step one: Audience targeting and segmentation. Audiences are often targeted by language groups or ethnicity but may also be targeted by other individual or contextual attributes. To whom should vaccine risk communications be addressed -- segments of the public, or providers? Many studies show that significant percentages of providers evidence misconceptions or disagreement with immunization experts, e,g, regarding appropriate vaccination contraindications. Because they sample an audience probably self-selected for concern about vaccine risks, Asch et al.[109] may have set an upper bound on percentages of the public that hold some common beliefs about and attitudes toward vaccines that may lead to non-vaccination decisions.
Step two: Qualitative risk. Very little is known of common beliefs about immunization and vaccines, including correct beliefs, knowledge gaps, misconceptions and disagreements with experts. Some evidence suggests widespread knowledge gaps about the effects of diseases and vaccines. Combing the literature/media is one way of investigating what beliefs are held, but this may miss segments of the population that are not well represented in the media. Research on expert decision models of the underlying hazardous processes is also lacking.
Step three: Quantitative risk. How should vaccine risks be quantified in risk messages? Research illustrates that the specific context and format of numerical estimates can influence perceptions, dramatically. Current vaccine communications provide odds ratios of various kinds, e.g., a 1994 Vaccine Information Statement (VIS) for measles mumps and rubella vaccines states that there is a 5-15% risk of a rash in the first or second weeks after the first dose, or verbal probability statements (in the same VIS the risk of seizure is described as "rare"). CDC's publication on misconceptions[110] and a recent issue of Mothering,[111] both provide comparative tables. Research is needed on how these are perceived and on the overall effects of various representations.
Step four: Communicating uncertainty. As illustrated, current vaccine risk communications convey uncertainty both explicitly, by use of ranges, and implicitly, by use of verbal probability statements. The effects of these on beliefs, attitudes and decision-making should be evaluated. Further research is needed on how to convey uncertainty appropriately, e.g., how to convey the magniture of uncertainty implied by the description of risk as "rare."
Step five: Increasing information input and exchange. As Chess et al.[112] discovered in the state level, government agencies with laudable goals may actually spend most of their time responding to inquiries, rather than initiating dialogue. Ways to increase and evaluate public input into and dialogue about vaccine safety research and vaccine risk management deserve more attention. Cursory evaluation of phone calls to the CDC regarding vaccine safety showed most (62%) to be from providers.[113] Further evaluation of such information could help agencies improve outreach. How to incorporate stakeholder input, evaluation and revision into an effective vaccine risk communication strategy deserves further consideration.
Step six: Incorporating evaluation. No expert can accurately predict effects of risk messages. Empirical evaluation should be included as an integral part of any risk communication effort. Ongoing research on the Vaccine Information Pamphlets and their successors, the Vaccine Information Statements, are likely to augment considerably what is known about vaccine risk perception and communication. Integrating evaluation of vaccine risk communication efforts into evaluations of vaccine risk management efforts is warranted.
** Dr. Bostrom is Asst. Professor, School of Public Policy, Georgia Institute of Technology. She holds a B.A. (English) from the University of Washington, M.B.A. from Western Washington University and Ph.D. (Public Policy Analysis) from Carnegie Mellon University. Email: ann.bostrom@pubpolicy.gatech.edu.
[1] Yet, Dr. Robert Chen of the Centers for Disease Control (CDC) argues that benefits of vaccines are not uncertain, but rather the need for vaccine risk communication stems from: successful reduction in incidence of vaccine preventable diseases by immunizations, making vaccines risks relatively more prominent, well documented benefits being "invisible" to the average parent; and uncertainty about risks of rare serious effects of immunization. (personal communication).
[2] Ricardo Alvarez, Daniel Sacolick & Maicie Wong, A Descriptive Study of Parents' Knowledge and Attitudes Regarding Immunization at the Sunset Park Family Health Center in Brooklyn (1995) (unpublished, Brooklyn Coll. Med.).
[3] CDC, National Immunization Program, 6 Common Misconceptions about Vaccines: And How to Respond to Them (1995).
[4] James R. Campbell et al., Intent to Immunize Among Pediatric and Family Medicine Residents, 148 Arch. Pediatr. Adolesc. Med. 926 (1994) -- erratum 149, at 60. See also Peter R Loewenson et al., Physician Attitudes and Practices Regarding Universal Infant Vaccination Against Hepatitis B Infection in Minnesota: Implications for Public Health Policy, 13 Pediatr. Infect. Dis. J. 373(1994) and P. J. Salsberry, J. T. Nickel & R. Mitch, Missed Opportunities to Immunize Preschoolers, 8 Appl. Nursing Research 56 (1995).
[5] Kathleen A. Woodin et al., Physician and Parent Opinions. Are Children Becoming Pincushions from Immunizations? 149 Arch. Pediatr. Adolesc. Med. 845 (1995).
[6] George L Askew et al., Beliefs and Practices Regarding Childhood Vaccination Among Urban Pediatric Providers in New Jersey, 96 Pediatrics 889 (1995).
[7] R. Ganguly & T. B. Webster, Influenza Vaccination in the Elderly, 5 J Investig Allerg Clin Immunol 73 (1995); Nicholas H Fiebach & Catherine M Viscoli, Patient Acceptance of Influenza Vaccination, 91 Am. J. Med. 393 (1991).
[8] Jeanette M Trauth et al., A Mental Models Approach to Parental Decision Making Regarding Childhood Immunizations, Poster, National Immunization Conference, Washington DC, April 1996.
[9] C. M. Harding & K. J. Bolden, Whooping Cough Vaccination: A Worrying Decision for Parents, 227 The Practitioner 283 (1983).
[10] David Asch et al., Omission Bias and Pertussis Vaccine, 14 Med. Decision Making 118 (1994); Jacqueline R. Meszaros et al., Cognitive Processes and the Decisions of Some Parents to Forego Pertussis Vaccination for their Children, 49 J. Clini. Epidemiol. 697 (1996) and Ilana Ritov & Jonathan Baron, Reference Points and Omission Bias, 59 Org. Behav. & Hum. Decision Processes 475 (1994).
[11] David Pilgrim & Anne Rogers, Mass Childhood Immunization: Some Ethical Doubts for Primary Health Care Workers, 2 Nursing Ethics 63 (1995).
[12] Meszaros et al., supra note 10.
[13] Eugene J. Gangerosa et al., An Historical Analysis of the Impact of the Antivaccine Movements on the Control of Pertussis (1996) (unpublished CDC).
[14] See, e.g., Nancy J. Binkin et al., Epidemiology of Pertussis in a Developed Country With Low Vaccination Coverage: the Italian Experience, 11 Pediatr. Infect. Dis. J. 653 (1992).
[15] CDC, State Immunization Requirements 1993-94 (1994).
[16] Peggy O'Mara, Vaccination: The Issue of Our Times, Mothering, Summer 1996, at 24. The risk of vaccine-associated paralysis of the person receiving OPV is estimated at approximately 1 in 1.5 million for the first dose; see CDC, Polio Vaccine Information Statement (1994).
[17] S. Plotkin, Pertussis, 7 Vaccine 195 (1989) (editorial).
[18] See, e.g., Baruch Fischhoff, Risk Perception and Communication Unplugged: Twenty Years of Process, 15 Risk Anal. 137 (1995); Baruch Fischhoff et al., Acceptable Risk (1981); Baruch Fischhoff et al., How Safe is Safe Enough? A Psychometric Study of Attitudes Towards Technological Risks and Benefits, 8 Policy Sci. 127 (1978) and Paul Slovic, Perceptions of Risk, 236 Science 280 (1987).
[19] Paul Slovic, Sarah Lichtenstein & Baruch Fischhoff, Modeling the Societal Impact of Fatal Accidents, 30 Management Sci. 464 (1984) and Jonathan Baron Thinking and Deciding (2d Ed.1994).
[20] Illustrated in Meszaros et al., supra note 10; C. C. G. Lord et al, Biased Assimilation and Attitude Polarization: The Effects of Prior Theories on Subsequently Considered Evidence, 37 J. Personality
& Soc. Psych. 2098 (1979); see also, Ido Erev et al., Vagueness, Ambiguity, and the Cost of Mutual Understanding, 2 Psych. Sci. 321 (1991).
[21] Fischhoff, supra note 18 and Francis M. Lynn & George J. Busenberg, Citizen Advisory Committees and Environmental Policy: What We Know, What's Left to Discover, 15 Risk Anal. 147 (1995).
[22] See, e.g., Lee Clarke & James F. Short, Jr., Social Organization and Risk: Some Current Controversies, 19 Ann. Rev. Sociol. 375 (1993); Karl Dake, Myths of Nature: Culture and the Social Construction of Risk, 48 J. Social Iss. 21, n. 4 (1992) and Mary Douglas & Aaron Wildavsky, Risk and Culture (1982).
[23] Sarah Lichtenstein et al., Judged frequency of Lethal Events, 4 J. Exp. Psych. Hum. Learn. & Memory 551 (1978). For a summary see, Baruch Fischhoff, Ann Bostrom & Marilyn Jacobs-Quadrel, Risk Perception and Communication, 14 Ann. Rev. Pub. Health 183 (1993).
[24] Amos Tversky & Daniel Kahneman, Judgment Under Uncertainty: Heuristics and Biases, 185 Science 1124 (1974).
[25] See Jeryl L. Mumpower, Lottery Games and Risky Technologies: Communications about Low-Probability/High-Consequence Events, 8 Risk Anal. 231 (1988) (draws parallels between lotteries and risks from technological hazards, pointing out that lotteries may be successful in part because they emphasize and elaborate the details of the extreme consequences, ignore probabilities, stress the notion of possibilities and emphasize absolute frequencies rather than base rates).
[26] Judgment Under Uncertainty: Heuristics and Biases (Daniel Kahneman, Paul Slovic & Amos Tversky, eds. 1982) and Daniel Kahneman & Amos Tversky, Subjective Probability: A Judgment of Representativeness, 3 Cogn. Psych. 430 (1972).
[27] National Research Council, Improving Risk Communication (1989).
[28] Sheila Jasanoff, Presentation, Symposium, Managing the Problem of Industrial Hazards: The International Policy Issues, National Academy of Sciences, Washington D.C 1989.
[29] Supra note 3.
[30] Id.; 6 Common Misconceptions targets providers but has not received extensive empirical evaluation.
[31] Caron Chess, Kandice L .Salomone & Peter M. Sandman, Risk Communication Activities of State Health Agencies, 81 Am. J. Pub. Health 489 (1991).
[32] See, e.g., Timothy C. Earle & George T. Cvetkovich, Social Trust: Toward a Cosmopolitan Society (1995).
[33] See, e.g., J. Marquart, G. J. O'Keefe & A. C. Gunther, Believing in Biotech: Farmers' Perceptions of the Credibility of BGH Information Sources, 16 Science Comm. 388 (1995).
[34] David B. McCallum, Sharon L. Hammond & Vincent T. Covello, Communicating about Environmental Risks: How the Public Uses and Perceives Information Sources, 18 Health Ed. Q. 349 (1991).
[35] Jon F. Merz et al., A Decision-Analytic Approach to Developing Standards of Disclosure for Medical Informed Consent, 15 J. Products & Toxics Liability 191 (1993).
[36] Helmut Jungermann Holger Schütz & M. Thüring, Mental Models in Risk Assessment: Informing People about Drugs, 8 Risk Anal. 147 (1988).
[37] Supra note 11.
[38] Jonathan Baron, Blind Justice: Fairness to Groups and the Do-No-Harm Principle, 8 J. Behav. Decision Making 71 (1995).
[39] Id.
[40] John C. Hershey et al., The Roles of Altruism, Free Riding, and Bandwagoning in Vaccination Decisions, 59 Org. Behav. & Hum. Decision Processes 177 (1994).
[41] Personal communication with Jenifer Lloyd, CDC. The study is described in a 1996 working paper, The Feasibility of Incorporating Inactivated Poliovirus Vaccine into the Childhood Immunization Schedule: Perspectives of Georgia Public Providers and Parents, by Lloyd & Kris Bisgard.
[42] Garrett Hardin, The Tragedy of the Commons, 162 Science 1243 (1968).
[43] Baruch Fischhoff & Jon F. Merz, The Inconvenient Public, Chemtech, Feb. 1995, at 47 and Merz et al., supra note 35.
[44] Peter Ubel & George Loewenstein, The Role of Decision Analysis in Informed Consent: Choosing between Intuition and Systematicity (1996) (unpublished, Center for Bioethics, U. Penn.).
[45] Designing Health Messages (Edward Maibach & Roxanne Louiselle Parrott, eds. 1995).
[46] Stages of change models show that attitudes and behaviors shift in stages that cumulatively result in health risk changes. See, e.g., David R. Holtgrave, Barbara J. Tinsley & Linda S. Kay, Encouraging Risk Reduction: A Decision-Making Approach to Message Design, in Designing Health Messages, supra; J. O. Prochaska et al., In Search of How People Change: Applications to Addictive Behaviors, 47 Am. Psych. 1102 (1992) and Neil D. Weinstein & Peter M. Sandman, A Model of the Precaution Adoption Process: Evidence from Home Radon Testing, 11 Health Psych. 170 (1992).
[47] See, e.g., Artur M. Galazka & Susan E. Robertson, Diptheria: Changing Patterns in the Developing World and the Industrialized World, 11 Europ. J. Epidemiol. 107 (1995) and Gangerosa et al., supra note 13.
[48] See, e.g., Holtgrave et al., supra note 46.
[49] See, e.g., Asch et al. and Meszaros et al., supra note 10.
[50] Cf. Roger E. Kasperson et al., The Social Amplification of Risk: A Conceptual Framework, 8 Risk Anal. 177 (1988) and Ortwin Renn et al., The Social Amplification of Risk: Theoretical Foundations and Empirical Observations, 48 J. Soc. Issues 137 (1992).
[51] Robert J. Griffin, Sharon Dunwoody & Christine Gehrmann, The Effects of Community Pluralisms on Press Coverage of Health Risks from Local Environmental Contamination, 15 Risk Anal. 449 (1995).
[52] Michael D. Slater, Choosing Audience Segmentation Strategies and Methods for Health Communication, in Designing Health Messages, supra note 45.
[53] Baruch Fischhoff, Treating the Public with Risk Communications: A Public Health Perspective, 12 Sci. Tech. Hum. Values 13 (1987).
[54] Min-Sun Kim, Attitude-Behavior Relations: A Meta-Analysis of Attitudinal Relevance and Topic, 43 J. Comm. 101 (1993).
[55] Marshall H. Becker, Theoretical Models of Adherence and Strategies for Improving Adherence, in The Handbook of Health Behavior Change (Sally A. Shumaker et al., eds. 1991).
[56] Icek Ajzen & Martin Fishbein, Understanding Attitudes and Predicting Social Behavior (1980).
[57] R. W. Rogers, A Protection Motivation Theory of Fear Appeals and Attitude Change, 91 J. Psych. 93 (1975); R. W. Rogers, Cognitive and Physiological Processes in Fear Appeals and Attitude Change: A Revised Theory of Protection Motivation, in Social Psychophysiology (John T. Cacioppo & Richard E. Petty, eds. 1983).
[58] Albert Bandura, Social Foundations of Thought and Action: A Social Cognitive Approach (1986).
[59] Sharon Sutton, Fear Arousing Communications: A Critical Examination of Theory and Research, in Social Psychology and Behavioral Medicine 303 (J. Richard Eiser, ed. 1987).
[60] Kim Witte, Putting the Fear Back into Fear Appeals: The Extended Parallel Process Model, 59 Comm. Monographs 329 (1992) and Designing Health Messages supra note 45
Tangential, yet related, are cognitive theories of memory and learning. The effects of common sense or "folk" models of physical phenomena and processes on how people learn and behave have been studied extensively, see, e.g., Mental Models (Diedre Gentner & A. L. Stevens, eds.1983) and Willett Kempton, Variation in Folk Models and Consequent Behavior, 31 Am. Behavioral Sci. 203 (1987). The overlap between these somewhat disparate areas has been examined in a few studies, see, e.g., Richard Lau et al., Further Explorations of Common-Sense Representations of Common Illnesses, 8 Health Psych. 195 (1989).
[61] See, e.g., Public Health Service Influenza and Pneumococcal Vaccination Coverage Levels Among Persons Aged 65 years -- United States, 1973-1993, 44 Morbidity & Mortality Wkly. Rep. 506-507, 513-515 (1995).
[62] For an exception, see, e.g., Neil D. Weinstein, Peter M. Sandman & Nancy E. Roberts, Deter
minants of Self-Protective Behavior: Home Radon Testing, 20 J. App.l Soc. Psych. 783 (1990).
[63] Fischhoff et al., supra note 18.
[64] See, e.g., Michilene T. Chi, P. J. Feltovich & R. Glaser, Categorization and Representation of Physics Problems by Experts and Novices, 5 Cogn. Sci. 121 (1981) and J. Otero & W. Kintsch, Failures to Detect Contradictions in a Text: What Readers Believe versus What They Read, 3 Psych. Sci. 229 (1992).
[65] See, e.g., Ann Bostrom, Baruch Fischhoff & M. Granger Morgan, Characterizing Mental Models of Hazardous Processes: A Methodology and an Application to Radon, 48 J. Social Issues 85 (1992). This approach has been used, e.g., to develop and test communications for indoor radon; see Cynthia J. Atman et al., Designing Risk Communications: Completing and Correcting Mental Models of Hazardous Processes, Part I, 14 Risk Anal. 779 (1994); Ann Bostrom et al., Evaluating Risk Communications: Completing and Correcting Mental Models of Hazardous Processes, Part II, 14 Risk Anal. 789 (1994); Ann Bostrom et al., Public Knowledge about Indoor Radon: The Effects of Risk Communication, in Decision Making under Risk and Uncertainty: New Models and Empirical Findings 243 (John Geweke, ed. 1992) and M. Granger Morgan et al., Communicating Risk to the Public, 26 Env'l Sci. & Tech. 2048 (1992). See also, Jungermann et al., supra note 36 for a "mental models" approach developed earlier.
[66] Id.; see also, Ann Bostrom et al., A Mental Models Approach to the Preparation of Summary Reports on Ecological Issues Related to Dispersant Use (1995) and Michael Maharik & Baruch Fischhoff, The Risks of Using Nuclear Energy Sources in Space: Some Lay Activists' Perceptions, 12 Risk Anal. 383 (1992).
[67] Morgan, supra note 65.
[68] Bostrom et al. (1992), supra note 65.
[69] Atman, supra note 65.
[70] See Designing Health Messages supra note 45 for several examples.
[71] See, e.g., Ann Bostrom et al., What Do People Know about Global Climate Change? 1. Mental Models, 14 Risk Anal. 959 (1994).
[72] See, e.g., K. Anders Ericsson & N. Charness, Expert Performance: Its Structure and Acquisition, 49 Am Psychologist 725 (1994) and James F. Shanteau, Competence in Experts: The Role of Task Characteristic, 53 Org. Behav. & Hum. Decision Processes 252 (1992).
[73] See, e.g., Chi et al., supra note 64.
[74] See Fischhoff & Merz, supra note 42; NRC, supra note 27 and Shanteau, supra note 72.
[75] Richard P. Barke & Hank C. Jenkins-Smith, Politics and Scientific Expertise: Scientists, Risk Perception, and Nuclear Waste Policy, 13 Risk Anal. 425 (1993).
[76] M. Granger Morgan & Max Henrion, Uncertainty (1990).
[77] Wibecke Brun, Risk Perception: Main Issues, Approaches and Findings, in Subjective Probability 295 (George Wright & Peter Ayton, eds. 1994).
[78] Norman R. Brown & Robert S. Siegler, Metrics and Mappings: A Framework for Understanding Real-World Quantitative Estimation, 100 Psych. Rev. 511 (1993).
[79] Id.
[80] See, e.g., Baron, supra note 38.
[81] See, e.g., David V. Budescu & Thomas S. Wallsten, Consistency in Interpretation of Probabilistic Phrases, 36 Org. Behav. & Hum. Decision Processes 391 (1985) and Thomas S. Wallsten et al., Measuring the Vague Meanings of Probability Terms, 115 J. Exp. Psych. General 348 (1986).
[82] Patricia W. Linville, Gregory W. Fischer & Baruch Fischhoff Perceived Risk and Decision Making Involving AIDS, in The Social Psychology of HIV Infection (J. B. Pryor & G. D. Reeder, eds. 1983).
[83] Cf. Ola Svenson, Are We All Less Risky and More Skillful Than Our Fellow Drivers? 47 Acta Psychol. 142 (1981).
[84] Lord et al., supra note 19; Meszaros et al., supra note 10 and Erev et al., supra note 19.
[85] Science Advisory Board, U.S. EPA,
An SAB Report: Review of EPA's Approach to Screening for Radioactive Waste Materials at a Superfund Site in Uniontown, Ohio (1994) (prepared by the ad hoc Industrial Excess Landfill Panel of the Board).
[86] CDC, supra note 3, at 29.
[87] See e.g., William R. Freudenburg & J. A. Rursch, The Risks of "Putting the Numbers in Context" -- A Cautionary Tale, 14 Risk Anal. 949 (1994) and Emilie Roth et al., What Do We Know About Making Risk Comparisons? 10 Risk Anal. 375 (1990).
[88] John B. Loomis & Pierre H. DuVair, Evaluating the Effect of Alternative Risk Communication Devices on Willingness to Pay: Results from a Dichotomous Choice Contingent Valuation Experiment, 69 Land Econ. 287 (1993).
[89] Peter M. Sandman, Neil D. Weinstein & Paul M. Miller, High Risk or Low: How Location on a "Risk Ladder" Affects Perceived Risk, 14 Risk Anal. 35 (1994).
[90] Uncertainty, supra note 76.
[91] Kip W. Viscusi, Wesley A. Magat & Joel Huber, Communication of Ambiguous Risk Information, 31 Theory & Decision 159 (1991).
[92] Christopher K. Hsee, Elastic Justification: How Tempting but Task-Irrelevant Factors Influence Decisions, 62 Org. Behav. & Hum. Decision Processes 330 (1995).
[93] Branden B. Johnson & Paul Slovic, Presenting Uncertainty in Health Risk Assessment: Initial Studies of Its Effects on Risk Perception and Trust, 15 Risk Anal. 485 (1995).
[94] Morgan et al. 1992, supra note 6.
[95] Bostrom et al. 1994a, supra note 65; Evaluation and Effective Risk Communication -- Workshop Proceedings (Ann Fisher, Maria Pavlova & Vincent Covello, eds. 1992) and Bernd Rohrmann, The Evaluation of Risk Communication Effectiveness, 81 Acta Psychol. 169 (1992).
[96] Karen Schriver, Evaluating Text Quality: The Continuum from Text-Focused to Reader-Focused Methods, 32 IEEE Trans. Prof. Comm. 238 (1989).
[97] See, e.g., K. Anders Ericsson, Concurrent Verbal Reports on Text Comprehension: A Review, 8 Text 295 (1988).
[98] See, e.g., Barbara Means & Elizabeth F Loftus, When Personal History Repeats Itself: Decomposing Memories for Recurring Events, 5 Appl. Cogn. Psych. 297 (1991).
[99] Bostrom, Fischhoff & Morgan, supra note 65.
[100] Id.
[101] See, e.g., Chi, Feltovich & Glaser, supra note 64.
[102] Katherine E. Rowan, When Simple Language Fails: Presenting Difficult Science to the Public, 21 Tech. Writing & Comm. 369 (1991).
[103] But see Baruch Fischhoff, Value Elicitation: Is There Anything in There? 46 The Am. Psych. 835 (1991).
[104] See, e.g., James Flynn et al., Trust as a Determinant of Opposition to a High-Level Radioactive Waste Repository: Analysis of a Structural Model, 12 Risk Anal. 417 (1992).
[105] See, e.g., Karl Dake & Aaron Wildavsky, Theories of Risk Perception: Who Fears What and Why?, 119 Daedalus 41 (1990).
[106] Morgan et al. 1992, supra note 65.
[107] Galen E. Cole, David R. Holtgrave & Nilka M. Rios, Systematic Development of Trans-Theoretically based Behavioral Risk Management Programs, 4 Risk 67 (1993).
[108] See, e.g., Roth et al., supra note 87.
[109] Asch, supra note 10.
[110] CDC, supra note 3.
[111] O'Mara, supra note 16.
[112] Chess, supra note 31.
[113] Penny Hatcher, Comments and Recommendations (CDC 1993) (summary of a 2.5 week phone survey during 1993).