全球变化知识资源中心

The Intergovernmental Panel on Climate Change (IPCC) uses verbal descriptions of uncertainty (for example, Unlikely) to convey imprecision in its forecasts and conclusions. Previous studies showed that the American public misinterprets these probabilistic statements. We report results from a multi-national study involving 25 samples in 24 countries and 17 languages. As predicted, laypeople interpret IPCC statements as conveying probabilities closer to 50% than intended by the IPCC authors. We show that an alternative presentation format supplementing the verbal terms with numerical ranges increases the correspondence between the public’s interpretations and the IPCC guidelines, and the terms are better differentiated. These qualitative patterns are remarkably stable across all samples and languages. In fact, interpretations of the terms in various languages are more similar under the new presentation format. These results suggest changing the way the IPCC communicates uncertainty.

The IPCC assembles and disseminates information about global climate change (GCC). Findings and conclusions are compiled into periodical Assessment Reports informing policymakers and the public on issues relevant to the understanding of GCC. One important issue facing the IPCC is how to communicate the uncertainties in its models and predictions. This challenge applies to all risk communications, but the debates surrounding climate change are much more intense and politicized. Questions about the reality, severity and sources of GCC and the best ways to address it frequently occur in public and political debates. Some sources of uncertainty are inherent to the climate science. Others reflect the public’s imperfect understanding of climate-related issues and misperceptions about scientific consensus on the topic¹.

Probabilistic judgments can be communicated as precise numerical probabilities (for example, there is a 0.4 chance that X will occur), imprecise numerical probabilities (for example, the probability that X will occur is between 0.3 and 0.6) or probability phrases (for example, it is improbable that X will occur). The challenge facing the IPCC is to convey information with the level of precision warranted by the available evidence². Using precise (numerical) probabilities could be misleading, as it would imply too high a level of precision, and of the consensus among experts. In recent assessments the IPCC has used verbal descriptions of uncertainty such as Likely accompanied by a translation table reproduced in Table 1 (ref. 3). Recent empirical work^{4, 5} has questioned the efficiency of this method and has documented the superiority of an alternative dual-scale combining probability phrases and numerical ranges.

Climate change is a global problem and the IPCC issues its reports in all UN official languages and reports are translated into many other languages (http://www.ipcc.ch/publications_and_data/publications_and_data.htm). We seek to determine whether the method used by the IPCC to communicate uncertainty (verbal terms with the translation table in an appendix) resonates identically in all countries and languages. Are risks, estimates and forecasts listed in the reports interpreted as more, or less, severe and as more, or less, uncertain in various countries, simply because of the use of probability phrases? To this end, we conducted a large-scale multi-national study. We focus on four verbal terms used in the IPCC reports, and address the following questions. Are probabilistic pronouncements of the IPCC reports interpreted similarly everywhere? Can the alternative Verbal–Numerical (VN) scale improve the effectiveness of communication? Are any cross-national differences in the interpretations of the verbal assessments related to the overall level of belief in, and attitudes to, GCC in these countries?

Figure 1 shows the distribution of estimates of the IPCC terms—Very unlikely, Unlikely, Likely and Very likely—averaged across items using the same term across all 25 samples, separately for each condition. For each term we show the central 90% of the distribution, with the box covering the central 50%, and mark the median and the mean. The plot also includes horizontal lines at thresholds suggested by the IPCC guidelines. The distributions are too regressive (closer to 50% than the IPCC prescriptions) but the distributions are more extreme under the VN presentation. Consequently, the terms are better differentiated under the dual format as shown in Supplementary Fig. 1.1 in Appendix 1. These results are replicated in all 25 samples (Supplementary Table 1.1 in Appendix 1).

For each term we show the central 90% of the distributions, with the box covering the central 50%, the horizontal line marking the median and the plus symbol marking the mean. The dashed horizontal lines mark the IPCC guidelines. T, Translation; VN, Verbal–Numerical.

We counted for every respondent the number of estimates, out of the 8, that were consistent with IPCC guidelines. Figure 2 shows the cumulative distribution of these counts in both conditions across all 25 samples. The distribution in the VN condition stochastically dominates (for every proportion there are more judges with higher levels of consistency under the VN condition) the one from the Translation (mean consistency of 40% versus 27%). This pattern holds for all terms, but the effect is more pronounced for Very unlikely and Very likely (Fig. 1.2, in Appendix 1). Figure 3 shows that consistency with the IPCC guidelines improves under the VN presentation mode in all samples. The overall proportional reduction in error (PRE = (VN compliance−Translation compliance)/(100% −Translation compliance) ranges from 7% (Korea) to 29% (Israel) with an overall mean of 17% (Supplementary Fig. 1.4 in Appendix 1).

The numbers in parentheses indicate the proportion of judges at each point.

Moving beyond the differences between the two conditions, we turn to agreement between samples exposed to the same terms in different languages. Let F_wi and F_wj be the distribution of numerical estimates for any two terms, W_i and W_j, and define the overlap between the two distributions: . This measure ranges from 0 (when the distributions do not overlap) to 1 (when the distributions coincide). We calculated overlap measures between the terms in all of the samples, and performed non-metric multidimensional scaling analysis. The two panels in Fig. 4 present solutions for the two conditions. Each plot includes 100 points (4 terms × 25 samples), and the closer two points are, the higher the overlap between their distributions. Both solutions show four clusters—one for each term. One dimension distinguishes the high/positive terms and the low/negative ones, and the other differentiates between the central and extreme terms. To highlight this pattern countries’ names are omitted (individual countries’ coordinates are presented in Supplementary Table 1.2 in Appendix 1).

See coordinates in Supplementary Appendix 1.

We correlated the level of consistency with IPCC guidelines in each sample with mean responses to the various questionnaires administered in that country. The level of consistency with the guidelines is higher, on average, in countries where respondents reported higher pro-environmental attitudes and higher levels of belief in, and concern about consequences of, GCC in both presentation modes. We will present these results in more detail in future papers. Importantly, no measure was significantly correlated with any measure of improvement in consistency under the VN presentation (Supplementary Table 1.3 in Appendix 1). Thus, the benefits of the improved presentation mode are, essentially, uniform across samples and languages.

In all samples, people interpreted the probabilistic pronouncements of the IPCC regressively, suggesting that they would underestimate (overestimate) the high (low) probabilities that the reports seek to communicate. What is the source of this excessive and, to some, surprising level of departure from the prescribed values? This pattern is, most likely, due to a combination of cognitive factors such as the vagueness and elasticity of language and the tendency of listeners to qualify and discount information provided by communicators, and motivated reasoning driven by preconceptions about the subject matter⁶. Next, we use auxiliary results in our survey—judgments of the same terms by the same respondents in the absence of any context—to highlight some explanations, and consider their implications for the IPCC.

It is possible that the guidelines seem ‘artificial’ and are incompatible with people’s ‘natural’ interpretation of the terms. There is some plausibility to this assertion: the median lower and upper judged bounds (out of context) of the target terms are less extreme than the values in the translation table: 9% ≤ Very unlikely ≤ 30%; 13% ≤ Unlikely ≤ 39%; 46% ≤ Likely ≤ 74%; 51% ≤ Very likely ≤ 86%. This can account for part of the regressive pattern, especially for the two extreme terms, and also explains why the VN presentation is especially effective in these cases. This result highlights the need to rethink the choice of terms and, more importantly, choose the bounds in the translation table on the basis of hard data.

Another possible explanation is that judges regress towards 50% to ‘signal’ that they do not know much about climate change, as responses to the knowledge items in our, and other surveys³³ indicate. We compared for each person the mean absolute deviation (MAD) from 50% across all four terms, in the absence of context, and for the IPCC items. When asked to quantify the terms in the absence of a specific context, the average MAD is 24.1 (23.7 in the Translation group and 24.6 in the VN group). In the Translation group, it is significantly lower (MAD = 20.2) when answering IPCC items, and 61% of the judges are more regressive in their interpretation of the same terms in the context of climate change. This pattern is significantly attenuated, in the VN group, where MAD = 23.0. Only 54% of the judges in this group are more regressive when answering IPCC items, and in 7 of the 25 samples, this pattern is actually reversed. Thus, another benefit of the VN format is that it mitigates the tendency to revert to the ignorance prior.

To conclude, the proposed method—presenting the verbal term and its corresponding numerical range—is highly effective and beneficial: the interpretations of the terms were more extreme and judges differentiated better between them; the range of values people associate with the various terms was reduced; and the level of agreement with IPCC guidelines increased significantly. Remarkably, these results were observed in all samples, regardless of their mean views of, levels of belief in, and experience with GCC. In fact, the VN format reduced inter-sample variability in interpretation, and the meaning of terms in various languages is more uniform.

These results make a compelling argument that to improve its communication, the IPCC should recalibrate the bounds defining their terms and adopt the dual VN format. Although our focus has been on how readers interpret the report, this change would also remind the authors of the intended meaning of the terms and maximize their level of adherence to the guidelines.

Instruments.

The survey contained the following components. (1) Sentences with probability phrases: participants judged 8 sentences from IPCC reports (listed in Supplementary Appendix 2) in their native language. We used two sentences containing each of the four target terms: Very unlikely, Unlikely, Likely and Very likely. For each sentence the respondents provided their best estimate and estimates of the lower and upper bounds of the report’s intended meaning. (2) Experience with, and perceptions of, global warming: a questionnaire probing the respondents’ experiences with, perceptions of, and attitudes to global warming (Supplementary Appendix 2) including subscales measuring belief in global warming, personal experience of global warming, perceived causes of global warming, perceived consequences of global warming and belief in free market system^{5, 28}. (3) Attitudes to environment: two scales measuring environmental worldview^{29, 30}. (4) Knowledge about global warming: two scales measuring endorsement of real and bogus causes of global warming³¹. (5) For intentions to act: three scales measuring endorsement of mitigation actions and the belief in self-efficacy^{28, 31}. (6) Numeracy: a scale consisting of 5 questions from ref. 32. (7) Context-free probability phrases: the participants were asked to provide the best numerical translation and range (lower and upper bounds) for each of the 4 target terms in everyday language (with no specific context). (8) For demographic information: age, sex, education, ethnicity, political affiliation, and income.

Translation.

The English forms of all these questionnaires were translated (and back translated for validation) to the various languages by the local coordinator. Official translations of the IPCC items were adopted if available.

Participants.

We administered the survey to 27 samples in 25 countries in 18 languages. We sampled adults (18 years or older) and the targeted sample size was 400 in every country. A total of 13,014 people completed the survey (recruited through Survey Sampling International, recruitment and compensation tailored to individual countries). After eliminating participants who completed the survey in less than 8 min; answered less than 50% of the questions, and answered identically (straight lining) questions on several pages, we retained a sample of 10,792 valid responses. The median duration of administration was 19 min. Supplementary Appendix 3 presents key demographic information for the various samples. The two Arabic-speaking samples (Egypt and Israel) were considerably younger and male dominated than the other samples, so they are excluded from the main analyses based on 25 samples and 10,239 respondents. This exclusion does not affect any of the key results as the same patterns were observed in these samples as well (Supplementary Table 1.1. in Appendix 1).

Experimental design.

Participants in each country were randomly assigned to one of two groups: Translation and Verbal–Numerical (VN). The difference between them was the way probability phrases were presented. The Translation group saw the IPCC items as they appear in the text of the report. The translation table was described in the instructions and accessible throughout the study (Supplementary Appendix 4). On average subjects in the Translation condition clicked to show the IPCC guid