Why stay-home messaging works - and why it doesn't

Social distancing measures have been the cornerstone of the global effort to fight COVID-19 (Gourinchas 2020, Baldwin and Weder di Mauro 2020) and the main reason for the economic crisis that ensued (Furman 2020, Baldwin and Weder di Mauro 2020). Mandatory restrictions played an important role, especially in countries like France and Italy where strict lockdowns were imposed, but the effectiveness of social distancing largely rested on individual initiative and voluntary compliance (Maloney and Taskin 2020). Unsurprisingly, therefore, reminding people to stay home through messaging campaigns became an important tool for health authorities across the globe.

Have these messaging campaigns been effective? Did people stay home more as a result?

To address these questions, we conducted an experiment in Denmark at the end of March 2020, during the most critical period of the crisis.1 In collaboration with the Danish statistical office, we distributed different types of reminders encouraging people to ‘stay home as much as possible’ to a random sample of Danish residents. We then compared the behaviour of treated subjects to a control group that received no reminder (Falco and Zaccagni 2020).

To identify what types of messages are most (and least) effective in convincing people, we randomly changed the focus of the reminder. Specifically, we varied the emotional proximity of the recipients to the subject of the message by emphasising the risks of non-compliance for themselves, their family, other people in general, and the country as a whole. This allows us to investigate pro-social motives in depth, by analysing who are the relevant ‘others’ people worry about when acting pro-socially. In doing so, we extend the recent literature on the subject (Jordan et al. 2020, Campos-Mercade et al. 2020, Borgonovi and Andrieu 2020).

Less than 15% of people stayed home in Denmark at the peak of the crisis

Our first finding is that the share of people who stayed home at the height of the crisis was quite low. Less than 15% of people across Denmark stayed home on any given day in the two weeks of the study (25 March – 7 April). Since our data is self-reported, this low share may even be an overestimate of the actual figure, as social desirability bias may lead respondents to over-report having stayed home. The proportion was higher among women and lower-income families.

We also find a large gap between what people say they will do and what they end up doing. By asking subjects whether they planned to go out the next day and then checking what their behaviour was with a follow-up interview, we discover that over 40% of the people who declared an intention to stay home went out instead.

People who receive the reminder say they will stay home more, but they often deviate from their intentions

People who received the reminder (with a focus on ‘you’ and ‘your family’) were more likely than non-recipients to say they would stay home the day after, but they were no more likely to report that they did stay home in the follow-up survey (see Figure 1, where asterisks indicate statistically significant effects).2 In other words, while reminders were effective in changing people’s intentions, they did not change their reported behaviour.

Figure 1 The effects of different reminders on intentions and actions

Notes: The reminder increases respondents’ intentions to stay home by 46% when it is framed with respect to personal risks and risks for one’s family. Those changes in intentions, however, do not translate into sizeable changes in actions (i.e. people actually staying home more), except for people in bad health (see Figure 2).

Technical notes: The effects are percentage changes relative to the share of people who intend to stay home or do stay home in the control group (below 15%). Stars reported at the top of the bars express the level of significance of the coefficient (** p<0.01, * p<0.05, * p<0.1). Bars without stars report treatment effects that are not significantly different from 0.

This is striking, especially in light of the fact that people who received the reminder may have felt compelled to over-report having stayed home. It is also an important contribution to the existing literature, which has typically drawn conclusions on the effectiveness of reminders by analysing intentions to comply with social distancing (Barari et al. 2020, Everett et al. 2020, Utych and Fowler 2020).

Reminders only change people’s intentions when they emphasise risks for the recipients and their families

We also show that some messages are more effective than others, albeit only in changing intentions to comply (Figure 1). When the reminders focus on personal risks and on the risks for the recipient’s family, they increase intentions to comply by almost 50%. On the other hand, when the reminders focus on the risks for other people in general or for the rest of society, we detect no significant impact. This poses an interesting contrast to the idea of social distancing as a civic duty.

While our findings are not inconsistent with the evidence that higher civic capital increases compliance with social distancing (Barrios et al. 2020, Durante et al. 2020), they indicate that people are most sensitive to the dangers to themselves and their families.

Reminders are more effective with people in poor health

While these results paint a sobering picture on the effectiveness of messaging campaigns, we find that reminders are effective in one important respect. They significantly increase, by more than 80%, the probability that people in bad health will stay home (Figure 2).

Figure 2 Treatment effects of the most effective reminders by the health status of the recipient

Notes: Among respondents who are in bad health, the share of those who declare they will stay home more than doubles after receiving a reminder that emphasises personal risks and risks for family, and the share of those who stay home increases by 80%. Among respondents who are in good health, the reminder increases intentions to stay home by 36.5% when the reminder emphasises personal risks and by 38.2% when it emphasises risks for family, but the results are only significant at the 10% level. The share of respondents in good health who do stay home increases by less than 10% after receiving those reminders and the effect is not statistically significant.

Technical notes: Respondents classify their health status on a 5-point scale. The lowest two values are considered bad health conditions for the purpose of this analysis. The effects are percentage changes relative to the share of people who intend to stay home or do stay home among control subjects in the same health category. Stars reported at the top of the bars express the level of significance of the coefficient (** p<0.01, * p<0.05, * p<0.1). Bars without stars report treatment effects that are not significantly different from 0.

This is an important conclusion for public authorities, who should prioritise groups at risk in their messaging campaigns. Such a strategy would bear more immediate results than trying to convince people who do not face immediate risks and appear to be largely insensitive to the reminders.

Our findings yield lessons for the international community as well. COVID-19 continues to be a threat in much of the world and promoting conscientious behaviour, including social distancing, remains a top policy priority. Yet, as governments relax their initial restrictions, keeping people alert will become progressively more difficult, and the gap between people’s intentions and their actions will exacerbate the problem. Our work indicates that reminders need to be targeted and timed well in order to make a difference.

Endnotes

1 In Denmark, the most stringent restrictions were in force from 11 March to mid-April. They began to be relaxed with the re-opening of nurseries and kindergartens on 15 April.

2 The fact that effects of 20% are not statistically significant should be interpreted in light of the observation that such impacts are in fact quite small and difficult to detect even with large samples. Since the share of people who stay home in the control group is below 15%, a positive effect of 20% translates into fewer than 3 extra people who stay home out of 100.