Imagine, all notifications would go silent. No more buzzing, flashing, beeping, or pop-ups.
This is exactly the situation that we created in the Do Not Disturb Study. The study was conducted as part of a collaboration between the Scientific Group of Telefonica R&D and Human-Computer Interaction Institute at Carnegie Mellon University, Luz Rello and I envisioned the Do Not Disturb Challenge.
The Do Not Disturb Challenge
We asked 30 volunteers to disable notification alerts for 24 hours across all devices and all services. We carefully walked the participants through all devices and services. Where possible, we used system-wide settings, such as the Do Not Disturb mode, to suppress all alerts. In other cases, such as Skype, we showed people how to disable notifications in the settings of the respective app. Please note that participants could still read messages or emails, they would simply not receive any alert.
To study the effect that notifications have on us, we captured self-reported feedback, and compared it to the same self-reported feedback collected via questionnaire during a normal baseline day. Furthermore, after the study, we conducted a post-hoc interview to uncover themes that we had not anticipated. We discovered the following main effects.
Drop in Engagement and Reduced Responsiveness
The absence of notifications had a significant effect on how participants perceived their engagement with the mobile phone. For example, Participant #02 “forgot my phone at work” because of not being reminded of the phone by notifications.
Increased Productivity
As expected, notifications distract. Hence, the answers of the questionnaire show that participants felt significantly less distracted and more productive: Participant #07 said that it was “easier to concentrate, especially when working on the desktop (computer).”
Lack of notifications caused to miss information
During the day without notifications, participants were significantly more likely to agree with the statements that they missed professional or personal information. During the post-hoc interview, we collected several anecdotes. For example: because of the lack of notifications, Participants #12 forgot to continue a chat with a friend. As a consequence, this friend got angry for not receiving replies.
Lack of notifications induced worry
Consequently, participants were significantly more likely to agree to the statement “I felt worried about missing notifications“. For example, Participants #04 “was meeting with [a friend] for lunch, and I knew that I was going to receive something from her“.
More frequent checking of the phone
During the day without notifications, agreement to the statement “I frequently turned on the phone to check for missed notifications“. For example, Participant #12 stated that “because of the reaction of my friend, who got angry because I forgot to respond, I was the whole afternoon with phone in my hand.”
Stress
Interestingly, there were no significant effects on the two stress related items, neither on “I felt stressed” nor on “I felt relaxed“. This might be explained by the finding that there are two opposing stress-inducing effects at work — stress from the interruptions and stress from being anxious to miss important information or violate expectations –, which influenced participants to different extents.
Reduced feeling of social connectedness
Our study revealed a link between notifications and staying emotionally in touch with one’s social group. During the day without notifications, agreement to the statement: “I felt connected with my social group” was significantly lower. These results contrast that — while work-wise, disabling notifications helped to be more focused and productive — socially, they negatively affect the feeling of being in touch with one’s social group.
Polarized reactions to being without notifications
The participants’ post-study reflections to having notifications disabled varied greatly. They ranged from very positive responses, such as “It was amazing! I felt liberated! (Participant #22) over neutral responses, such as “It was not a big deal, since I am usually not checking notifications and people know that I am not responsive” (Participant #25) to very negative responses, such as “I was paranoid and I even left the screen on not to miss a friends notification“} (Participant #04).
The main predictor for the participants’ attitude that we observed was to what extent others typically expected them to respond quickly to messages: the faster the usual response, the more negative the experience.
Signs of notifications overload
For more than two-third of the participants, the participation in the Do Not Disturb Study caused them to reflect on their notifications usage. Almost half of the participants stated the plan to use Do Not Disturb or similar similar notification-suppression modes in the future. For example, Participants #24 realized that “when I need to really get things done, I need to turn notifications off.”
One third stated the plan to manage notifications more consciously. For example, Participants #20 was “considering to only keep notifications for the important things, so people can better reach me“. Participants #26 had come to the conclusion that the “important apps are Messenger, Hangout and WhatsApp.” This shows how important instant messaging has become: people depend on notifications to maintain the expected level of responsiveness. This also shows that – despite the negative effects of notifications – disabling them altogether is not an option.
Two years later, we contacted the 22 participants who intended to manage notifications differently in the future. More than 75% of the participants had followed or followed partially through with their plans.
The fact that more than half of the participants reduced the number of notifications that they are exposed to on a daily basis is a warning sign that our participants were realizing a sense notification overload.
Conclusions
In conclusion, our results show strong and polarized reactions to being without notifications:
Notifications negatively impacted focused work, as participants reported to feel significantly less distracted and more productive without them. This is evidence that disabling notifications can have positive effects.
At the same time, disabling notifications also had significant negative effects: it made participants more worried to miss important information, not being responsive enough, and feeling less connected with their social network. Thus, disabling notifications altogether is not an option.
In contrast to a previous deprivation study, where all participants re-enabled work email notifications after the study, about one-third of our participants expressed the intention to disable some sources of notifications, and about half of our participants expressed the intention to use Do Not Disturb (and equivalent settings) more often in the future. Two years later, 60% of these participants are still following through with their intentions. Another 18% have changed their notification-related behavior.
TL/DR: Notifications. Can’t live with them, can’t live without them.
The work will be presented at ACM MobileHCI ’17, the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services, which will take place in September 2017 in Vienna, Austria.
Citation:
Productive, Anxious, Lonely – 24 Hours Without Push Notifications.
Martin Pielot and Luz Rello.
MobileHCI ’17: ACM International Conference on Human-Computer Interaction with Mobile Devices and Services, 2017.
Press:
In this research, we – Kostadin Kushlev, University of Virginia, Bruno Cardoso, KU Leuven, and myself – explored to what extent users’ current affect, that is, how they are feeling, impacts user engagement. To this end, we analyzed data from a study conducted by Telefónica Research where the participants (N = 337) downloaded a custom-developed app that delivered notifications.
After attending to a notification (N = 32,704), participants reported how they felt in a mini questionnaire. Besides asking how the participants felt, the questionnaire also offered them to voluntarily engage with further content. Participants were not aware that we our main interest was in observing their interaction with said content — they believed that it was mainly there as a courtesy to make their participation in the study more fun.
