Today, we are always online and connected through a multitude of personal mobile devices. More than ever, we receive messages, emails, and status-updates, which desire our attention: we have gotten used to the constant stimuli to a point where we become anxious when there are no new signals.

In this world, human attention has become scarce and thus valuable. Many services can greatly be greatly improved if they don’t interrupt people and request their attention at the right time. At the same time, we need to preserve downtimes to facility self-reflection and creativity.

Below you find a selection of research and innovation projects I did as member of the Scientific Group at Telefónica Innovation Unit in Barcelona, Spain.

2018

Smart Notifications

Smart Notifications is the name of an innovation project that started in 2016 following the Lean Startup method. The goal was to create product around an AI for the real-time prediction of opportune moments to be receiving push notifications. It emerged out of my research on prediction episodes of boredom from mobile phone use patterns. With different internal and external business units, we proved that the AI technology can significantly increase click-through rates for push notification campaigns.

In June 2018, the project consists of 3 developers, 2 data scientists, 3 people with business background, and myself as founder & scientific lead. The technology currently is being transferred to Telefonica’s business units.

More info: Preciso.ai

Studying Typical Phone Use Habits

Mobile phone use is suspected to have negative impacts, such as being addictive, tearing us apart, and thus affecting our mental well-being. We ran an unsupervised machine-learning algorithm on phone use data traces of 340 mobile phone users. We discovered 5 types of phone use: limited use, business use, power use, and two types of users with signs of negative well being. The most important insight is that intense phone use does not predict negative well-being. Instead, nightly use sessions were far more predictive.

Typical Phone Use Habits: Intense Use Does Not Predict Negative Well-Being.
Kleomenis Katevas, Ioannis Arapakis, Martin Pielot
MobileHCI ’18: ACM International Conference on Human-Computer Interaction with Mobile Devices and Services, 2018.

2017

Beyond Interruptibility: Predicting Opportune Moments to Engage Mobile Phone Users.

Push notifications are a popular way to engage mobile app users. However, badly timed push notifications can make feel people interrupted and annoyed, which can damage the brand perception. Based on 120 Million phone-use events and 78,930 emotion self-reports of 337 mobile phone users, we build a machine-learning model that — before delivering a notification — predicts whether a participant will click on the notification and subsequently engage with the offered content.

Beyond Interruptibility: Predicting Opportune Moments to Engage Mobile Phone Users.
Martin Pielot, Bruno Cardoso, Kleomenis Katevas, Joan Serrà, Aleksandar Matic, Nuria Oliver.
UbiComp ’17: ACM International Joint Conference on Pervasive and Ubiquitous Computing, 2017.

The Do Not Disturb Challenge : 24 Hours Without Notifications

How are notifications affecting our lives?

Before the success of smartphones, notifications were rare and usually came from calls, SMS, and alarms. Today, people handle dozens, some hundreds of notifications per day. How does this affect us?

We asked people to disable notifications on their phones, tablets, and computers for a full day. For many participants, it was a strong, insightful experience. While some participants felt less stressed and more concentrated, others felt no longer able to meet these social expectations towards responsiveness without notifications. In contrast to a previous deprivation study, where all participants re-enabled work email notifications after the study, about 22 of the 30 participants expressed plans to change their notification-related behaviour the future. Two years later, 60% of these participants are still following through with their intentions (more).

Productive, Anxious, Lonely – 24 Hours Without Push Notifications.
Martin Pielot and Luz Rello.
MobileHCI ’17: ACM International Conference on Human-Computer Interaction with Mobile Devices and Services, 2017.

The Do Not Disturb Challenge – A Day Without Notifications (Poster).
Martin Pielot and Luz Rello.
CHI EA ’15: Extended abstracts on Human factors in Computing Systems, 2015.

How Emotions Affect what Kind of Distractions we Welcome

How does the current emotional state of mobile phone users affect their openness to distractions?

In this research, we analyzed the data of 337 mobile phone users. These participants installed a study app that frequently delivered a notification, asking to report their emotional state. Clicking on the notification opened a questionnaire where they could report their affect. In addition, each questionnaire suggested two additional “distractors”, such playing a fun game (diverting tasks) or reading the article of the day (mentally demanding tasks).

When feeling good, people tend to avoid mentally demanding tasks. Hence, proactive recommendations for content that requires mental effort should target moments of neutral or even negative valence.
When tense, people tend to avoid diverting tasks. Thus, people who want to reduce task-induced stress might want to rely on external timers to schedule regular breaks with fun activities.
When energetic, people tend to avoid suggestions for further distraction altogether. Hence, proactive recommendations should target moments of low energetic arousal, such as moments of boredom.

Too Tense for Candy Crush: Affect Influences User Engagement With Proactively Suggested Content.
Kostadin Kushlev, Bruno Cardoso, Martin Pielot.
MobileHCI ’17: ACM International Conference on Human-Computer Interaction with Mobile Devices and Services, 2017.

2015

Detecting Boredom From Mobile Phone Usage

Can your mobile phone estimate when you are bored?

For two weeks, we logged how 54 people use their mobile phone, and frequently let them rate how bored they are. From this data, we show that it is possible to detect boredom from mobile phone. We exploit that fact that people often use their phone to kill time when they are bored, and that typical “killing-time” patterns can be learned by a machine-learning algorithm. We further show that during phases of detected boredom, people are significantly more open to suggested content (more, slides), which, amongst other things, enables boredom-triggered proactive recommendations.

When Attention is not Scarce – Detecting Boredom from Mobile Phone Usage (best-paper award).
Martin Pielot, Tilman Dingler, Jose San Pedro, and Nuria Oliver
UbiComp’ 15: ACM International Joint Conference on Pervasive and Ubiquitous Computing, 2015.

Inferring Boredom Proneness From Mobile Phone Usage

Can your phone infer whether you are prone to be bored?

Boredom proneness is a personal trait regarding your tendency to experience boredom of all types. For 22 participants of the boredom study, we had the scores of the Boredom Proneness Scale (BPS), an estimation of how prone to boredom they were. Then, we computed statistics of daily phone usage, such as how many apps they launched or how frequently they turned on the phone on a typical day. The data could distinguish with 80% accuracy whether a participant’s BPS score was above- or below-average. Participants with high boredom proneness tended to have more unstable phone usage patterns. However, they did not use the phone more then below-average scorers (more).

Boredom-Computer Interaction: Boredom Proneness and The Use of Smartphone.
Aleksandar Matic, Martin Pielot, and Nuria Oliver.
UbiComp’ 15: ACM International Joint Conference on Pervasive and Ubiquitous Computing, 2015.

Quantifying Attentiveness to Mobile Messages

Previous work has shown that people often attend notifications within minutes. But, what fraction of time during the day are people that attentive?

To answer this question, we analyzed a data set of how 42 people used their phone for two weeks, containing over 55,000 notifications from mobile messengers (e.g. WhatsApp, SMS, Facebook Messenger, ..) . For each minute of the day, we ran our previously proposed algorithm to predict attentiveness to mobile messages.We found that people were attentive to messages 12.1 hours a day, i.e. 84.8 hours per week, and provide statistical evidence how very short people’s inattentiveness lasts: in 75% of the cases mobile phone users return to their attentive state within 5 minutes. In particular the last finding is interesting, as it shows that delaying message notifications for a few minutes to wait for an opportune moment may be a viable strategy to avoid distractions.

I’ll be there for you : Quantifying Attentiveness towards Mobile Messaging.
Tilman Dingler and Martin Pielot.
MobileHCI ’15: ACM International Conference on Human-Computer Interaction with Mobile Devices and Services, 2015.

Boredom-Triggered Proactive Recommendations

Why is it interesting to know when a person is bored?

In this position paper, we argue that boredom may be an ideal, content-independent trigger for proactive recommendations. Restricting proactive recommendations to moments when a person is likely to be bored has advantages for both sides: publishers may achieve higher conversion rates, and at the same time, recommendations are less likely to arrive in situations where the simply annoy the receiver (more).

Boredom-Triggered Proactive Recommendations.
Martin Pielot, Linas Baltrunas, and Nuria Oliver.
Smarttention, Please! Intelligent Attention Management on Mobile Devices — Workshop @ MobileHCI, 2015.

2014

An In-Situ Study of Mobile Phone Notifications

How do people cope with notifications and how do notifications affect them?

We collected real-world notifications through a smartphone logging application alongside subjective perceptions of those notifications through an online diary. We found that notifications are often attended within minutes and that notification volume significantly correlates with emotions. We also found that notifications are often from social communication (messengers, email, social networks) where there are high expectations towards responsiveness. On the basis of our findings, we argue that avoiding interruptions from notifications may be viable for professional communication, while in personal communication, approaches should focus on managing expectations (blog post, poster, video, slides).

An In-Situ Study of Mobile Phone Notifications (best-paper award).
Martin Pielot, Karen Church, Rodrigo de Oliveira.
MobileHCI ’14: ACM International Conference on Human-Computer Interaction with Mobile Devices and Services, 2014.

Large-Scale Evaluation of Call-Availability Prediction

Can a mobile phone estimate whether its user will pick up a call or not?

To answer this question, we analyzed how 418 people handled 31311 phone calls. This work shows that mobile phones can predict with an accuracy of 83.2% (generic model) whether its user will accept an incoming phone call or not. When personalizing those models, the accuracy can be increased to 87% (blog post, slides).

Large-Scale Evaluation of Call-Availability Prediction.
Martin Pielot.
UbiComp ’14: ACM International Joint Conference on Pervasive and Ubiquitous Computing, 2014.

Didn’t You See My Message?

Can a mobile phone estimate whether its user is likely to see a message (WhatsApp, Facebook, SMS, …) within a few minutes?

In response to the finding from our In-Situ Study of Mobile Phone Notifications, this work proposes a way of communicating (non-)availability to messaging. This work shows that by monitoring phone sensors and user activity, the phone can predict whether its owner is likely to see a message within a few minutes (6.15) or not. The rationale is the following: if the prediction is shared with friends, they already know when they cannot expect a quick response to their messages. Our hope is that this will free people from the expectations towards responding timely (more). Try out Gone Fishing, the app that implements this algorithm to share your availability with another person.

Didn’t You See My Message? Predicting Attentiveness to Mobile Instant Messages
Martin Pielot, Rodrigo de Oliveira, Haewoon Kwak, Nuria Oliver
CHI 2014: ACM SIGCHI Conference on Human Factors in Computing Systems, 2014.

Large-Scale Assessment of Mobile Notifications

What notifications do people receive?

We made a large-scale analysis of the push notifications the people receive to their Android phones. We derive a holistic picture of notifications on mobile phones by collecting close to 200 million notifications from more than 40,000 users (slides).

A Large-scale assessment of mobile notifications
Alireza Sahami Shirazi, Niels Henze, Tilman Dingler, Martin Pielot, Dominik Weber, Albrecht Schmidt
CHI 2014: ACM SIGCHI Conference on Human Factors in Computing Systems, 2014.

2013

Peripheral Vibro-Tactile Displays

Can vibration pulses become peripheral?

For three days, 15 people carried a phone which emitted heartbeat-like vibro-tactile pulses. We asked them to set intensity of the vibrations to a level where they were barely perceiving it. They idea was that participants over time would forget about the vibration being there. To see whether the participants were still subconsciously aware of the vibration pulses, we stopped the vibration at random times. Participants then had to take the phone, confirm this, and fill our a short questionnaire. The results provide evidence that the vibration pulses entered the periphery of attention (more).
Peripheral Vibro-Tactile Displays.
Martin Pielot and Rodrigo de Oliveira.
MobileHCI ’13: 15th International Conference on Human-Computer Interaction with Mobile Devices and Services, 2013.

Ambient Timer – Unobtrusively Reminding Users of Upcoming Tasks

Can ambient light be used to create a non-distracting timer?

We equiped the back of a monitor with LEDs that would turn the wall behind the monitor into an ambient light display. We used this display to show whether an appointment in the calendar was nearing. The goal was to slowly catch the user’s attention without distracting them from their current work. We provide evidence that ambient light can form peripheral reminders, i.e. they allow office workers to focus on the tasks while creating a gently increasing sense of approaching appointments. (more).

Ambient Reminder: Unobtrusively Reminding Users of Upcoming Tasks with Ambient Light.
Heiko Müller, Anastasia Kazakova, Martin Pielot, Wilko Heuten and Susanne Boll.
INTERACT ’13: 14th IFIP TC13 Conference on Human-Computer Interaction, 2013.

My previous research focussed on navigation & orientation with non-visual user interfaces.