App Store Studies : How to Ask for Consent?

App Stores, such as Apple’s App Store or Google Play, provide researchers the opportunity to conduct experiments with a large number of participants. If we collect data during these experiments, it may be necessary to ask for the users’ consent beforehand. The way we ask for the users’ consent can be crucial, because nowadays people are very sensitive to data collection and potential privacy violations.

We conducted a study suggesting that a simple “Yes-No” form is the best choice for researchers.

Tested Consent Forms

We (most of the credit goes to Niels Henze for conducting the study) tested four different approaches to ask for the consent to collect non-personal data. All consent forms contain the following text:
By playing this game you participate in a study that investigates the touch performance on mobile phones. While you play we measure how you touch be we DON’T transmit personalized data. By playing you actively contribute to my PhD thesis.

Checkbox Unchecked

The first tested consent form showed an unchecked check next to a text reading “Send anonymous feedback”. In order to participate in the study a user had to tick the checkbox and then press the “Okay” button.

Checkbox Checked

The second consent form is the same as the previous one, except that the checkbox is pre-checked. To participate in the study the user has to merely click the “Okay” button.

Yes/No Button

The third consent form features two buttons are provided reading “Okay” and “Nope”. To participate the user has to click “Okay”. Clicking “Nope” will end the app immediately.

Okay Button

The foorth consent form only contains a single “Okay” Button. By clicking “Okay” the user participates in the study. To avoid participation, the user has to end the app through the phone’s “home” or “return” buttons.


These consent forms were integrated into a game called Poke the Rabbit! by Niels Henze. At first start, the application randomly selected one of the four consent forms. If the use accepted to participate in the study, the app transmitted the type of the consent form to a server.


We collected data from 3,934 installations. The diagram below shows the conversion rate. The conversion rate was estimated by dividing the number of participants per form by 983,5 (we assume perfect randomisation, i.e. each consent form was presented in 25% of the installations).

Conversion rate per consent form. The x-axis shows the type of consent form. The y-axis shows the estimated fraction of users that participated in the study after download.

We were surprised about the high conversion rate. Only the consent form with the unchecked checkbox yielded in a too low conversion rate.

Conclusions – use Yes/No Buttons

We suggest using the consent form with Yes-No buttons. The consent form with the checked checkbox may considered unethical, since the user may not have read the text and was not forced to consider unchecking the checkbox. The consent form with the “Okay” button may be considered unethical, too, because users may not be aware that they can avoid data collection by using the phone’s hardware buttons. The “Yes-No” form, in contrast, forces users to think about their choice and offer a clear way to avoid participating in the study.

Yes-No buttons are ethically safe and resulted in the second highest conversion rate.

Would you suggest otherwise? We are not at all saying that this is definite! Please share your opinion (comments or mail)!

More Information

This work has been published in the position paper App Stores – How to Ask Users for their Consent? The paper was presented at the ETHICS, LOGS and VIDEOTAPE Ethics in Large Scale Trials & User Generated Content Workshop. It took place at CHI ’11: ACM CHI Conference on Human Factors in Computing Systems, which was held in May 2011 in Vancouver, Canada.


The authors are grateful to the European Commission, which has co-funded the IP HaptiMap (FP7-ICT-224675) and the NoE INTERMEDIA (FP6-IST-038419).


Share this:

Will they use it? Will it be useful? In-Situ Evaluation of a Tactile Car Finder.

When we develop new technology, we want to know if it will have the potential to be successful in the real world.

This is not trivial! People may sincerely enjoy our technology when we expose them to it in a lab- or a field study. They may perform better than with previous solutions at the tasks that we ask them to fulfill as part of the study.

However, once they leave our lab they never again encounter the need to use it in their daily routines. Or, the utility we prove in our studies may not be evident in the contexts where this technology is actually deployed.

In our work, we made use of Google Play to answer these questions in a novel way. We wanted to study if a haptic feedback can make people less distracted from the environment, when they use their phone for pedestrian navigation in daily life. We developed a car finder application for Android phones with a simple haptic interface: whenever the user points into the direction of the car, the phone vibrates.

The data provides evidence that about half of the users use the vibration feedback. When vibration feedback is enabled, users turn off the display and stow away the device more often. They also look less at the display. Hence, when using vibration feedback, users are less distracted.

Our work shows that app distribution channels, such as Google Play or the iOS Store, can serve as a cheap way of bringing a user study into the daily life of people instead of bringing people into the lab. Compared to the results of a lab study, these findings have high external validity, i.e. we can be sure that our findings can be generalized to a large number of users and usage situations.

This work will be presented at NordiCHI ’12: The 7th Nordic Conference on Human-Computer Interaction, which takes place in Copenhagen in October 2012. The paper is available here (pdf).

Thanks to for summarising this work so nicely in their article Buzzing app helps smartphone dudes locate their car.

Share this:

To blog or not to blog ongoing research?

Research ideas are old when they are finally presented. The time from the submitting a paper to its presentation at a conference can take more than half a year. When submitting a paper the authors have already spent a lot of time working on the idea and writing it down. When the paper is accepted and finally available to the public, I have often already lost the initial excitement for the research idea.

But can researchers blog about ongoing research?

The excitement would still present at the time of writing. Other researchers’ comments could be incorporated before all the work has been done and documented. You could sort out ideas early that will not make it through the reviewing process anyways.

Unfortunately we are living in a world where novel and original ideas are one of the main assets of a researcher. The originality of a submission is usually one of the main review criteria’s. But if a research idea has been published in a blog others may already have picked it up so it is not original anymore. The submission containing the idea may get rejected and not appear as a publication in the researchers CV.

So, as long as researchers are judged by their publications novel ideas will probably remain in the vault until published. I wonder how a world would look like, where this is not necessary anymore.

Share this: