5 A Controlled Study of Interventions for Self-Control on Facebook
5.1 Introduction
Beyond being the world’s largest social network, Facebook is also one of its greatest sources of digital distraction. Indeed, research on ‘Problematic Facebook Use’ has investigated correlations between Facebook use and negative effects on outcomes such as level of academic achievement (Gupta and Irwin 2016) and subjective well-being (Marino et al. 2018a, 2018b). Here, a cross-cutting finding is that negative outcomes are associated with subjective difficulty at exerting self-control over use, as well as specific use patterns including viewing friends’ wide-audience broadcasts rather than receiving targeted communication from strong ties (Burke and Kraut 2016; Marino et al. 2018a). Much of this work has focused on self-control over Facebook use in student populations (Al-Dubai et al. 2013; Khumsri et al. 2015; Koc and Gulyagci 2013), with media multitasking research finding that students often give in to use which provides short-term ‘guilty pleasures’ over important, but aversive academic tasks (Rosen, Mark Carrier, and Cheever 2013; S. Xu, Wang, and David 2016; Meier, Reinecke, and Meltzer 2016).
In the two previous chapters, we took a broad approach to our main research question of how existing digital self-control tools (DSCTs) can help us identify effective design patterns, and analyse a large number of tools in online stores. In the present chapter, we focus in on how this can inform subsequent targeted studies of promising design patterns. Thus, in surveying existing DSCTs, we encountered many examples of tools providing interventions aimed specifically at supporting self-control over use of Facebook (e.g., Newsfeed Eradicator, JDev (2019)). However, no existing studies have evaluated their effectiveness. This chapter presents a controlled study exploring how UI interventions drawn from popular DSCTs on the Chrome Web store affect patterns of use and perceived control over Facebook use. We randomly assigned 58 university students to one of three intervention conditions: goal reminders, newsfeed removed, or white background (control). Their Facebook use was logged for 6 weeks, with interventions applied in the two middle weeks, and we administered biweekly surveys as well as post-study interviews.
We found that both goal reminders and removing the newsfeed helped participants stay on task and avoid distraction. In terms of use patterns, goal reminders led to less scrolling, fewer and shorter visits, and less time on site, whereas removing the newsfeed led to less scrolling, shorter visits, and less content ‘liked.’ However, goal reminders were often experienced as annoying, and removing the newsfeed made some participants fear missing out on information. In post-study interviews, participants suggested a range of design solutions to mitigate self-control struggles on Facebook, including controls for filtering or removing the newsfeed, reminders of time spent and use goals, and removing features that drive engagement.
5.2 Background
5.2.1 Struggles with Facebook use
Whereas many uses of Facebook offer important benefits, such as social support, rapid spread of information, or facilitation of real-world interactions (Ryan et al. 2014), a substantial amount of research has focused on negative aspects (Marino et al. 2018a). For example, studies have reported correlations between patterns of Facebook use and academic achievement (Rouis, Limayem, and Salehi-Sangari 2011; Y. Wang and Mark 2018), low self-esteem, depression and anxiety (Labrague 2014), feelings of isolation and loneliness (Al-Dubai et al. 2013), and general psychological distress (W. Chen and Lee 2013).
As mentioned in Chapter 2, such ‘Problematic Facebook Use’ (PFU) has been studied under various names (including ‘Facebook dependence,’ Wolniczak et al. (2013), and ‘Facebook addiction,’ Andreassen et al. (2012)), but a recent review summarised a common definition across papers as ‘problematic behavior characterized by addictive-like symptoms and/or self-regulation difficulties related to Facebook use leading to negative consequences in personal and social life’ (Marino et al. 2018a).
A large number of studies have in turn correlated measures of PFU with patterns of use and personality traits. Here, researchers often distinguish between use that is more ‘active’ (creating content and communicating with friends) and use that is more ‘passive’ (consuming content created by others without actively engaging), with the former being linked to more positive correlates of subjective well-being (Burke, Marlow, and Lento 2010; Ellison, Steinfield, and Lampe 2007; Gerson, Plagnol, and Corr 2017; Grieve et al. 2013) and the latter to more negative (Krasnova et al. 2013; Verduyn et al. 2015).
Moreover, most studies have found that ‘problematic users’ tend to spend more time on Facebook (Marino et al. 2018a), including a recent study by researchers at Facebook with direct access to server logs: users who experienced their use as problematic (i.e., reported negative impact on sleep, relationships, or work/school performance, plus a lack of control over use) spent more time on the platform, especially at night, as well as more time looking at profiles and less time browsing the newsfeed, and were more likely to deactivate their accounts (J. Cheng, Burke, and Davis 2019).
As we saw in Chapter 2, prevalence estimates of ‘problematic’ Facebook use vary widely depending on the specific tools and thresholds used, from 3.1% in a representative sample of US users (J. Cheng, Burke, and Davis 2019) to 47% in a study of Malaysian university students (Jafarkarimi et al. (2016), see also Bányai et al. (2017); Khumsri et al. (2015); Wolniczak et al. (2013)). The upper bounds of such estimates suggest that, at least at a mild levels, it is very common for people to struggle with using Facebook in accordance with their goals (Guedes et al. 2016). This is supported by the studies of multitasking and media use finding that people very often perceive their use of digital media to be in conflict with other important goals (61.2% of use occurrences in an experience sampling study by Reinecke and Hofmann (2016) (cf. section 2.1.2), and that Facebook in particular is one of the most common sources of media-induced procrastination (Rosen, Mark Carrier, and Cheever 2013; S. Xu, Wang, and David 2016).
5.2.2 Self-control interventions for Facebook
Among DSCTs in online stores (see Chapters 3 and 4), many browser extensions focus on adjusting Facebook in ways intended to help self-control. Such tools may, for example, enable users to remove the newsfeed (JDev 2019) or hide numerical metrics such as like count (Grosser 2019).
No studies have assessed how interventions found in these tools may alleviate self-control struggles on Facebook. However, recent studies have investigated how temporarily deactivating or not logging in to Facebook affect subjective well-being (Allcott et al. 2019; Mosquera et al. 2019; Tromholt 2016; Vanman, Baker, and Tobin 2018). The findings from these studies have largely been in agreement, with Allcott et al. (2019) the largest to date: in a study where 580 participants were randomly assigned to deactivate their accounts for four weeks and compared to 1,081 controls, Facebook deactivation increased offline activities (including socialising with family and friends and watching TV) and subjective well-being, and decreased online activity (including other social media than Facebook). Moreover, Facebook deactivation caused a large and persistent reduction in Facebook use after the experiment.
For many users, however, deactivating or deleting one’s Facebook account present too tall a barrier to action for tackling problematic use. Most users have more targeted non-use goals than “abstinence,” such as reducing time scrolling the newsfeed, but not time posting in a university social group, or reducing time spent on Facebook during final exams, but not during vacations (cf. J. Cheng, Burke, and Davis 2019; Y. Wang and Mark 2018). Some existing research similarly supports positive effects on well-being of targeted non-use, including research on active versus passive social media use (Burke and Kraut 2016; Hiniker et al. 2016; Verduyn et al. 2015). Therefore, investigating interventions found in digital self-control tools for Facebook presents an exciting research opportunity, as they represent less extreme measures than deactivation that may have positive effects.
5.3 Overview of study
On this background, we set out to study how two interventions found in popular browser extensions for scaffolding self-control on Facebook — specifically, adding goal prompts and reminders and removing the newsfeed — affect patterns of use and perceived control on Facebook among university students. We designed a mixed-methods study that attempted to address common limitations in related studies:
Most studies rely on self-reported Facebook use, which complicates interpretation because self-report often correlates poorly with actual use of digital devices (Ellis et al. 2018, 2019; Orben and Przybylski 2019b; Scharkow 2016). Therefore, we combined surveys and interviews with logging of use, to triangulate subjective self-report and objective measurement.
Nearly all studies, apart from deactivation studies, have used cross-sectional designs, making it very difficult to interpret causality (Marino et al. 2018a). Therefore, we randomly assigned participants to intervention groups and compared an initial baseline to a subsequent intervention as well as post-intervention block.
Our choice of interventions is described in the ‘Pre-study’ section below. Based on existing research on self-control struggles in relation to Facebook use, our research questions were as follows:
- RQ1 (Amount of use): How do goal reminders (Cgoal) or removing the newsfeed (Cno-feed) impact time spent and visits made?
- RQ2 (Patterns of use): How do goal reminders (Cgoal) or removing the newsfeed (Cno-feed) impact patterns of use (e.g. passive / active)?
- RQ3 (Control): How do goal reminders (Cgoal) or removing the newsfeed (Cno-feed) impact perceived control?
- RQ4 (Post-intervention effects): Do the effects (RQ1-3) of goal reminders (Cgoal) or removing the newsfeed (Cno-feed) persist after interventions are removed?
- RQ5 (Self-reflection): Do the interventions enable participants to reflect on their struggles with Facebook use in ways that might inform the design of more effective interventions?
Whereas RQ1-4 follow from the background literature reviewed, RQ5 was a generative research question pointing towards new design solutions. We did not envision participants being ‘vessels of truth’ in relation to which interventions would solve their struggles, but were interested in what suggestions the interventions might inspire as design probes.
5.4 Methods
Study materials, anonymised data, and analysis scripts are made available via the Open Science Framework on osf.io/qtg7h.
5.4.1 Pre-study: Review of Facebook self-control tools
In February 2018, we searched for browser extensions for supporting self-control on Facebook on the Chrome Web store and identified 50 such extensions implementing a range of design patterns (see open materials or Appendix A). Most (36/50) let the user remove or alter distracting elements, with more than half (27 out of 50) specifically hiding the newsfeed (e.g., ‘Newsfeed Eradicator,’ JDev (2019), removes it and optionally replaces it with a motivational quote). Others implemented design patterns such as time limits (e.g., setting a daily limit and prompting the user to stop using Facebook, or as in the case of Auto Logout, Labs (2019), force closing it when the time has passed), goal reminders (e.g., Focusbook, Forst (2016), asks the user what they need to do on Facebook and subsequently provides reminders) or providing rewards or punishments (e.g., transferring money out of one’s bank account if use is above a set limit, Timewaste Timer, Prettymind.co (2018)).
To categorise these interventions, we relied on Chapter 3‘s grouping of design patterns in DSCTs into the main types block/removal, self-tracking, goal advancement, and reward/punishment, and mapping to psychological mechanisms in a dual systems framework. To recap, this framework distinguishes between behaviour under non-conscious ’System 1’ control, i.e., when the external environment or internal states trigger habits or instinctive responses; and behaviour that is under conscious ‘System 2’ control, i.e., when goals, intentions, and rules held in working memory trigger behaviour. ‘Self-control’ is the capacity of conscious System 2 control to override System 1 responses when the two are in conflict. For example, one might have a conscious goal to not check one’s phone at the dinner table and need to use self-control to suppress an automatic checking habit to align one’s behaviour with this goal.
Viewed through this lens, removing the newsfeed represents a block/removal design pattern which scaffolds self-control by preventing unwanted System 1 control from being triggered by the newsfeed, and supporting System 2 control by preventing distracting information from crowding out working memory and make one forget one’s intentions for use. We chose this intervention for our first experimental condition, as it was by far the most common among the extensions reviewed.
To compare this to a different design pattern, we selected a goal advancement pattern as a second experimental condition, specifically the one implemented by Focusbook (Forst 2016), which prompts the user to type in their goal when visiting Facebook and then periodically reminds them of this goal. This design pattern scaffolds self-control by keeping the goals the user wishes to achieve present in working memory, thereby enabling System 2 control. We chose this particular option because it had the highest number of users among the extensions reviewed that implemented alternatives to block/removal patterns.
5.4.2 Materials
5.4.2.1 Study conditions
The study conditions are shown in Figure 5.1. In addition to the experimental conditions goal reminder (Cgoal) and no newsfeed (Cno-feed), we included a control condition (Ccontrol). In Ccontrol, we changed the background colour of Facebook from light grey to white, which we did not hypothesise to have any significant effect on behaviour.
We implemented the interventions as Chrome extensions written in JavaScript and CSS:
For Ccontrol, the extension script turned the background colour of Facebook white during the intervention block.
For Cno-feed, the extension script hid the webpage elements containing the newsfeed.
For Cgoal, the extension script was a modified version of Focusbook (the source code for which is available on GitHub, Forst (2016)), where we forced safe-for-work-mode (i.e., avoiding foul language in reminders) and altered prompts that expressed disapproval to neutral reminders (e.g., changing “Fine, just tell me why you needed to open Facebook” to “Tell me why you needed to open Facebook”).
The extension prompted the user to type in why they opened Facebook when they went to the site, and after 1-3 minutes popped up a reminder of what they typed, along with a snooze button.
Until the snooze button was pressed, the banner containing the prompt slowly expanded to take up more and more screenspace.
5.4.2.2 Logging of use
Following recent work (Y. Wang and Mark 2018), we used the open-source browser extension ‘Research tool for Online Social Environments’ (ROSE) (Poller 2017; Poller et al. 2014) to log Facebook use in the Google Chrome browser. We used this extension to record usage metrics (e.g., timestamps when a browser tab with Facebook was brought in and out of focus, number of clicks) and specific interactions (e.g., viewing a profile, liking content). To preserve privacy, the extension gave interactions (e.g., content liked) an anonymous identifier in stored data without storing any identifying information about the actual content engaged with. The ROSE extension was installed on participants’ laptop in addition to the extension for their intervention condition.
5.4.2.3 Surveys/interviews
Opening survey
The opening survey included (i) demographic information, (ii) basic information about their use of Facebook (when they got an account, devices they use to access the site, prior use of self-control tools for Facebook), and (iii) two individual difference measures (susceptibility to types of distraction, adapted from G. Mark, Czerwinski, and Iqbal (2018), and a Big Five personality measure, adapted from Gosling, Rentfrow, and Swann Jr. (2003)).
Repeated surveys
The survey adminstered after each study block included the three following measures:
(i) The Passive and Active Facebook Use Measure (PAUM; Gerson, Plagnol, and Corr (2017)), which assesses frequency of activities on Facebook.
The measure is factored into the usage dimensions ‘active social’ (items including “Posting status updates,” “Chatting on FB chat”), ‘active non-social’ (e.g., “Creating or RSVPing to events,” “Tagging photos”), and ‘passive’ (e.g., “Checking to see what someone is up to,” “Browsing the newsfeed passively (without liking or commenting on anything)”).
(ii) The Multidimensional Facebook Intensity Scale (Orosz, Tóth-Király, and Bőthe 2016), which assesses agreement with statements about Facebook use (e.g., “I feel bad if I don’t check my Facebook daily”) and is factored into the dimensions ‘persistence,’ ‘boredom,’ ‘overuse’ and ‘self-expression.’
(iii) The Single-Item Self-Esteem Scale (Robins, Hendin, and Trzesniewski 2001), a commonly used measure of self-esteem in psychological research.
In addition, the survey after the intervention block included items on whether the changes affected perceived control, or how participants accessed Facebook on laptop vs smartphone.
Interviews
After the study, we conducted semi-structured interviews with all participants. Main topics probed were (i) whether the interventions worked as expected, (ii) how participants experienced the interventions (example question: “When [changes in the participant’s condition], what was that like?”), (iii) what changes participants might wish to make to Facebook to support their intended use (example question: “If you could build any extension you wanted to change the way Facebook appears and works to make it work better for you, what might you want to do?”).
5.4.3 Recruitment
Participants were recruited at colleges at the University of Oxford, using a combination of mailouts, posters, and Facebook posts. Recruitment materials described the study as a study on ‘Facebook distraction,’ investigating ‘which parts of Facebook distract users, and what might be done about it.’ Recruitment targeted non-first year students aged 18-30, who felt they were ‘often distracted by Facebook.’ Participation was compensated with a £20 Amazon gift card.
5.4.4 Procedure
A flowchart of the study procedure is shown in Figure 5.2.
Participants were randomly assigned to conditions. At an initial meeting, participants filled in the opening survey and installed two extensions on their laptop for the Chrome browser: the ROSE extension for logging use and our extension for modifying Facebook according to their assigned condition. Participants were instructed to use Chrome whenever they accessed Facebook on their laptop throughout the study period, and informed that the extensions would ‘anonymously measure how you spend time on the site’ and ‘may change how Facebook appears at some point during the study period.’ The logging period lasted six weeks, grouped into three two-week blocks. By the end of each block, participants were sent a survey link on Friday at 3pm and a reminder two days later. The first block served as a baseline, with no changes made to Facebook. In the second block, interventions were applied from Monday 9am (announced with a pop-up the first time participants visited Facebook) to Monday 9am two weeks later. The third block served as a new baseline measurement (post-intervention) with Facebook returned to normal. By the end of this block, a pop-up thanked participants for taking part and directed them to sign up for an interview and debriefing.
A subset of participants (n = 11) began the tracking period one week later than the others.
5.4.5 Data pre-analysis
Quantitative data
On rare occasions, the ROSE extension did not correctly record entries to or exits from Facebook, which resulted in some instances where the calculated duration of active focus on a tab with Facebook was unrealistically long (more than 24 hours in one case). To handle such instances, we excluded visits longer than one hour when analysing visit durations (144 tab visits out of a total of 120,002).7
Interview transcription and thematic analysis
A collaborator and myself transcribed and conducted thematic analysis of all the interviews and free-text survey responses. The recordings were iteratively transcribed and analysed using an open-coding approach (cf. Tran et al. 2019). We reviewed transcripts and identified emerging codes individually, and regularly discussed emerging codes.
Thematic analysis was conducted in the Dedoose software; quantitative analyses were conducted in R.
5.5 Results
58 students (21 male) took part. For 8 participants, the intervention failed (on some Windows laptops, security settings prompted participants to turn the extensions off), and 1 participant deactivated his Facebook account during the study. Survey and logging data from these participants, as well as their interview statements about the interventions, were excluded from analysis. In addition, 2 participants deleted the ROSE extension before the debriefing - and with it their logged use - and for 1 participant the interview recording device failed. This left us with survey data from 49 participants (15 goal, 15 no feed, 19 control), logging data from 46 participants (15 goal, 14 no feed, 17 control), and interview data from 57 participants (20 goal, 19 no feed, 19 control) for analysis. Median interview length was 23m 51s (sd = 5m 5s).
In the following, we first report general characteristics of participants and their Facebook use, as well as introductory notes on how interventions were used and perceived. Afterwards, we report results grouped by research question.
5.5.1 Participant characteristics
Participants’ median age was 22.5 (min = 19, max = 38) years. 90% had had a Facebook account for six years or longer, and the median number of Facebook friends was 900 (min = 200, max = 2200). All participants routinely used Facebook on their laptop. 96% also used it on their smartphone; most (78%) used the Facebook and Messenger apps, 8% used their smartphone’s web browser (instead of the Facebook app) plus the Messenger app, 6% used only the Messenger app, and 2% (1 participant) used only their smartphone’s web browser.
Most participants (71%) had never used digital self-control tools for Facebook. Among those who had, the most commonly used tools blocked access (7 participants) or removed the newsfeed (3 participants). 3 participants currently used such tools; one used Newsfeed Eradicator (which removes the newsfeed), another used Self-control (which blocks social media), and the third used an ad blocker (which we did not consider a self-control tool).
5.5.2 Overall Facebook use
Across all participants and the entire study period, the median number of daily tab visits to Facebook was 23 (min = 5, max = 138). The median break length between visits to Facebook was 69.5 seconds (min = 11, max = 445). The median of participants’ average amount of daily time spent was approximately 21 minutes (min = 4m, max = 2h 56m).
Often, a number of successive tab visits were logged within a short span of time (e.g., if participants switched back and forth between active application windows). Following J. Cheng, Burke, and Davis (2019), we calculated the number of ‘sessions’ as the number of times where the break between two visits to Facebook was longer than 60 seconds. The median number of daily sessions on Facebook was 11 (min = 1, max = 101).
5.5.3 Intervention use and perceptions
The Cgoal extension did not record what participants typed when prompted for their goal, as we wanted to study effects of goal reminders without participants adapting or self-censoring from knowing responses might be read by the researchers. However, we asked in the interviews how they had used it. Most said they wrote short, descriptive, but generic notes for what they did (“I would type shorthand in for what I was about to do, so most of the time I would say something like ‘reply to messages’ or just ‘messages’ or ‘post something on a group’ or something like that,” P4). Some also said they occasionally wrote meaningless or ‘unsavory’ things when they found the goal prompt annoying or disruptive (“I think sometimes I tried to type in, like, not really proper words and it said, ‘give me a proper answer’ and I was like ‘dammit!’,” P27). In Cno-feed, one participant said the newsfeed occasionally flashed on screen very briefly before being hidden by our script (“sometimes i saw like a millisecond of something and I was like ‘oh that’s interesting, I would like to see that’ but then it wasn’t there,” P56).
In Ccontrol, a couple of participants said the white background made content stand out less on their screen (“white background definitely makes it harder to… I don’t think it’s easier to read…,” P1). Others, however, found it aesthetically pleasing (“I just liked Facebook more… it felt more… I mean it felt more Nordic, it wasn’t grey and boring, it was white and nice…,” P30) and wanted it to persist (“is there a way that I can keep the background white?” P15).
5.5.4 RQ1 (Amount of use): How do goal reminders or removing the newsfeed impact time spent and visits made?
The logging data and qualitative data suggested that Cgoal led to less time spent and fewer and shorter visits, whereas Cno-feed led to shorter visits (Figure 5.3):
Usage logging showed that in Cgoal, average daily time on Facebook was significantly lower during the intervention block than in the baseline (Wilcoxon signed rank test, p = 0.01, median daily time in baseline: 27m 14s, median in intervention: 15m 5s); number of daily visits declined (Wilcoxon signed rank test, p = 0.01, median number of visits in baseline = 29.4, median in intervention = 10.6); and there was a trend towards shorter visits (t(14) = 1.96, p = 0.07; mean tab visit duration in baseline = 1m 25s, mean in intervention = 1m 15s). In Cno-feed, only visit length declined significantly (t(13) = 2.81, p = 0.015; mean visit length in baseline = 1m 12s, mean in intervention = 56s).
Participants’ reports in the surveys and interviews agreed with the logging data:
In Cgoal, two common themes were that the intervention reduced amount of time on Facebook on laptop (“yeah I think I used it less and when I was using it I wasn’t using it for very long, like a minute maybe,” P45interview8; “definitely used it a bit less,” P21interview) and that reduced use was partly caused by the intervention being annoying/stressful (“This programme made me annoyed thus I would spent [sic] less time on Facebook,” P32survey; “The changes stressed me to get done with my task and then close facebook,” P40survey).
In Cno-feed, participants had mixed opinions on whether or not it reduced amount of use. Some felt it reduced their use (“limited overall usage,” P28survey, “I think I used it less erm for shorter periods of time,” P55interview) but others felt it only changed their newsfeed use without affecting amount per se (“The lack of newsfeed is welcome … Facebook usage on my laptop has not changed/barely changed,” P27survey; “I spent a lot of time actually on facebook but messaging other people and not just looking through my wall,” P54interview).
5.5.5 RQ2 (Patterns of use): How do goal reminders or removing the newsfeed impact patterns of use?
The logging, survey, and interview data suggested that both Cgoal and Cno-feed affected patterns of use: Cgoal selectively reduced passive scrolling of the newsfeed, whereas Cno-feed (as expected) reduced all behaviour related to the newsfeed (Figure 5.4).
Thus, usage logging showed that average daily scrolling declined by 42% in Cgoal (comparing intervention to baseline, t(14) = 2.39, p = 0.03), and by 73% in Cno-feed (t(13) = 4.15, p = 0.001). Moreover, in Cno-feed, the number of times content was liked declined (Wilcoxon signed rank test, p = 0.002, median number of likes during baseline = 16, median during intervention = 7).
In the surveys, scores on the Passive and Active Facebook Use Measure dimensions showed that participants in Cno-feed had substantially lower scores on ‘passive’ use in the intervention than in the baseline block (t(13)=4.79, p = 0.003). We explored effects on more granular elements of Facebook use by comparing baseline and intervention scores separately for each item of the PAUM.9 Two items showed significant variation with condition: “Browsing the newsfeed passively (without liking or commenting on anything)” and “Browsing the newsfeed actively (liking and commenting on posts, pictures and updates)”: In Cgoal, participants reported less passive, but not active, browsing of the newsfeed during the intervention block compared to baseline (Passive browsing: p = 0.029, Active browsing: p = 1, Wilcoxon signed rank test). In Cno-feed, participants reported less active as well as less passive newsfeed browsing (Passive browsing: p < 0.001, Active browsing: p = 0.013, Wilcoxon signed rank test). Moreover, participants in Cno-feed showed a trend towards lower scores on “Commenting (on statuses, wall posts, pictures, etc)” (p = 0.086, Wilcoxon signed rank test).
The quantitative results were supported by the qualitative data:
For participants in both experimental conditions, a recurrent theme was that the interventions caused decreased browsing of the newsfeed (“I did feel very aware when scrolling down my newsfeed, and cut it down,” P19goal_survey; “definitely meant I spent less time scrolling on newsfeed on my laptop,” P55no-feed_survey), and increased use of Facebook for other, more deliberate purposes (“a big facebook post or whatever not just passively…scrolling,” P41goal_interview; “messaging other people and not just looking through my wall,” P54no-feed_interview).
In Cgoal, participants said the effects were driven by the intervention making them search for reasons to justify being on the site (“Being asked why I was opening Facebook was really helpful as it made me question why,” P41goal_survey; “less likely to aimlessly browse, as I couldn’t justify it,” P45goal_survey). In Cno-feed, participants said the lack of a newsfeed made them seek out alternative options that were often more productive and deliberate (“procrastination was more productive in that I was uhm seeking things out to read or to do that were more intentional, I suppose, and less kind of mindless which I guess the newsfeed is,” P12no-feed_interview). (Changed patterns of use related to perceived control are reported below.)
5.5.6 RQ3 (Control): How do goal reminders or removing the newsfeed impact perceived control?
The qualitative evidence suggested that Cgoal and Cno-feed supported control in the sense of helping participants avoid unintended use and staying on task, but at the cost of being annoying/frustrating (Cgoal) or leading to fear of missing out (Cno-feed).
Thus, in both Cgoal and Cno-feed, it was a strong theme in the surveys and interviews that the interventions helped participants stay on their intended task during use (“used it less for stuff that I wasn’t intending when I opened it,” P4goal_interview; “I’ll kind of forget that I’m doing work and start scrolling so it was useful to not be able to do that,” P47no-feed_interview). A subtheme was that this included being easier to disengage from use (“it’s good to get this reminder of ‘hey you can get off this thing’,” P31goal_interview; “it was easier just to log out, just check what I had to and then leave facebook,” P54no-feed_interview).
In Cgoal, participants said the reason the intervention helped them stay on task was that it helped them snap out of automatic use, that is, stop themselves when they engaged in unintended behaviour (“[the reminder] sort of snaps you out of that trance, you know what I mean?” P21interview). In Cno-feed, participants said it was because it stopped unintended behaviours from being triggered in the first place (“there is nothing here [referring to the newsfeed], like ‘what did I want?’ you know, so then I went and contacted the person or looked at the specific thing that I wanted, not what I saw and kinda wanted at the moment,” P56interview).
The downsides were that Cgoal was frequently annoying or frustrating, especially because it was not sensitive to context (“I use facebook just to message people and I found this extremely annoying because I need to tell someone something and then this thing comes up and I’d just get annoyed…” P32interview), and that Cno-feed led to fear of missing out (“missing out on a lot because actually a lot of the ways I interact with people on facebook is things I see on the newsfeed,” P12interview).
Perhaps reflecting this ambiguity, participants were more or less evenly split when asked directly in a survey item following the intervention block whether they felt the changes made to Facebook made them feel less or more in control of their use (Figure 5.5): When asked whether the interventions changed how they used Facebook on smartphone vs. on laptop, 86% of participants in Cgoal and 57% in Cno-feed answered ‘Yes’ (compared to 5% in Ccontrol).
Unpacking this in the qualitative data, participants in both experimental conditions expressed that cross-device access helped them manage the interventions’ downsides, while still enjoying the positive effects (“I could reap the benefits of the newsfeed but without being sucked into it on two platforms,” P28no-feed_survey; “if I was scrolling through the newsfeed or checking events, then it wouldn’t be annoying because I shouldn’t be doing that on my laptop while I’m working, and if it was something like sending messages about work, contacting friends and asking for help then I could use my phone,” P40no-feed_interview), and so they sometimes used their smartphone for activities on Facebook the interventions interfered with, but as a deliberate choice (“the time I did spend on my phone was more, like, focused because I was actually looking for things I missed out on on my laptop,” P55no-feed_interview, “you’re working on your laptop, uhm, and then it’s very easy to just click new tab, but having to get your phone out…,” P19goal_interview).
Finally, when exploring survey responses in the Multidimensional Facebook Intensity Scale, the only of its four dimensions that showed significant differences between the baseline and intervention blocks was overuse: Scores on this measure trended towards a decrease during the intervention in all conditions (Ccontrol: t(14) = 1.7, p = 0.037, Cno-feed: t(13) = 1.99, p = 0.07, Cgoal: t(14) = 1.7, p = 0.1), perhaps suggesting that simply taking part in the study made participants reflect on use. (Note that the benefits of staying on task and engaging less in unintended use were not expressed by any participants in the control condition.)
5.5.7 RQ4 (Post-intervention effects): Do the effects (RQ1-3) of goal reminders or removing the newsfeed persist after interventions are removed?
Comparing post-intervention to baseline, Cgoal and Cno-feed were associated with some persisting effects, with participants in Cgoal engaging in fewer daily visits and some feeling that the intervention helped build a habit of more intentional use, and participants in Cno-feed engaging in less passive newsfeed browsing.
Thus, in terms of amount of use, participants in Cgoal made fewer daily visits post-intervention compared to baseline (median number of daily visits in first baseline = 29.4, median in post-intervention block = 10, Wilcoxon signed rank test, p = 0.003).
In terms of patterns of use, participants in Cno-feed reported less passive browsing of the newsfeed post-intervention compared to baseline (p = 0.007, Wilcoxon signed rank test).
In the interviews, some Cno-feed participants expressed feeling less attracted by the newsfeed when it returned (“I found myself less interested in the newsfeed,” P10interview).
In terms of perceived control, some participants in Cgoal said the intervention helped them build a persisting habit of asking themselves what their intention of use was when visiting the site (“from this week there is a habit being built… asking myself why I’m opening Facebook and that habit’s perpetuated more or less to this week,” P34interview, “I’m still aware every time I open Facebook, I’m just a bit more aware every time… it’s not the reflex anymore now that I’ve had that experience where I have to write everything down,” P1interview).
5.5.8 RQ5 (Self-reflection): Do the interventions enable participants to reflect on their struggles in ways that might inform the design of more effective interventions?
In the interviews, nearly all participants expressed feeling conflicted about Facebook, in that they found it too useful or engrained in their lives to do without, but also an ongoing source of distraction and self-control struggles. They readily suggested a range of design solutions to mitigate self-control struggles. The extent to which interventions were perceived as freely chosen was important to how it was received, and participants did not trust Facebook to provide solutions.
5.5.8.1 Struggles with Facebook use
Too useful to do without, but source of distraction and self-control struggles
On the one hand, Facebook provided functionality participants could not - or would not - do without, particularly messaging, events, groups, and pages. On the other, Facebook was frequently distracting and caused them to waste time and feel frustrated (“I just want…to hack myself to have the self-control to, like, not get distracted… I literally just use it as distraction,” P42no-feed). In particular, participants struggled to use Facebook in line with their intentions. Main aspects included (i) going to the site to do one thing, but then forgetting this goal (“there is one specific trigger that I need to open facebook, but because when I open the page immediately there is tons of information there, like erm notifications, and you scroll down endless streaming… so very easily I could be distracted,” P34goal), (ii) internal conflict between short-term gratification and longer-term goals (“might find them [videos] funny in the short term but when I think about it in the bigger picture it is a complete waste of time,” P48control), and (iii) using Facebook purely out of habit. In relation to the latter, emotional states, especially boredom, were mentioned as triggers of habitual use (“if I’m in that erm not very motivated state… I’ll literally just find myself opening it, without even thinking that I’m doing it,” P17control).
5.5.8.2 Specific suggestions for design solutions
Four themes emerged in relation to specific design suggestions for mitigating these struggles:
Control over the newsfeed
More than half of participants explicitly said the newsfeed did not give them what they wanted and desired easy ways to filter it, limit it, or turn it off. Some had tried customising their newsfeeds, but found Facebook’s means of doing so tedious and ineffective (“I browse through shit that I don’t want to see and I keep on clicking on ‘I don’t like this,’ ‘this is not interesting’ and of course it keeps on adding new stuff so that doesn’t solve the problem basically,” P51control). Solution suggestions included simple ways to filter the newsfeed (“a slider to modify the amount you see people who are on your newsfeed at different percentiles,” P49goal, “two different ones, like you could have a ‘friends’ or like ‘photos’ or something,” P17control), reducing the amount of information (“maybe it should be limited to like ten posts and you wouldn’t get another ten until the next hour,” P45goal, “if it was instead like blank and then you opt-in to who you actually wanna see on your newsfeed as opposed to opt-out,” P44no-feed), or being able to remove it altogether.
Raise awareness of time spent or usage goals
Participants often lost track of time spent, or of their usage goals, and wanted reminders that raised awareness. These should be easily accessible (“you wouldn’t want it to be buried in settings, something that was actively shown to you I think that would be useful,” P52control), and let users judge whether their use was appropriate (“if I saw like ‘you’ve spent 2 minutes today,’ like ‘great, i’ve got loads of time that i can waste tomorrow because i’ve been good today’,” P6goal). Participants in Cgoal said the timing and intrusiveness should be calibrated differently to the reminders they experienced in the study (“less in-your-face… so maybe more, longer intervals and not the expanding thing… if I could change it to longer intervals and maybe a bit less invasive then I think it would actually help,” P4goal).
Remove ‘addictive’ features
Participants wished to remove or modify features driving them to use the site. Specific features mentioned included notifications (“get rid of notifications… if I didn’t have things popping up every 30 minutes like ‘this has happened’ I don’t think I would think about Facebook’, P6goal), viral videos, and games (”things like game suggestions and like all that sort of stuff I would definitely get rid of cause… I don’t want to play games … ‘stop bugging me’“, P55no-feed). One interesting suggestion was to be able to display content as text-only (”limit it to like text-only posts when you’re working so that you’re not bothered by videos and algorithms and photos", P45goal).
Flexible blocking to meet individual definitions of distraction
Participants suggested blocking solutions that could adapt to the type — or timing – of use they found distracting. Thus, some said blocking access altogether was too inflexible to be useful (“there are useful uses of Facebook that aren’t just waste of time…a blanket, like, ‘don’t do anything on Facebook’… it’s not practical for those people who have to use Facebook,” P41goal). Suggestions for more useful solutions included being able to block or allow only specific functionality within Facebook, block access only during specific times (“sync it with a timetable, like lectures or something,” P45goal), or even automatically detect if activity is engaged with as a distraction.
5.5.8.3 Generic solution needs
People differ in what they seek on Facebook and the design solutions they prefer
Some participants wanted to block or remove distractions, whereas others preferred less intrusive solutions, such as goal reminders. Similarly, even though most participants were dissatisfied with the newsfeed, some wanted it to prioritise close ties, whereas others wanted it to prioritise pages they follow (“I wouldn’t want to see anyone’s posts, I would only want to see posts by things I wanted to follow, whether that’s petitions or science papers,” P20no-feed).
Interventions can ‘backfire’ if overly intrusive and/or not freely chosen
Participants felt interventions could make people rebel against them if too intrusive and/or if they did not feel in charge. In terms of intrusiveness, some felt blocking tools could backfire for this reason (“I feel like most people in their nature, if you have something restrictive… then you kinda want to rebel against it,” P56no-feed). In terms of feeling in control, some participants suggested this could change their reaction to the very same intervention. For example, a participant in Cgoal felt the goal reminders were too intrusive and led to resistance (“I got very used to clicking out of it and like, I’m just gonna stay on just out of spite,” P19goal), but thought she would react differently if she controlled the reminders herself (“it would be a bit different if it was me, if I could actually write the messages… I think that’d help me, and knowing it was me, so it wasn’t anyone else”).
Scepticism about design solutions coming from Facebook
Participants did not trust Facebook to provide effective solutions for mitigating self-control struggles, because this was seen as going against their business interests (“you wonder how much they’d try to just give people the information that doesn’t really reflect badly on them,” P36control; “Facebook’s interest is for people to spend more time on it ’cause then then they’ll get more ad revenue, so…,” P45goal).
5.6 Discussion
Figure 5.6 summarises findings from RQ1-4:
Both Cgoal and Cno-feed reduced unintended Facebook use (RQ3), with the downside that Cgoal was often experienced as annoying and Cno-feed made some fear missing out on information (cf. “FOMO,” Przybylski et al. (2013)).
On amount of use (RQ1), Cgoal reduced daily time, number of visits, and visit length, whereas Cno-feed reduced visit length.
On patterns of use (RQ2), Cgoal and Cno-feed reduced scrolling and passive newsfeed browsing, and Cno-feed in addition reduced active newsfeed browsing and amount of content ‘liked.’
On post-intervention effects (RQ4), Cgoal was associated with fewer visits and Cno-feed with less passive newsfeed browsing.
In terms of reflections on struggles and solutions (RQ5), participants felt conflicted because Facebook was a source of distraction and self-control struggles but also vital to staying connected, i.e., too useful to avoid.
They suggested specific design solutions related to control over the newsfeed, reminders of time spent and usage goals, removing ‘addictive’ features, and flexible blocking.
Their preferred solutions (as well as the information sought on Facebook) differed, however, and they felt that solutions might ‘backfire’ if overly intrusive and/or not freely chosen.
We now discuss design implications as well as some of the limitations and future work.
5.6.1 Evaluating the experimental interventions
Focusing specifically on the ability to use Facebook in line with one’s conscious intentions — which is at the very core of self-control (Angela L. Duckworth and Steinberg 2015) — which of the two experimental interventions is more effective? Goal reminders and removing the newsfeed represent contrasting, and potentially complementary design patterns. In our study, both interventions had a positive effect on perceived control and a significant effect on behaviour, with Cgoal helping people ‘snap out’ of unintended behaviour and Cno-feed preventing unintended behaviours from being triggered. While these results suggest that both interventions have potential, as an exploratory study with a restricted sample, further research with larger samples will be needed to draw definitive conclusions about the robustness, effect sizes, and individual differences. However, contextualising our study within related research provides some predictions:
One possible approach is to, once again, apply Chapter 3‘s dual systems framework:
From this perspective, goal reminders are a ’System 2’ intervention which supports conscious self-control by bringing the goals into working memory that the user wishes to control her behaviour in relation to.
Removing the newsfeed is both a ’System 1’ and ’System 2’ intervention which prevents unwanted automatic responses from being triggered by the newsfeed, and supports conscious self-control by preventing attention-grabbing information from crowding out working memory and making the user forget her goal.
As mentioned in Chapter 2, a recent, comprehensive review of digital behaviour change interventions found that providing information about the consequences of behaviour (a System 2 intervention) tends to be unsuccessful, despite being the most common technique.
The authors argued that targeting unconscious habit formation (System 1) should be the focus for interventions that aim at long-term efficacy (Pinder et al. 2018).
Similarly, psychological research has found that people who are better at self-control tend to develop habits that make their intended behaviour more reliant on automatic processes (System 1) and less on conscious in-the-moment self-control (System 2), and/or reduce their exposure to ’temptations’ in the first place (Galla and Duckworth 2015; Angela L. Duckworth and Steinberg 2015; Angela L. Duckworth et al. 2016).
As outlined in Chapter 3, this may be because effective System 2 control depends not only on remembering longer-term goals, but also on the motivation to exert control relative to those goals, which can fluctuate with emotional state (cf. participants who said they were more likely to go on Facebook when bored or unmotivated Berkman et al. (2017); Inzlicht, Schmeichel, and Macrae (2014); M. K. Lee, Kiesler, and Forlizzi (2011)).
I therefore expect removing the newsfeed to be more generally effective than goal reminders, because it reduces the amount of potentially distracting information and thus the need for in-the-moment conscious control. In our study, qualitative data did suggest that Cgoal fostered a habit of asking oneself about one’s purpose when visiting Facebook. However, given the above, the likelihood of effective control through a habit of goal awareness should depend on what content is available on Facebook and how that content is perceived: the more ‘engaging’ the content, the greater the risk that goal awareness will not by itself provide sufficient control motivation (Berkman et al. 2017; Tice, Bratslavsky, and Baumeister 2001; M. K. Lee, Kiesler, and Forlizzi 2011). Goal reminders should therefore exhibit greater variation in effectiveness, and may be less useful for individuals whose newsfeeds contain more attention-grabbing content and/or who struggle more with inhibiting distractions in general. This would align with recent findings that those who find Facebook more valuable are also (somewhat paradoxically) more likely to find their use problematic (J. Cheng, Burke, and Davis 2019). Similarly, as mentioned in Chapter 2, blocking of online distractions has been found to be more effective for individual who are more susceptible to social media distractions (G. Mark, Czerwinski, and Iqbal 2018; cf. M. K. Lee, Kiesler, and Forlizzi 2011; Miri et al. 2018).
Taken together, removing the newsfeed is likely to be more consistently effective than are goal reminders for helping people align use of Facebook with their conscious goals. However, for individuals who are less impulsive — or who are simply more concerned with missing out on information than occasionally failing to stay on task — goal reminders may be as appealing. Additionally, these design patterns are not mutually exclusive and can be combined in effective interventions, as is already the case in many digital self-control tools (e.g., Todobook, Yummy Apps (2019), which removes Facebook’s newsfeed and replaces it with a todo-list reminding the user of her goals).
5.6.2 Designing future interventions
Broadly, participants’ suggested design solutions related to either altering the information landscape (by filtering the newsfeed, removing features driving engagement, or blocking distracting elements) or raising awareness to help navigation within this landscape (by adding reminders of time spent or usage goals). These suggestions could be compared to the many existing interventions in online stores; analysed using a dual systems or other framework; and design patterns more likely to be effective implemented and evaluated. Here, we discuss implications of the cross-cutting theme that interventions should be experienced as freely chosen and not overly intrusive to avoid ‘backfiring’ and motivate people to rebel against an intervention instead of being helped by it (cf. M. K. Lee, Kiesler, and Forlizzi 2011).
Given that participants preferred different interventions — with some wanting restrictive blocking tools — it is not a solution to only consider e.g. non-intrusive addition of user controls (Harambam et al. 2019). Rather, designers should keep in mind that the effectiveness of the exact same restriction or intrusion may depend on whether it is perceived by the user as self-imposed or externally imposed (Brook 2011; Swim and Bloodhart 2013; Bryan, Karlan, and Nelson 2010). The implication is that interventions should be carefully framed as being supportive of the user’s personal goals (cf. Swim and Bloodhart 2013; Bandura 1982). For example, blocking tools may wish to remind the user why their past self decided to impose restrictions on their present self (Angela L. Duckworth, Milkman, and Laibson 2018). Current examples ‘in the wild’ include browser extensions for website blocking that display motivational quotes or task reminders when users navigate to distracting sites (cf. Chapter 3).
One exciting avenue for future tools is systems that can learn the user’s personal definition of distraction and in what contexts to, e.g., automatically impose or not impose limits. This was suggested by one of our participants, and is being explored in some HCI research, e.g., HabitLab, which rotates between interventions to discover what best helps a user limit time on specific websites (Kovacs, Wu, and Bernstein 2018). A useful such system in the context of Facebook would not simply limit time, but rather assist the user in carrying out their goals, for example by dynamically blocking elements such as the newsfeed if the user’s current goal is to create an event. Such a hypothetical system could be highly useful, but it would be crucial to its success that its interventions were perceived by the user as being in her own interest. In addition, it would need to really understand the user to be functional (Lyngs et al. 2018), creating a possible trade-off between privacy and the ‘fit’ of the intervention. Facebook itself, with its deep knowledge of user behaviour, might be in the best position to take this approach, but we note that participants in our study were deeply sceptical about Facebook’s motivations and did not expect design solutions coming from Facebook to be ‘on their side’ (cf. Creswick et al. 2019; Perez Vallejos et al. 2017).
5.6.3 Limitations
This study has a number of limitations.
Confounding variables
A possible criticism is that less scrolling and shorter visits from removing the newsfeed is simply because there was nothing to scroll. We note that removing the newsfeed did not make scrolling impossible — it remained relevant on all other pages than the home screen — and thus scrolling remained a useful measure. Moreover, reduced time is often an explicit goal for users, and so time spent in the face of reduced content is a relevant outcome.
Lack of cross-device tracking
We investigated Facebook use on laptop only. We did assess participants’ sense of how interventions affected cross-device use (as mentioned in Chapter 2, recent research has found that productivity interventions does not just displace procrastination from one medium to another, Kovacs et al. (2019)), but adding objective measurement of ‘spillover’ effects would be highly useful in future work (Lascau et al. 2019).
Retrospective self-report
In the surveys and interviews, participants retrospectively reported their experience, which is subject to recall biases (Kahneman and Riis (2005); Redelmeier and Kahneman (1996)). As self-control often involves one’s past self setting goals for one’s future self (e.g., in blocking tools), retrospective reflection is highly informative (Lyngs et al. 2018), but it would be interesting to include experience sampling in future work (cf. section 7.2.2).
Granular interventions and usage measures
Standard measures of Facebook use were not optimal for assessing granular interventions on laptop only: most measures consider global use and factor into broad dimensions. For example, we found the overall dimensions of the Passive and Active Facebook Use Measure too broad to capture the behavioural changes our interventions introduced. We flag this as a consideration for future study designs.
Sampling
Our participants were all students at the University of Oxford and our sample size limited to allow for interviews to be conducted with all participants. Future research with larger samples is needed to assess the replicability and generalisability of our results (Cockburn, Gutwin, and Dix 2018), how individual differences may predict the design patterns’ relative usefulness, and how implementation details might minimise their downsides.
5.7 Conclusion
Imagining what success for digital self-control on Facebook looks like is not an academic exercise, but a practical and urgent concern as evidenced by the recent hearing on ‘Persuasive Technology’ in the US senate (“Optimizing for Engagement: Understanding the Use of Persuasive Technology on Internet Platforms” 2019), and a UK All Party Parliamentary Group’s call for a ‘duty of care’ to be established on social media companies (All Party Parliamentary Group on Social Media and Young People’s Mental Health and Wellbeing 2019). We hope the work presented in this chapter illustrates how assessments of possible interventions, with open and transparent research methods, may help provide the evidence base needed to assist regulators in moving towards a benevolent future (Grimpe, Hartswood, and Jirotka 2014).
In relation to this thesis’ main research question (How can existing digital self-control tools help us identify effective design patterns for supporting self-control over digital device use?), this chapter showed how tools in online stores can be used to drive focused studies of specific, promising design patterns, in this case for self-control on Facebook. In the next chapter, we zoom back out to explore how a broader sample of existing tools might help us understand personal digital self-control struggles among university students, as well as their needs and preferences for potential solutions.