Introduction
The fifth (and latest) edition of the APA’s Diagnostic and statistical manual of mental disorders (DSM-5) - globally used as a manual for diagnosis of mental disorders - describe anxiety disorders as "...disorders that share features of excessive fear and anxiety and related behavioural disturbances.". Broadly distinguishing the two, whereas fear is a response to a perceived immediate threat, anxiety is a response to a future threat. Physical and mental symptoms include surges of arousal relating to the fight or flight response (fear), and muscle tension (anxiety).
Panic attacks are a specific type of anxiety disorder and manifest as a sudden surge of fear or distress. These attacks usually build up and reach a peak within minutes after onset, during which symptoms such as accelerated heart rate; sweating; trembling; feelings of choking; feeling dizzy; nausea; chills or heat sensations; may be experienced. Patients often report a fear of losing control or ’going crazy’, and fears of dying. About 2 to 3 percent of US and EU adults and adolescents are estimated to suffer from panic disorder (PD), afflicting females more than males (at a ratio of approximately 2 to 1). The median age at which patients describe symptoms of panic disorder is 20 to 24 years.
It is obvious that patients suffering from PD experience extreme discomfort in their daily lives. Not only is the attack itself very stressful; the worry about future panic attacks and (often unfounded) worry that a more severe disease is the cause of the attack (such as cardiac disease or cancers), can hamper a patient in their everyday routine. Embarrassment for the physical and mental symptoms may also lead to avoidance behaviour, restricting social contact. Patients may not want to (re-)visit areas that trigger panic attacks. These areas are usually crowded and hard to get out of quickly such as supermarkets or bus and train transportation. More alarmingly, absence from medical facilities and work floor seriously deteriorate the patient’s quality of life.
Since stress is linked to increased heart rate, the application proposed in this research could also be a positive contribution to the quality of life for people dealing with stress. Stress is present in everyone’s life, be it from professional life or personal worries. A little bit of stress is a commonality, but prolonged and intense stress may cause feelings of fear, or can lead to a burnout. Therefore it is important for people to keep an eye on how stressed they are, and relax when possible if they feel too stressed. Mindfulness is a kind of practice that could be of great benefit to reducing stress levels. This form of mediation teaches you to let your thoughts come and go again, whether they are positive or negative. Stress can lead to people over-analysing their troubles, and getting sucked from one anxious thought into the other by association. Mindfulness therapy aims to teach people to live in the here and now, without worrying about failure (now, or in the future). This allows for better understanding of the negative thoughts, and creates time to reflect on the severity of the perceived problems.
Treatment of patients that experience anxiety or mental stress varies from breathing exercises (although these are used less frequently today) to physical exercise to iterating through a list of small tasks. Breathing exercises are aimed at restoring normal breathing patterns. Physical exercise training is focussed on developing the sense of recognition in patients. Since symptoms of anxiety attacks and stress are very similar to side-effects of physical exercise (such as increased heart rate, sweating, etc.), this type of treatment enables patients to prepare for, and become more resilient to, the sudden surge of discomfort. Another type of treatment focuses on distracting the patient from their bodily functions. The experience of an anxiety attack turns a patient inwards, attentive of their body. Distraction exercises provide the patient with simple tasks such as ”What is the song called that is currently on the radio?”, or ”What does it smell like in here?” to shift their attention away from their body.
It is important for the patient to recognize that anxiety attacks themselves are harmless. Research done in the field of panic and anxiety disorders link low heart rate variability to panic disorder. Heart rate variability (HRV) is defined as the variation in the time interval between consecutive heartbeats and can be detected using for example ECG, or blood pressure analysis. Patients with anxiety disorders have been shown to have a reduced HRV, meaning a more stable pattern of intervals between heartbeats.
Towards a digital assistant
A panic attack itself is characterized by increased heart rate and sweating. These are factors also prevalent in increased mental stress, which may be measured and analysed by a smartwatch. Not only is wearable technology such as the smartwatch a tool that is easy to use and quick to access, it also provides a non-intrusive method to detect panic attacks.
Patients diagnosed with anxiety disorders and more specifically panic disorder can currently receive treatment in the form of therapy sessions with trained psychologists. As part of these treatments, patients are often challenged to do exercises at home as well. For patients with agoraphobia (fear of open and busy spaces) this may consist of visiting a local shopping mall, or taking a busy metroline. Patients that experience panic attacks can be given a short list of tasks that help them distract themselves from the panic attack when it strikes, or homework exercises to help their respiratory pattern.
Wearable technology can possibly help these patients in detecting, helping them during, and recovering from, an anxiety attack or stress. Wearable technology is still up and coming and it is clear that not much research has been done in this field. Through a small feasibility study one learns that there are already apps out there on the consumer market that track a user’s heart rate, as well as HRV. However, none of these focus on coping with anxiety or panic disorders. There are some apps that try to decrease stress, but none of these utilize the smartwatch capabilities and the way of presenting therapy exercises proposed in this research.
Related work
As described above, not much academic research has been done specifically for these types of applications. However, a number of commercial applications have been developed. We will discuss a number of them now to give an overview of the relevant related applications in the current field.
Salahuddin et al. investigated the possibility of using ultra short term analysis of heart rate variability in mobile applications using ECG. 24 subjects were tested during a baseline reading and an experiment reading consisting of the Stroop color-word test. Findings indicate that mental stress levels could be accurately measured within 50 seconds.
Work by Muaremi et al. presents a solution for stress monitoring using a smartphone that uses features from both the phone as well as wearable belts. Audio, physical activity, and communication data were recorded during the day, and heart rate data was collected at night. 35 subjects were tested over a period of 4 months to evaluate the system. This yielded an accuracy rate of 61% for all features combined for stress level classification (3 levels: low, moderate, and high perceived stress). Using just the features from the smartphone, the accuracy dropped down to 55%.
In the commercial world, examples of similar applications include Samsung’s S Health app, and Embrace, a project that recently went live on crowdfunding website Indiegogo. Due to the novelty of these products it is hard to gauge their effectiveness, but we will nonetheless discuss these examples. S Health uses an oximeter to measure heart rate data, and calculates HRV to determine stress levels. Embrace is an upcoming monitoring system that combines a wearable watch with a smartwatch and web application that measures mainly electrodermal activity (autonomic data that is activated by regions deep in the brain involved in emotions such as fear, anxiety, and positive excitement).
Digital assistant
The system that was developed for this research was dubbed Halo (after the Latin verb for to breathe). It consists of an Android mobile application which serves as a platform on which to send and view therapy exercises; an Android wearable application which serves to monitor the user’s physiological data and receive notifications from the mobile application; an online web application that aggregates the collected data and generates a visualization of that data; and a backend to and from which the various applications may read and write data. Therapy exercises that are presented on the mobile and wearable applications during moments of heavy anxiety or stress were designed in collaboration with experts in the field. To test the implementation and study effectiveness, participants were asked to use the system for 2 full days, after which a post-experiment interview was held to evaluate the system. We will discuss each of these aspects of our methodology to a greater extent of detail in the subsections below, starting with the therapy exercises.
Therapy exercises
Together with two experts in the field of Psychology, Health, and Technology at the University of Twente a number of therapy exercises were developed that could reduce stress levels for users of the Halo system. These exercises were generated as an Android notification on the mobile application, and in a shorter form served on the wearable application as well. An example of the exercises used for this experiment was the following.
Notification as served on the wearable application:
Listen to how many different sounds can you hear right now.
Notification as served on the mobile application:
Relax, and try to listen to how many different sounds can you hear right now. First take in the most prominent sounds, the ones with the loudest volume. Discover what other sounds are hiding behind that. Try closing your eyes to focus even more on what you are hearing.
The aim of the exercises is to decrease the users’ stress levels and thereby the heart rate of the user. The exercises find their roots in mindfulness therapy, which has been elaborated upon in the previous section. A set of 15 such exercises were designed for the purposes of this experiment, each containing both a short as well as a full-length version. Part of the exercises contain breathing exercise elements, and most ask the user to be (briefly) more aware of their surroundings. Closing of the eyes is also encouraged if possible, to enhance the stimulation of the other senses.
Exercises more directly connected to the user’s heart rate were considered, such as showing the current heart rate on a radial plot, and also showing the optimal heart rate on that same plot. This would allow a user to discover optimal behaviour (such as breathing patterns) that leads to reduced stress and/or anxiety levels. These exercises were not implemented in the final iteration of the system however, after receiving expert feedback stating that this would likely lead to a user concentrating even more on their own bodily functions (and thus leading to increased stress/anxiety). Instead, the final iteration of the exercises’ design focusses on turning people outward and towards the world, providing a safe vantage point from which to reflect on what is causing one’s stress.
Implementation
In this section we will expand on the implementation of the different components of the Halo system. We will highlight features of the wearable application (based on an open source heart rate monitoring algorithm), the mobile application, and the web application, respectively.
The mobile and web application are tied to a backend using the Parse cloud platform that contains a database, a registration system, and a login system. A graph showing these connections and dependencies is shown in Figure 1.
The wearable application saves sensor event data into the shared data layer, and is able to notify the mobile application of changes in the data layer, or open a notification, through intents transmitted over Bluetooth. The mobile application, in turn, pushes notifications to the wearable device over Bluetooth. It also stores pulse data, metadata, and other information registered during the experiment to an external database. The web application retrieves this data to build a visualization model. It also features a registration system that connects to the database.
Figure 1. Halo system overview.
Wearable application
The wearable Android application is adapted from the Moto360 monitor and utilizes, besides the heart rate sensor to monitor a user’s heart rate, the (significant) motion sensors, step detector sensors, as well as humidity sensors to detect events related to sporting activity or other types of activity that could increase a user’s heart rate but are not stress-related.
During these periods of time, sensor data indicating a high stress level moment is disregarded. Sensor events are fired a couple of times per second and registered on the smartwatch application. Sensor events have raw sensor data associated with them (i.e. in the case of a heart rate sensor event, the beats per minute), as well as an accuracy value, and a timestamp for the event. The accuracy value is a value between (and including) 0 and 3, and indicates how accurate the reading was.
To ensure that only reasonably accurate data is used, all sensor events of accuracy 0 are disregarded. Sensor events of accuracy 1 were opted to be used for this experiment, due to the occurrence of periods of sparsity in sensor events with a high-level accuracy (> 1). Sensor event data is aggregated every 10 seconds, averaged, and stored as a DataItem into the shared storage existing between the wearable and mobile Android applications. A DataItem is synchronized across all devices in an Android Wear network. The mobile application is then notified of a change in this storage, and the new values can be retrieved.
The interface of the wearable application shows only the Halo logo, since for the purposes of this experiment, the watch is used merely to record physiological data, and show notifications. Once a stress event is fired (more on this in the following subsection), the notification will be pushed to the smartwatch. The user is then able to read the (short form) therapy exercise, and can opt to open it on his/her smartwatch as well to read the long form. A visualization of this is given in Figure 2.
Figure 2. Left: A therapy exercise is visible on the smartwatch application. Right: An action button allows a user to read the longer version of the exercise on their mobile application.
Mobile application
The mobile component of the Halo system consists of the Android application. This component reads the DataItem’s pushed from the wearable application into the shared storage after being notified by the wearable application. From this, it retrieves periodic sensor data, such as heart rate data (in beats per minute), and step detector data (which triggers an event each time the user takes a step). Physiological is retrieved from storage about every 10 seconds, depending on the accuracy of the readings.
The interface of the mobile application is designed to be very straightforward and user friendly. One switch starts and ends a baseline reading, and another starts and ends an experiment reading. A purposeful decision was made not to show feedback to the user regarding the physiological readings, since this may in fact trigger an anxiety attack or raise stress levels further. A visualization of the mobile UI is shown in Figure 3.
Sensor data related to a user’s pulse (heart rate), is stored from the mobile application using the Parse backend into a database. Besides heart rate, a timestamp and reading type (baseline or experiment) are also stored. Metadata such as the average heart rate, and standard deviation is also saved to the database. This information is used to determine when a user should receive a notification with a therapy exercise. Once a user’s heart rate exceeds the threshold of 2 standard deviations above their mean heart rate and a gradient is detected in heart rate values (i.e. there is a significant rise in heart rate over a short period of time), a stress event is registered, which triggers the generation of a therapy exercise notification. Statistically this will happen in about 2.5% of pulse measurements. Given an average of 53.54 pulse measurements in 8 hours, this would result in 1 to 2 peak moments on average. A therapy exercise is then selected at random from a list of 15 exercises in total. Once a notification has been triggered, no subsequent notifications were generated within the next 30 minutes. This refractory period was implemented to prevent the notification system from firing too often when the sensors return inaccurate data. Another reason is that the therapy exercises often take a minute or two to complete, and that a user would probably not have time to continually preform these exercises if they follow each other too rapidly.
Once the notification has been pushed to the smartwatch, the user is able to open it on their mobile phone as well. If this action button is pressed, the application will fire up and display an extended version of the therapy exercise. Since the mobile application provides more space to work with, and room for a user to navigate, additions such as a favourite exercise system can be implemented more easily in contrast to the wearable application.
Figure 3. The interface for the mobile application. Left: Baseline reading button highlighted. Right: Experiment reading button highlighted.
Web application
A user may view his or her recorded physiological data through the Halo web application. The web application consists of a landing page on which the various components of the system are showcased, a registration/user system that allows a user to sign up and use the web application securely, and a dashboard page. A user manual that explains how to operate the devices and use the system, as well as terms and conditions for using the system may also be found here. The dashboard page plots the user’s physiological data. The vertical axis shows the BPM values of each measured Pulse object, and the horizontal axis shows the timestamp at which this value was received by the mobile application and sent to the external database. Figure 4 shows a screenshot of the dashboard page.
Since a user’s mean heart rate is saved during the baseline reading, it is used in the visualization to indicate areas of the curve with BPM values that exceed this value (marked orange), and areas with BPM values that fall below this line (marked blue). By default, the view is set to that of half a day, but the entire history can easily be viewed, and one can zoom to a specific point in time to inspect that part of the graph with greater care. Also shown in Figure 4 is the button to turn on live updates. Switched on, the graph will only show the latest 5 pulse measurements. This could be useful for real-time therapy sessions, where it is less important to view the activity of a long period of time, but rather the immediate response in stress is considered. The web application was shown to the participants after the second day of the user evaluation and before the post-experiment interview. This was done to simulate such a therapy session setup, and to prevent the observance of the recorded data interfering with the user’s self-reported stress levels.
The web application was constructed to serve as a guide to both therapists and patients dealing with anxiety and stress, as well as normal people striving to decrease their stress levels. Since all physiological data is timestamped, it is easy for a user to review his or her day. While doing so, he or she could recap critical moments at which a rapid increase in heart rate was measured. Reflecting on this, perhaps in the company of a therapist, one could then focus on why this occurred at this exact moment. This is crucial because patients often have trouble distinguishing what event lead to the anxiety attack or stress, as described in the previous section. The dashboard might help with exposing these underlying factors.
Figure 4. Fragment of the dashboard page of the web application. Shown is a graph of a user’s heart rate (BPM versus time), as well as an option to turn on live updates (showing only the latest 5 pulse measurements). Values in the plot above the measured mean heart rate are coloured orange, and values below the mean heart rate are coloured blue.
User evaluation
An experiment was set up using the implementation of the Halo system as described in the previous section. As noted before there is difficulty in testing an application such as this in a clinical setting with patients suffering from anxiety or panic disorder in the scope of this research project. For this reason, experts in the field (rather than patients) were asked to participate, and a subjective post-experiment interview with accompanying questionnaire was used.
Participants were invited to participate in an experiment lasting two days. On the first day, a baseline reading was performed, and on the second day an experiment reading was performed. During the baseline reading, a series of pulse readings were tracked for an average period of 8 hours over all participants. The participant was instructed to keep wearing the smartwatch for as long as it was comfortable, and that if he or she might find it wearisome or annoying to wear, the reading could be stopped prematurely as well.
During the experiment reading, similar pulse readings were tracked. The participant was instructed to wear the smartwatch for the same amount of time as had been done during the baseline reading. When an anxiety or stress event had been detected, a therapy exercise would be provided, as previously described in the previous section. In addition to these exercises, the participants would also receive therapy exercises at 5 random points during the reading (relatively evenly spaced out) as a control measure for evaluation and to ensure that at least some messages were generated in case participants would remain very relaxed.
The participant was instructed to follow these instructions as he or she would see fit. The participants had been given an activity form to be filled out after the end of the experiment reading. On this form, the participants were asked to draw a graph of their perceived heart rate on the second day ofthe experiment, and indicate moments of activity or events and the kind of activity or events. The graph was to be drawn on a timescale from waking up to going to bed. The marked events were also to be placed on a circle scale to indicate how stressful this event was for the participant. After this, the participant was invited for an interview.
Results
Participants evaluated the application as High Excellent (89.06 System Usability Scale (SUS) score). These were the most interesting answers to the interview questions:
What did you feel were the main problems with using the Halo system?
- The ability to just read small messages on the smartwatch is hindering. Users should be able to read large messages on there as well if they want to.
- Users that are stressed (in a hurry, or busy with other things) sometimes simply do not have the time to do a large therapy exercise, thus negating the effect of the exercise.
- Therapy exercises should not be repeated in a single day if the user has not explicitly chosen for this.
What did you feel were the main advantages of using the Halo system?
- It is very helpful to stop for a minute or two to look at things from a higher point of view for a second. Taking a step back and focussing on a therapy exercise helps to relax.
- The system is easy to use since a watch is always on you, as is a mobile phone (usually). This makes it so that the system falls to the background and you don’t think about it any more.
- The insights from the web application are very useful for users wanting to better themselves. Especially since users may sometimes not know what caused a stressful moment, or if they feel like something was very stressful while their heart rate shows otherwise. This opportunity to reflect is very helpful in self-improvement.
What are some main points of improvements for the Halo system?
- Generate a summary for the user at the end of the day, and push this to the wearable/mobile devices as well.
-Allow for an option to do certain exercises a bit later as well, when more time is available to the user.
- Generate short therapy exercises, or exercises that can be easily performed doing other labour during working hours. More time-demanding exercises can be generated outside of these hours.
Activity form
Data from the activity form showed that participants had 3-4 stressful events during the experiment reading, ranging from 2-4 on a 1-5 scale. When compared to the heart rate activity as measured by the system there was about 70% overlap. A relatively large amount of peaks in the recorded data could be related to the perceived drawn heart rate graph. However there were instances in which the recorded peaks differed from the peaks reported the activity form. In one participant’s activity form for instance, one perceived peak was the inverse of the recorded data. In other words, the participant reported that she believed her heart rate was rising, when in fact it was decreasing.
Concluding
This project looked at the possibility to implement a digital anxiety disorder and stress assistant on wearable technology. In the course of the project, the focus was enlarged to also incorporate normal users, rather than patients suffering from anxiety disorders. Detection of high stress levels has shown to be very possible using wearable technology. They can also be detected in time to provide relevant support. The wearable device is able to read sensor data a couple of times per second, which is more than enough to provide real-time support. Wearable technology is also a very useful medium to provide users with therapy exercises.