Tracking social rhythms of the heart From dataism to art Veera mustonen

We conducted a curiosity-driven study to explore what a vast body of self-tracking data could reveal about the rhythms of everyday life. We instructed thirty-six research participants to engage in self-tracking for a week. They measured their physiological stress and recovery 24/7 for this period. In addition to that the participants recorded their subjective experiences of stress and recovery. Using different methods of analysis and interviews, we were able to form data sets demonstrating both individual behaviour and interpretations of the data and the collective rhythms of all the participants. Our analysis contrasted the aggregate-level ‘big data’ of all the participants and the personal-level ‘small data’. People’s subjective evaluations of their stress and recovery systematically differed from the physiological measurements. The big data revealed behavioural patterns and causalities that were not recognized at the individual level. The small data, on the other hand, offered rich material for personal interpretations and reflections of the individuals’ own lives. To communicate both levels of the data our science project resorted to artistic expressions.


Tracking social rhythms of the heart
From dataism to art Veera mustonen & mika pantzar W e conducted a curiosity-driven study to explore what a vast body of self-tracking data could reveal about the rhythms of everyday life. We instructed thirty-six research participants to engage in self-tracking for a week. They measured their physiological stress and recovery 24/7 for this period. In addition to that the participants recorded their subjective experiences of stress and recovery. Using different methods of analysis and interviews, we were able to form data sets demonstrating both individual behaviour and interpretations of the data and the collective rhythms of all the participants. Our analysis contrasted the aggregate-level 'big data' of all the participants and the personal-level 'small data'. People's subjective evaluations of their stress and recovery systematically differed from the physiological measurements. The big data revealed behavioural patterns and causalities that were not recognized at the individual level. The small data, on the other hand, offered rich material for personal interpretations and reflections of the individuals' own lives. To communicate both levels of the data our science project resorted to artistic expressions.
Wearable and portable tracking devices are a way to augment human functioning with technology. The emerging technologies generating vast amounts of data of the self are commonly referred to as quantified self technologies 1 . Enhancing oneself with data via self-tracking one's everyday life is becoming more and more common. Today consumers can select of over 20,000 applications such as heart-rate monitors, or mood or physical activity trackers to monitor their body, behaviour or environment. So far most people are not using any of these applications. However, the lead users are already there, most notably the pioneers of the quantified self (QS) movement which started in California 2008 (Swan 2013, Pantzar andRuckenstein forthcoming 2013). QS activists meet regularly to share their personal data and visualisations of -for example -their quality of sleep versus environmental pollution levels or food intake. The core of the QS movement is the data, the magnitude and constant flow of which opens new possibilities. Mika Pantzar and Minna Ruckenstein (forthcoming 2013) introduced the concept of 'dataism' to underline the QS movement's belief in the significance of a data-led everyday life. Personal exchange is an important force in the quantified self movement. The excerpt below illustrates how personal interest, manufacturer interests and curiosity collided during a conference at Palo Alto 15.9.2012: Back on September 15th-16th I had the pleasure of attending the third-ever Quantified Self Conference at Stanford University in Palo Alto, California. It was an amazing event, packed full of great show-and-tell talks, break-out sessions, and product demos. With so many overlapping talks taking place, my only regret was not being able to catch every presentation. I came away from the event inspired, a little bit tired, but managed to put together some of my highlights and observations: … Heart Rate Variability (HRV) is a fascinating subject. One of the areas I am most interested in is quantifying, correlating, and reducing stress. Ronda Collier from SweetWater Health (they make an HRV app called SweetBeat) gave a great talk on the subject: Heart rate variability (HRV) is the physiological phenomenon of variation in the time interval between heartbeats. It is measured by the variation in the beat-to-beat interval. In other words, having an extremely steady heartbeat is a bad thing. Low HRV is an indicator of stress, depression, lack of willpower, and inflammation. Talk about a mind/body connection! Athletes can use low HRV as an indication of overtraining and when to take a day off. Daily slow, deep breathing exercises can increase vagal nerve tone, and bring your nervous, cardiovascular, hormonal, and immune systems into a state of 'coherence' (working harmoniously)the more time spent in coherence, the healthier and happier we are. 2 (Quantified Bob 2012) Curiosity was one motivator that drove us to explore how dataism could work out in the context of everyday life. We collected a massive body of data by using tracking devices to explore the times and places of stress and recovery for thirty-six people for a week. The participants volunteered to carry three portable or wearable tracking devices all day and all night for a week. The monitoring devices utilized in the study collected data from the users via different sensors. The products used were wearable/portable in different ways: (1) a smartphone, tracking place, acceleration and levels of noise, (2) a wrist watch, measuring activity levels and (3) a heart rate variability (HRV) monitor, measuring levels of stress and recovery, which took the form of electrodes taped on to the skin of the chest. As a result we got a large body of data, consisting of millions of heartbeats, GPS location coordinates and dozens of hours of interviews in which the people who had taken part reflected on their everyday whereabouts and levels of stress or recovery.
Thus we required people to track themselves, using the quantified self type of tracking devices. Then we aggregated the data, producing a body of 'big data' detailing daily rhythms of stress and recovery. As a result we had on our hands two different sets of data: an aggregate, big-data set and individual, small-data set tracking histories consisting both of physiological data and subjective reports. Aggregate data emphasize the routine and sameness of life (McCarthy and Wright 2004), whereas the stories describing episodes as they were experienced, the small data, il-2 The stress meter has three related attributes: quantifying, correlating and reducing stress (Quantified Bob 2012).
lustrate how people coped with the routines. Big data revealed causalities that could not have been detected from small data, such as the fact that fitness is the most effective way to reduce stress levels. Simultaneously big data marginalizes individuals, or segments them and thus obcures the variety, change and complexity of everyday life. On the other hand, the small data is rich in detail and surprises.
Performing the research and collecting the data was a very thought and emotion-provoking experience, both for the participants and for the researchers. We observed that the participants found their own self-tracking and data curious. Many would like to have continued tracking their stress and recovery patterns. Moreover, both the act of measuring and the data generated seemed to raise more questions than it gave answers to. The art of self-tracking seemed to open up possibilities and prompt people to ponder their implications. As researchers we were facing a similar question as were the people tracking themselves: how to interpret the data? What does it tell us about the self -or about humans in general -or about the human-machine relationship? Are new self-tracking technologies making life more measurement oriented? One can also -as did Professor Alf Rehn during the Aboagora panel discussion 'Technical enchantments' , out of which this article stems -pose the Lacanian question: what is the object of desire of the quantified self?

Social rhythms of the heart
What did we learn? First of all, individual tracking data can be aggregated to produce big data, showing interesting rhythms and patterns. The social life of Minna Ruckstein measures her brainwaves with a realtime 3D construction at a QS-event in Stanford, Palo Alto. the heart can be measured. Importantly, the heartbeat does not utter purely internally in relation to bodily phenomena, but also to interrelated activities within social networks ('biosocial circuits'). At the level of social rhythms, our aggregate findings replicated earlier findings of circadian and weekly rhythms (Foster andKreitzman 2004, Koukkari andSothern 2006). For instance, the days were repeating clear biosocial rhythms: stress peaks occurred everyday at 8 a.m., 3 p.m. and 8 p.m. Moreover, we found clear evidence for a lower level of stress on Saturdays and lower levels of recovery on Fridays. However personal rhythm profiles vary to a great extent, especially in the case of recovery. Fitness seemed to affect stress, recovery and wellbeing more than gender, age or body mass index. The daily stress level of physically active people decreased during working hours. This was not the case with physically less active people. Having children at home seems to increase average stress levels but diminish extremes. These findings suggest the need for further research.

Expressions of stress and recovery
When comparing physiological and self-reported expressions of stress and recovery, interesting yet systematic differences were found. All in all, we analyzed 161 episodes from the transcribed interviews, which were subjective, temporal, compact stories, each having a clear start, end, goal and actor(s). The duration of the episodes varied from a few minutes to a few hours. The episodes were subjective narratives and so the duration depended on the personal experi ence of the onset and offset of the story. In all of the episodes either the experience of stress or recovery, or lack of them, was an attribute. Only in less than a half (42 %) of the captured episodes, did the heart and mouth agree about the experience of stress or recovery. The majority of cases in which the expressions of the heart and the mouth differed could be classified into three categories. In the first category, a strong and fixed cultural interpretation of a certain situation (e.g. 'work stress' or 'drinking as leisure') leads the mouth to express cultural conventions while the heart disagrees. The most typical case in this category was when the heart expressed recovery while the mouth did not. This case was typically related to different routine tasks, for instance, home chores and routine work activities and meetings. A software developer was physiologically recovering while working through the night. Likewise many knowledge workers were measured to have recovered during meetings, although they denied having experienced any moments of relaxation. Another type of contradiction classified in the data was episodes where one attempted to relax, but physiologically that was not measured as recovery; that is to say the mouth expressed recovery while the heart did not. In cases of preparing for a future task (e.g. transition from home to work), physiological levels of stress in-Weekly rhythms of stress. Each coloured line represents a one-day aggregate moving average of the level of stress during the day. The stress peaks are around 8 a.m., 2.30 p.m. and 8 p.m. each day.
creased, but the subjects did not report experiencing it to be so.
Obviously the expression of the heart and the mouth are connected in several ways. The machine measuring the physiological stress is still a cultural artifact, built upon the theories of stress. Thus, its algorithms are based on human and social conceptions of stress and recovery (although a scientific expert knowledge is also included). The heart itself as an integral organ of the body is in constant interaction with the world outside in the form of the social rhythms of everyday life. Thus the expressions of the heart are also social. Likewise speech reflects both individual (cognitive, emotional, biological) and sociocultural representations of stress. People reflect their stress data in terms of activities and interactions, thus the social rhythms.
At the personal sense-making level of the data, people found the differences between the physiologically measured and the self-reported levels of stress and recovery intriguing. For most people the study period functioned as an intervention, enhancing selftranscendence, although no goal of this sort was set. Most of the participants did find one or two things in their everyday habits that they considered altering as a result. One person started to go to work earlier in the morning in order to get a couple of tranquil hours there before the arrival of two colleagues whose presence caused his stress level to rise. Data of the self may thus increase awareness. Could self-tracking technologies be seen as expressive technologies (Mumford 1964), that is, as technologies supporting one's self-expression and transcendental purposes?

From data to art
Already some artists are investigating the possibilities of self-tracking for artistic expressions. Laurie Frick (2013) has turned her personal minute by minute data of her sleep, eating and other daily activities to aesthetically intriguing 2D works and installations. She transforms her self-surveillance to art which seems to reveal patterns and language of everyday life.
As researchers, reporting the findings in the ordin ary way, that is, in scientific papers, did not seem to reveal the essence of the experience we had witnessed. Therefore, our social science research project took on an artistic twist. Our first orientation towards the artistic sphere was through visualisation of the data. We developed a specific interface which enabled us to compare different people's levels of stress and recovery, and to aggregate overall series. The aesthetic experience of 'social life of the hearts' excited us. Seeing both the 'big data' aggregate level, the social rhythms, and the individual 'small data' differences at the same time, and in constant flux, was a more revealing experience than analysing the numeric data. In the animation the multidimensionality and liveliness of the data of the heart became more evident. We were able to see how people are waking up around the same time. In the animation we could grasp different individual patterns: some people had more stress peaks than others. We could also detect individual stories: we knew that a certain stress peak in Lisa's life meant that her daughter had escaped from the kindergarten playground. Or we knew that in the case of Max, a certain night lacking recovery of sleep was due to a teambuilding event after which his roommate snored loudly having had a few beers.
Maps, charts, diagrams and graphs make things stable, mobile, durable, and comparable, and they convince others: 'Inscriptions devices are "intellectual techniques": material techniques of thought that make possible the extension of authority over that which they seem to depict ' (Rose 1999: 37). In the coming years the visual displays of information will play a key role in solving the problem of the abundance and reception of data in businesses, scientific work and everyday life alike. New information technologies and growing masses of data could lead to radically new (scientific, cognitive) visualisations that are combinations of theory, experiment and art. Maps of the future will involve, besides colour and form symbols, also soundscapes and possibly animation and simulation, through which we can identify dynamics in everyday life (Greenacre and Hastie 2010). A development optimist sees in animations an opportunity to learn, share and generalise the personal experiences of millions of people. Animations can be used to perceive and illustrate, for ex ample, what kind of an effect eating one potato less per day will have on the long-term progression of a person's weight. Peer groups can also share life management solutions through statistical data they have gathered about themselves.
An essential feature of this current development is that maps are being produced, not only in businesses and various fields of science, but also on the terms of individual people and in light of their own data. And the nature of reality depends on the angle from which it is observed. ' A' can be at the same time true and false, always depending on the perspective from which it is being viewed. As a result of this kind of thinking, the assumption of scientific representations as unambiguous, truthful depictions of reality has been increasingly questioned (Knuuttila and Honkela 2005). One could in fact speak of a crisis of representation. For this reason, for example, John Bender and Michael Marrinan (2010) propose that instead of representation and depiction we should turn our gaze towards a 'culture of diagram' . The relationship of the diagram to its object of representation is always a sum total of many choices: ' A diagram is a proliferation of manifestly selective packets of dissimilar data correlated in an explicitly process-oriented array that has some of the attributes of the representation but is situated in the world like an object. Diagrams are closer in kind to a Jackson Pollock than to a Rembrandt' (Bender and Marrinan 2010: 7).
A curiosity-driven study, such as ours, may be seen more as a form of art than of science. Moving from dataism to art, we want to pose the questions of the data to a larger audience. Scientific research can be seen as a cultural commentary or as a cre ation (Wilson 2002). Communicating to a wider audience than other researchers may be one of its aims. Moreover, art enables the aesthetic incorporation of personal stories and aggregate patterns. It initiates a dialogue and leaves interpretations open. Interviews with the participants taught us that everyone came up with different views of their personal data. Transforming data into art enables the voices of individuals to be represented. The next step of the art project is to hand over our material to an artist and digital media producers. The goal is to present the social rhythms of the heart on big urban screens in the city of Helsinki. Before reaching the wider audience, the social rhythms of the heart will find new forms of expression.
What then do we want to communicate by means of art? Rather than a clear message related to stress, recovery or health in general in the era of the quantified self, we want to share our curiosity. We wish to express stress and recovery in an artistic way, as felt experiences, as well as a social phenomenon which has been mediated via technology. The question is not: is the algorithm 'true' , nor 'is this real stress or recovery'? Rather the question is: how could this contribute to my life? What can the quantified-self type of data do to augment individual and social lives? Could measurement technology and the data be taken as a sensual, intellectual and emotional experience? We believe that the audience will enjoy a multitude of insights, awakenings of imagination or aesthetic experiences when getting in touch with the 'social rhythms of the heart' .  Veera Mustonen (MA) is a cognitive scientist and senior researcher at the National Consumer Research Centre in Helsinki, Finland. She is a doctoral student in industrial engineering and management at Aalto University. Her research focuses on the emerging digital Health 2.0 field, behavioural change and the human-machine relationship. Prior to her current academic pursuits, she worked for 13 years in the industry developing human technologies such as smartphones (for Nokia) and eLearning systems (for Sanoma WSOY). Email: veera.mustonen(at)aalto.fi Mika Pantzar is a research professor at the National Consumer Research Centre in Helsinki, Finland. He received his doctorate in the field of management and organizations at the Helsinki School of Economics and Business Administration. Among his current research interests are the economics of sport, health and wellbeing, and, relatedly, the big data and quantified self movement. He has published articles widely within consumer research, design and technology studies, the rhetoric of economic policy, food and future studies and systems research. Recently he co-authored a book with Elizabeth Shove and Matt Watson entitled Everyday Life: The Dynamics of Social Practice (Sage 2012). Email: mika.pantzar(at)ncrc.fi