Validation of consumer wearable activity tracker as step measurement in free-living conditions

Different generations of consumer wearable activity trackers are prevalent with the increasing demands in health and physical activity monitor. This pilot study aims to validate one of the consumer wearable activity trackers, the Mi band 2 as a step measurement in free-living conditions. Thirty-one healthy volunteers, aged 23 to 45 with 16 female (52%), wore both Mi band 2 and ActiGraph GT9X Link on their dominant hand’s wrist for seven consecutive days. The validity of the electronic activity devices was assessed objectively by average steps/day using i) Paired sample t-tests; ii) Pearson correlation. In addition, BlandAltman plots was constructed to visually inspect the data and to assess agreement with the ActiGraph accelerometer. There was a high correlation in steps/day between the reference device, Actigraph accelerometer and Mi Band 2 (r = 0.97, p < 0.001). No significant mean different in steps/ day and no apparent systematic biases in the BlandAltman plots between step count measurements obtained using the Mi Band 2. Xiaomi Mi Band 2 provided valid step count measurement in the free-living conditions.


Introduction
Healthy practices could be facilitated by different technological means [1][2].In the recent years, the increasing popularity of consumer wearable activity tracker brings researchers' interests in using it as one of the tools in physical activity [3][4] and walking [5] interventions.As consumer wearable devices provide feedback and offer interactive behavior change tools via a mobile device, or computer for long-term tracking and data storage, there are concerns in the validation of these devices [6].
Steps is intuitive, objective and easy to measure as a metric for assessing physical activity and the number of steps per day is usually used for quantifying ambulatory physical activity [7].The validity for steps measurement of consumer wearable activity trackers are usually performed by comparing its counted steps against different criterion measures, such as manual step counting, either in-person [8] or with video recordings [9], or steps recorded by pedometer [10] or accelerometer [11].Different brands and models of wrist-worn consumer wearable devices were examined for its accuracy in step measurement in laboratory setting, e.g.treadmill 11. 3.2019 FinJeHeW 2019;11(1-2) 69 walking [12], level walking [13] and stair walking [10].
Most of these consumer wearable devices showed high in correlations in step measurement with the reference criteria in the systematic review [14].However, mixed results of different brands and models were found in evaluating its step accuracy in free-living conditions [15].For example, Garmin Vivofit showed higher validity than Polar Loop in measuring daily step counts in free-living conditions [16] while Fitbit One and Fitbit Flex showed no difference in step measurement with the criterion measures in free-living conditions [17].
To our knowledge, the validation of popular new device, Mi Band 2 (Xiaomi Corp. China) has not yet been covered in free-living conditions.Therefore, the purpose of this pilot study is to assess the validity of the step count of this popular model of electronic activity monitor device in a population of healthy adults during free-living environment.

Study population
A convenience sample of 40 healthy volunteers was recruited.This is comparable to the sample size used in previous electronic activity devices validation studies of step measurement in free-living conditions [11,18].It was suggested that step count of consumer-level activity monitors in free-living conditions correlated with reference devices above r =.80 [3].To detect weather a correlation of 0.80 differs from zero, with α =.05 and β =0.20, a sample size of 9 participants would be needed [19].Participants were eligible for inclusion if they were aged 18 years or above, willing to monitor their activity for a seven day period, worked or studied in the University and could walking freely without restrictions and aids.Participants were excluded if they were in injured or being affected by illness for mobility.In this study, 40 volunteers with the ability to walk without aid were recruited from the university community.Participants' step count, measured by the accelerometer, ActiGraph GT9X Link, was taken as the criterion measure for steps.

Research ethics
The study was approved by the Senate Committee on the Use of Human and Animal Subjects in Teaching and Research, of the University.The participants were asked to consent to the research and were informed about the use of the data.

Procedure
The Informed consent was obtained from the volunteers after the explanation of possible risks and benefits associated with the experimental procedure.Participants' demographic data (age, height, mass, gender, and dominant hand) were measured and collected at the beginning of the test.Height, mass and gender data were entered into the Mi Band 2 account and setting in accelerometer for each participant prior to the 7-day walking test.The devices were set up with unique user accounts.

Instruments
The Mi Band 2 devices were bought from a retailer directly while the manufactures of the Mi Band 2 device has no role in the funding, design or conduct of the study, or analysis of the results.Mi Band 2 has a military-grade accelerometer that converts acceleration to step counts using proprietary algorithms.Both designs of Mi Band 2 and ActiGraph GT9X Link are fit for wearing as wristband.
All individuals were invited to wear a Mi Band 2 (Xiaomi Corp., China) with and ActiGraph GT9X Link accelerometer (Actigraph Inc., USA) at their wrists of dominant hands.Participants were asked to wear both devices during the waking hours, except swimming and bathing.They were also suggested to remove the devices during sleeping.Participants completed an online daily diary to record number of steps.
After seven days, participants were invited to return the devices and provide their written feedback on Mi Band 2, using a utility questionnaire adapted from previous research of Tully [20] in order to further investigate

Data treatment
Daily steps were recorded for each participant.An average steps/day was calculated for each participant.This was calculated by summing the total number of steps taken between testing and dividing by the number of valid whole days (i.e. 5 days).Data was cleaned by removing non-wear time for the Actigraph accelerometer.Non-wear time was analysed as a run of zero counts lasting more than 150 minutes.To be included in the analysis, subjects had to provide at least five valid days of the Actigraph data.A valid day was defined as a 24 hour period in which at least 10 hours of data wear time was recorded in an electronic spreadsheet.Data from Mi Band 2, i.e. steps per day was recorded from the apps.The researcher conducted the data at the end of the 7-day wear period and average steps/day was calculated.At the end of the study, the Actigraph data was analysed using Actilife version 6.13.3 (Actigraph Inc., USA) to calculate steps per day in these seven days.

Data analysis
Statistical analysis was performed using Statistical Package for the Social Science (SPSS) (Version 23).Descriptive statistics were calculated for each variable.Normality was assessed (Shapiro-Wilk test of normality) on the step count data to determine the use of nonparametric or parametric techniques.
The validity of Mi Band 2 as step measurement of freeliving physical activity was assessed by comparing its output (steps/day) with that of the Actigraph accelerometer (steps/day).The testing protocol is based on the recommendations from Welk's study [21] and previous activity monitor validation study [20].To test the validity of electronic wearable activity tracker, Welk and his research team suggested that three aspects are needed to demonstrate the following agreement.First, the two measures being compared must yield equivalent group estimates (evidenced by mean difference).
Besides, the measures must be associated with each other (evidenced by correlation coefficients).In addition, the measures must be free from bias (evidenced by Bland-Altman plots).
As normality shown on data, it was analysed into three aspects:1) Paired sample t-tests were used to evaluate mean difference in step counts between the Mi Band 2 and the ActiGraph accelerometer.2) Pearson's correlation coefficients and the p-values were calculated to provide an indication of the relationship between the recorded step counts from the Mi Band 2 and ActiGraph accelerometer.In order to assess the agreement between measurement of these two devices, Bland-Altman plot was used as the standard method [22], both visually and statistically interpretation.The difference in the step count measured by the two devices, is plotted against the averages.

Results
There were 40 volunteers for the study at the beginning of study.Three individuals failed to participate in the test while six individuals' records were less than four valid days.At the end of the one week recording period, valid data was available for 78% (n=31/40) of those who participated.There were 31 participants, mean (SD) age 32.5 (7.15) years, participated and 53% of (n=16) of the cohort were female.The mean and interquartile range of cohort's characteristics are provided in Table 1.
There are no statistically significant difference observed in steps between the Mi Band 2 and Actigraph accelerometer.Comparing the Mi Band 2 with the reference device demonstrated high correlation with steps/day measured (r = 0.97).3).According to the rule of thumb for interpreting the size of correlation coefficient [23], it was the very high correlation.The limits of agreement (± 1.96SD) reflect where 95% of all differences between measurements are expected to lie.There was a significant (p < 0.001) and very strong correlation between step count measured by the Mi Band 2 and corresponding step count measurement using the Actigraph accelerometer.
For visually and statistically interpretation, Bland-Altman plot was used to illustrate the differences be-tween step count measurement by Actigraph accelerometer and Mi Band 2. It revealed no systematic differences between the Mi Band 2 and Actigraph measured steps/day (Figure 1).
Overall speaking, there was a high acceptability of the Mi Band 2 among the respondents.All of the participants commented that it was easy to use the Mi Band 2 in measuring step every day.The majority of respondents rated the Mi Band 2 as acceptable to use and easy to integrate into their daily routine.All of the participants commented that it was not annoying to use the Mi Band 2 (Table 4).

Discussion
The use of consumer wearable activity tracker is associated with modest changes in steps, blood pressure and HDL cholesterol in previous study [4].As the number of steps per day is usually used for quantifying ambulatory physical activity [7], it is crucial to have accurate step measurement of these trackers for health benefits.In this pilot study, Mi Band 2 is a valid device for monitoring step counts in free-living conditions.It is proven by the following evidences.First, there is no significant difference between Mi Band 2 measured steps/day with that of the criterion measure, Actigraph accelerometer.Besides, the high level of Pearson's correlation coefficients (0.97, p<0.001) of Mi Band 2 and the accelerometer reflected that their number of measured steps are highly associated.In this experiment, most data collected by Mi Band 2 fell within the 95% limits of agreement with that of the accelerometer.Furthermore, from the Bland-Altman plot, we found that there is no apparent systematic bias.Therefore, the use of Mi Band 2 as a measure of step in free-living environment is recommended.This is echoed with author's research that Mi Band 2 is a reliable and valid device for step counts in the laboratory setting [24].The result is also consistent with the systematic review that consumer wearable activity trackers are high in validity in step measurement in different circumstances [3].
Besides, as consumer wearable activity devices have become more and more popular, it is important that these devices are user-friendly for health status monitoring, recording and exercise promotion.User perceptions and experiences in using these devices are therefore crucial.In the cross-sectional study of users' experiences of wearable activity tracker [25], it is clearly stated that users find activity trackers appealing and useful tools for increasing perceived physical activity levels over a sustained period.The survey from this pilot study supported that most of the participants favourably rated the utility of the Mi Band 2 and it is not difficult to use it every day for step recording.Consumer wearable devices are one of the useful intervention tools for increasing physical activity among different populations [26][27][28].This implied that Mi Band 2 could be further used as one of the interventional tools in promoting daily step and physical activity level.
There were limitations of this pilot study.First, the convenience sampling may limit the generalizability of the study.The participants of this pilot study were university students or employees.Further investigation may necessary for validation in other population, e.g.different age groups.However, the included participants undertook a wide range of physical activity levels (ranged from 6712 to 14901 steps/day), suggesting they are representative of the population.Besides, validation of other new models and brands could also be included in the further study in order to have a comprehensive comparison among these consumer wearable activity trackers in measuring steps in free-living conditions.
In conclusion, Xiaomi Mi Band 2 provided valid step count measurement in the free-living conditions.Furthermore , the relatively low cost of Mi Band 2 (USD $30) may attract more and more people willing to take the lead to use electronic activity devices to facilitate their health status monitoring, recording and physical activity participation.

Figure 1 .
Figure 1.Bland and Altman representing comparison between the criterion measure (Actigraph Accelerometer) and the Mi Band 2 step count output.Solid line indicates the mean difference between Actigraph Accelerometer and Mi Band 2, and the dashed lines indicate the limits of agreements (1.96 ± SD).SD, standard deviation.

Table 2 .
Pair Sample t-test on step measurements of Mi Band 2 and the Actigraph (n=31).

Table 3 .
Comparison of the Mi Band 2 with the Actigraph Accelerometer (n=31).

Table 4 .
Participants' responses towards utility of the Mi Band 2 physical activity monitor.