Details for fitbit_sleep_analysis.ipynb

Published by Wenqiu999

Description

Analysis of sleep data collected by Fitbit.

0

Tags & Data Sources

fitbit connection sleep fitbit connection

Comments

Please log in to comment.

Notebook
Last updated 3 days, 3 hours ago

This notebook is created to analyze sleep data collected by fitbit. It used public data as a demonstrate to show how the plots look like.

You can use the code below to analyze your own sleep data. And This notebook is free to reuse and adapt, distributed under an MIT license: https://opensource.org/licenses/MIT

In [1]:
import pandas as pd
import json
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import urllib
import requests
import os
import tempfile
from datetime import datetime
from ohapi import api


user_details = api.exchange_oauth2_member(os.environ.get('OH_ACCESS_TOKEN'))
for i in user_details['data']:
    if i['basename'] == 'fitbit-data.json' and i['source'] == 'direct-sharing-102':
        fitbit = requests.get(i['download_url']).json()
'''
data = requests.get('https://www.openhumans.org/api/public/project/102/datafiles/').json()
for i in data['results']:
    if i['id'] ==  9135000:
        fitbit = requests.get(i['download_url']).json()
'''
In [2]:
DateTime = []

You can choose the year you want to analyze.

In [3]:
year = '2018'
In [4]:
def GetData(obj,year,key):
    name = key.split('-')[-1]
    name = []
    for day in fitbit[obj][year][key]:
        name.append(day['value'])
    return name

for day in fitbit['minutes-to-sleep'][year]['sleep-minutesToFallAsleep']:
    DateTime.append(datetime.strptime(day['dateTime'],'%Y-%m-%d'))
    

    
sleepData = pd.DataFrame(
    data = {
        'minutesToFallAsleep': GetData('minutes-to-sleep',year,'sleep-minutesToFallAsleep'),
        'minutesAfterWakeup': GetData('sleep-minutes-after-wakeup',year,'sleep-minutesAfterWakeup'),
        'startTime': GetData('sleep-start-time',year,'sleep-startTime'),
        'minutesAwake': GetData('awake-minutes',year,'sleep-minutesAwake'),
        'timeInBed': GetData('time-in-bed',year,'sleep-timeInBed'),
        'awakeningsCount': GetData('sleep-awakenings',year,'sleep-awakeningsCount'),
        'sleepEfficiency': GetData('sleep-efficiency',year,'sleep-efficiency'),
        'minutesAsleep':GetData('sleep-minutes',year,'sleep-minutesAsleep')
        
    }
)
In [5]:
DateTime = pd.DataFrame(DateTime)
In [6]:
sleepData = pd.concat([DateTime, sleepData], axis=1)
sleepData.rename(columns={sleepData.columns[0]: 'DateTime'}, inplace=True)
sleepData = pd.DataFrame(sleepData.set_index('DateTime', drop= False))

There might be some days when your device failed to collect your data. Let's remove the dates that data were not collected.

In [7]:
col = ['minutesToFallAsleep','minutesAfterWakeup','timeInBed','awakeningsCount','minutesAsleep','sleepEfficiency']
sleepData[col] = sleepData[col].mask(sleepData[col].applymap(str).eq('0'))

First, by calculatting the average sleep hours and the standard deviation, you can know whether you are getting the required hours of sleep?

In [8]:
import matplotlib.dates as mdates

sleepData['timeInBed'] = sleepData['timeInBed'].astype(float)
sleepData['hourInBed'] = sleepData['timeInBed']/60
sleepDesc = pd.DataFrame(sleepData['hourInBed'].describe())
sleepDesc
Out[8]:
hourInBed
count 352.000000
mean 7.324337
std 1.035574
min 2.466667
25% 6.795833
50% 7.416667
75% 7.916667
max 10.233333

The value of mean tells you your average sleeping hours and the standard deviation can tell your how your sleeping data spread out. The smaller the standard deviation is, the more centralized your data is. Then, let's use a bar chart to explore the distribution of your sleeping hours.

In [9]:
avgSleepHours = round(sleepDesc['hourInBed']['mean'],2)
summary = 'Averaging a sleep of {} hours with a standardized deviation of {} hours'.format(avgSleepHours, round(sleepDesc['hourInBed']['std'],2))

fig, ax = plt.subplots(figsize = (20,6)) 
plt.hist(sleepData['hourInBed'], bins = 8, range = (3, 10), color="navy")
plt.xlim(3, 10)
plt.xticks(range(3, 10))
plt.axvline(avgSleepHours, color="orangered", linestyle='--')
plt.xlabel('Hours in Bed')
plt.ylabel('Count');
plt.title('Figure 1. Bar chart of Hours in Bed', fontsize=15)
bbox_props1 = dict(boxstyle='round4, pad=0.6', fc='cyan', ec='b', lw=.5)
ax.annotate(summary,             
            fontsize=12,
            fontweight='demi',
            xy=(avgSleepHours, 120),  
            xycoords='data',
            xytext=(10, -30),      
            textcoords='offset points',
            arrowprops=dict(arrowstyle="simple",), bbox=bbox_props1)
plt.show()
/opt/conda/lib/python3.6/site-packages/numpy/lib/function_base.py:748: RuntimeWarning: invalid value encountered in greater_equal
  keep = (tmp_a >= mn)
/opt/conda/lib/python3.6/site-packages/numpy/lib/function_base.py:749: RuntimeWarning: invalid value encountered in less_equal
  keep &= (tmp_a <= mx)

From the bar chart above, you can learn about how many hours you sleep most of the time. Let's also plot your sleeping hours through the whole year with the average line.

In [10]:
fig, ax = plt.subplots(figsize = (20,6))
plt.plot(sleepData['DateTime'],sleepData['hourInBed'], linestyle='-', 
         markersize=10, color='c', label='% Light', linewidth=3.0, alpha=0.9)
plt.ylabel('hourInBed', fontsize=14)
plt.axhline(avgSleepHours, color="orangered", linestyle='--')
plt.title('Figure 2. Line plot of Hours in Bed over {}'.format(year), fontsize = 15)
ax.annotate
ax.xaxis.set_major_locator(mdates.WeekdayLocator(byweekday=6))
ax.xaxis.set_major_formatter(mdates.DateFormatter('%D'))
ax.grid(True)
plt.xticks(rotation=90)
plt.plot()
plt.show()

When you're in bed, you're not asleep all the time. Let's take a look at the sleep pattern while you're in bed.

In [11]:
SleepPattern = sleepData[['minutesAwake','minutesAsleep']].astype(float)
avgSleep = SleepPattern.mean()
In [12]:
fig = plt.figure(figsize = (6,6))
labels=['Awake', 'Asleep']
plt.pie(avgSleep, colors = ['lightskyblue', 'yellowgreen'], autopct='%1.1f%%', labels=labels, textprops=dict(color="w"))


my_circle=plt.Circle( (0,0), 0.7, color='white')
p=plt.gcf()
p.gca().add_artist(my_circle)

plt.title('Figure 3. Average of types of sleep over {}'.format(year), fontsize=15)
plt.legend()
plt.show()

Now, let's take a look at whether there is any pattern of your sleep based on different days of a week.

In [13]:
sleepData['Day of week'] = sleepData.index.weekday_name
weekday = ['Monday','Tuesday','Wednesday','Thursday','Friday']
weekend = ['Saturday','Sunday']
sleepData.loc[sleepData['Day of week'].isin(weekday), 'IsWeekday'] = 'Weekday'
sleepData.loc[sleepData['Day of week'].isin(weekend), 'IsWeekday'] = 'Weekend'
In [14]:
sleepData[['minutesAsleep', 'minutesAwake', 'sleepEfficiency']] = sleepData[['minutesAsleep', 'minutesAwake', 'sleepEfficiency']].astype(float)
dayGroupedData = sleepData.groupby(['Day of week']).mean()
dayGroupedData
Out[14]:
minutesAwake timeInBed sleepEfficiency minutesAsleep hourInBed
Day of week
Friday 54.576923 401.580000 86.200000 344.820000 6.693000
Monday 66.566038 444.596154 84.788462 376.750000 7.409936
Saturday 64.326923 468.040816 85.530612 399.693878 7.800680
Sunday 64.134615 458.220000 85.480000 390.300000 7.637000
Thursday 58.403846 437.820000 86.100000 376.960000 7.297000
Tuesday 64.538462 452.823529 85.450980 386.666667 7.547059
Wednesday 59.865385 413.240000 85.220000 350.920000 6.887333
In [15]:
dayofweek = ['Mon','Tue','Wed','Thu','Fri','Sat','Sun']
fig = plt.figure(figsize = (10,5))
plt.bar((dayGroupedData.index), dayGroupedData['minutesAwake'],  width = 0.4, color='slateblue', label='Minutes Awake')
plt.bar((dayGroupedData.index), dayGroupedData['minutesAsleep'], width = 0.4, color='dodgerblue', label='Minutes Asleep', bottom=dayGroupedData['minutesAwake'])
plt.bar((dayGroupedData.index), dayGroupedData['timeInBed'],width = 0.4, color='lightskyblue', label='Time in Bed', bottom=dayGroupedData['minutesAsleep'])
plt.xticks(np.arange(len(dayofweek)),dayofweek)
plt.title('Figure 4. Wake/Asleep Minutes While in Bed over One Week ', fontsize = 15)
plt.legend(loc='upper right')
plt.show()
In [16]:
fig, ax = plt.subplots(figsize=(10, 5))
ax.plot('Day of week', 'timeInBed', data=sleepData, marker='o', markersize=8, linestyle='None', color ='salmon')
dayGroupedData = dayGroupedData.reset_index(drop=False)
ax.plot(dayGroupedData['timeInBed'],color='r', linewidth=2, label='Average Time in Bed' )
ax.set_ylabel('Time in Bed')
ax.set_xlabel('Day of Week')
plt.title('Figure 5. Run Chart of Minutes in Bed with Average value by Day of the Week', fontsize = 15)
plt.xticks(np.arange(len(dayofweek)),dayofweek)
plt.legend()
plt.show()

Let's check out how your sleep varies for weekdays vs weekends.

In [17]:
sleepData.groupby('IsWeekday').mean()
Out[17]:
minutesAwake timeInBed sleepEfficiency minutesAsleep hourInBed
IsWeekday
Weekday 60.812261 430.217391 85.545455 367.375494 7.170290
Weekend 64.230769 463.080808 85.505051 394.949495 7.718013
In [18]:
fig, ((ax1, ax2), (ax3, ax4)) = plt.subplots(ncols=2,nrows=2,figsize=(10, 10))
xticks = ['Weekday','Weekend']
sleepData.groupby('IsWeekday')['timeInBed'].mean().plot(kind='bar', ax = ax1)
ax1.set_title('Comparison of Time in Bed')
ax1.set_xticklabels(xticks, rotation = 0, ha="right")
ax1.xaxis.label.set_visible(False)
sleepData.groupby('IsWeekday')['minutesAsleep'].mean().plot(kind='bar', ax = ax2)
ax2.set_title('Comparison of Minutes Asleep')
ax2.set_xticklabels(xticks, rotation = 0, ha="right")
ax2.xaxis.label.set_visible(False)
sleepData.groupby('IsWeekday')['minutesAwake'].mean().plot(kind='bar',  ax = ax3)
ax3.set_title('Comparison of Minutes Awake')
ax3.set_xticklabels(xticks,rotation = 0, ha="right")
ax3.xaxis.label.set_visible(False)
sleepData.groupby('IsWeekday')['sleepEfficiency'].mean().plot(kind='bar', ax = ax4)
ax4.set_title('Comparison of Sleep Efficiency')
ax4.set_xticklabels(xticks,rotation = 0, ha="right")
ax4.xaxis.label.set_visible(False)

plt.suptitle('Figure 6. Sleep Comparison Between Weekday and Weekend')

plt.show()

Camparing the boxes above, you can learn about how your sleep varies during a week. And the change of seasons may affect your sleep,too. Let's how your sleep changes over month in one year.

In [19]:
sleepData['Month'] = sleepData.index.month
In [20]:
fig, axes = plt.subplots(3, 1, figsize=(12,10), sharex=False)
Month = ['Jan','Feb','Mar','Apr','May','Jun','Jul','Aug','Sep','Oct','Nov','Dec']
for name, ax in zip(['timeInBed','minutesAsleep', 'minutesAwake','sleepEfficiency'], axes):
    sns.boxplot(data=sleepData, x='Month', y=name, ax=ax)
    ax.set_ylabel('Minutes')
    ax.set_title(name)
    ax.set_xticklabels(Month)
    if ax != axes[-1]:
        ax.set_xlabel('')
plt.suptitle('Figure 7. Sleep Change over Year {}'.format(year))
plt.show()
/opt/conda/lib/python3.6/site-packages/seaborn/categorical.py:454: FutureWarning: remove_na is deprecated and is a private function. Do not use.
  box_data = remove_na(group_data)
/opt/conda/lib/python3.6/site-packages/seaborn/categorical.py:454: FutureWarning: remove_na is deprecated and is a private function. Do not use.
  box_data = remove_na(group_data)
/opt/conda/lib/python3.6/site-packages/seaborn/categorical.py:454: FutureWarning: remove_na is deprecated and is a private function. Do not use.
  box_data = remove_na(group_data)
/opt/conda/lib/python3.6/site-packages/seaborn/categorical.py:454: FutureWarning: remove_na is deprecated and is a private function. Do not use.
  box_data = remove_na(group_data)
/opt/conda/lib/python3.6/site-packages/seaborn/categorical.py:454: FutureWarning: remove_na is deprecated and is a private function. Do not use.
  box_data = remove_na(group_data)
/opt/conda/lib/python3.6/site-packages/seaborn/categorical.py:454: FutureWarning: remove_na is deprecated and is a private function. Do not use.
  box_data = remove_na(group_data)
/opt/conda/lib/python3.6/site-packages/seaborn/categorical.py:454: FutureWarning: remove_na is deprecated and is a private function. Do not use.
  box_data = remove_na(group_data)
/opt/conda/lib/python3.6/site-packages/seaborn/categorical.py:454: FutureWarning: remove_na is deprecated and is a private function. Do not use.
  box_data = remove_na(group_data)
/opt/conda/lib/python3.6/site-packages/seaborn/categorical.py:454: FutureWarning: remove_na is deprecated and is a private function. Do not use.
  box_data = remove_na(group_data)
/opt/conda/lib/python3.6/site-packages/seaborn/categorical.py:454: FutureWarning: remove_na is deprecated and is a private function. Do not use.
  box_data = remove_na(group_data)
/opt/conda/lib/python3.6/site-packages/seaborn/categorical.py:454: FutureWarning: remove_na is deprecated and is a private function. Do not use.
  box_data = remove_na(group_data)
/opt/conda/lib/python3.6/site-packages/seaborn/categorical.py:454: FutureWarning: remove_na is deprecated and is a private function. Do not use.
  box_data = remove_na(group_data)
/opt/conda/lib/python3.6/site-packages/seaborn/categorical.py:454: FutureWarning: remove_na is deprecated and is a private function. Do not use.
  box_data = remove_na(group_data)
/opt/conda/lib/python3.6/site-packages/seaborn/categorical.py:454: FutureWarning: remove_na is deprecated and is a private function. Do not use.
  box_data = remove_na(group_data)
/opt/conda/lib/python3.6/site-packages/seaborn/categorical.py:454: FutureWarning: remove_na is deprecated and is a private function. Do not use.
  box_data = remove_na(group_data)
/opt/conda/lib/python3.6/site-packages/seaborn/categorical.py:454: FutureWarning: remove_na is deprecated and is a private function. Do not use.
  box_data = remove_na(group_data)
/opt/conda/lib/python3.6/site-packages/seaborn/categorical.py:454: FutureWarning: remove_na is deprecated and is a private function. Do not use.
  box_data = remove_na(group_data)
/opt/conda/lib/python3.6/site-packages/seaborn/categorical.py:454: FutureWarning: remove_na is deprecated and is a private function. Do not use.
  box_data = remove_na(group_data)
/opt/conda/lib/python3.6/site-packages/seaborn/categorical.py:454: FutureWarning: remove_na is deprecated and is a private function. Do not use.
  box_data = remove_na(group_data)
/opt/conda/lib/python3.6/site-packages/seaborn/categorical.py:454: FutureWarning: remove_na is deprecated and is a private function. Do not use.
  box_data = remove_na(group_data)
/opt/conda/lib/python3.6/site-packages/seaborn/categorical.py:454: FutureWarning: remove_na is deprecated and is a private function. Do not use.
  box_data = remove_na(group_data)
/opt/conda/lib/python3.6/site-packages/seaborn/categorical.py:454: FutureWarning: remove_na is deprecated and is a private function. Do not use.
  box_data = remove_na(group_data)
/opt/conda/lib/python3.6/site-packages/seaborn/categorical.py:454: FutureWarning: remove_na is deprecated and is a private function. Do not use.
  box_data = remove_na(group_data)
/opt/conda/lib/python3.6/site-packages/seaborn/categorical.py:454: FutureWarning: remove_na is deprecated and is a private function. Do not use.
  box_data = remove_na(group_data)
/opt/conda/lib/python3.6/site-packages/seaborn/categorical.py:454: FutureWarning: remove_na is deprecated and is a private function. Do not use.
  box_data = remove_na(group_data)
/opt/conda/lib/python3.6/site-packages/seaborn/categorical.py:454: FutureWarning: remove_na is deprecated and is a private function. Do not use.
  box_data = remove_na(group_data)
/opt/conda/lib/python3.6/site-packages/seaborn/categorical.py:454: FutureWarning: remove_na is deprecated and is a private function. Do not use.
  box_data = remove_na(group_data)
/opt/conda/lib/python3.6/site-packages/seaborn/categorical.py:454: FutureWarning: remove_na is deprecated and is a private function. Do not use.
  box_data = remove_na(group_data)
/opt/conda/lib/python3.6/site-packages/seaborn/categorical.py:454: FutureWarning: remove_na is deprecated and is a private function. Do not use.
  box_data = remove_na(group_data)
/opt/conda/lib/python3.6/site-packages/seaborn/categorical.py:454: FutureWarning: remove_na is deprecated and is a private function. Do not use.
  box_data = remove_na(group_data)
/opt/conda/lib/python3.6/site-packages/seaborn/categorical.py:454: FutureWarning: remove_na is deprecated and is a private function. Do not use.
  box_data = remove_na(group_data)
/opt/conda/lib/python3.6/site-packages/seaborn/categorical.py:454: FutureWarning: remove_na is deprecated and is a private function. Do not use.
  box_data = remove_na(group_data)
/opt/conda/lib/python3.6/site-packages/seaborn/categorical.py:454: FutureWarning: remove_na is deprecated and is a private function. Do not use.
  box_data = remove_na(group_data)
/opt/conda/lib/python3.6/site-packages/seaborn/categorical.py:454: FutureWarning: remove_na is deprecated and is a private function. Do not use.
  box_data = remove_na(group_data)
/opt/conda/lib/python3.6/site-packages/seaborn/categorical.py:454: FutureWarning: remove_na is deprecated and is a private function. Do not use.
  box_data = remove_na(group_data)
/opt/conda/lib/python3.6/site-packages/seaborn/categorical.py:454: FutureWarning: remove_na is deprecated and is a private function. Do not use.
  box_data = remove_na(group_data)
In [ ]:

In [ ]: