Details for plot-overland-paris.ipynb

Published by gedankenstuecke

Description

A GPS visualization of Paris for Overland data

0

Tags & Data Sources

overland GPS paris Overland connection

Comments

Please log in to comment.

Notebook
Last updated 3 years, 12 months ago

Analyzing your GPS data from Overland

This notebook requires you to use the Overland connection to get your GPS data from your iPhone into Open Humans.

This notebook then uses the GPS data to plot your personal movement history using R.

For a start let's load our required packages:

Updating HTML index of packages in '.Library'
Making 'packages.html' ... done

Attaching package: ‘purrr’

The following object is masked from ‘package:jsonlite’:

    flatten

With that out of the way we can access our Overland data from our Open Humans account. As the GPS records are can grow pretty large, each Year-Month will get it's own file. The code below will randomly use one of your files.

If you want to adjust it to a specific month you can change overland-data to something like overland-data-2018-12 following the overland-data-YYYY-MM format. By default it uses 2019-06 as the data points

Now we can start downloading the data:

[1] "read first month"
longitudelatitudeactivityaltitudebattery_levelbattery_statedeferreddesired_accuracyhorizontal_accuracymotionpausessignificant_changespeedtimestampvertical_accuracywifivelocitydatelonlat
2.397232 48.85328 other 55 0.49 unplugged 0 100 65 stationary False 0 -1 2020-03-01T00:02:55Z10 -1 2020-03-01T00:02:55Z2.397232 48.85328
2.397190 48.85325 other 55 0.49 unplugged 0 100 65 stationary False 0 -1 2020-03-01T00:03:03Z10 -1 2020-03-01T00:03:03Z2.397190 48.85325
2.397235 48.85334 other 55 0.49 unplugged 0 100 65 stationary False 0 -1 2020-03-01T00:03:14Z10 -1 2020-03-01T00:03:14Z2.397235 48.85334
2.397191 48.85333 other 55 0.49 unplugged 0 100 65 False 0 -1 2020-03-01T00:03:22Z10 -1 2020-03-01T00:03:22Z2.397191 48.85333
2.397147 48.85335 other 55 0.49 unplugged 0 100 65 stationary False 0 -1 2020-03-01T00:03:30Z10 -1 2020-03-01T00:03:30Z2.397147 48.85335
2.397289 48.85334 other 55 0.49 unplugged 0 100 65 stationary False 0 -1 2020-03-01T00:03:44Z10 -1 2020-03-01T00:03:44Z2.397289 48.85334

Now we can look into how our data looks like:

longitudelatitudeactivityaltitudebattery_levelbattery_statedeferreddesired_accuracyhorizontal_accuracymotionpausessignificant_changespeedtimestampvertical_accuracywifivelocitydatelonlat
2.397232 48.85328 other 55 0.49 unplugged 0 100 65 stationary False 0 -1 2020-03-01T00:02:55Z10 -1 2020-03-01T00:02:55Z2.397232 48.85328
2.397190 48.85325 other 55 0.49 unplugged 0 100 65 stationary False 0 -1 2020-03-01T00:03:03Z10 -1 2020-03-01T00:03:03Z2.397190 48.85325
2.397235 48.85334 other 55 0.49 unplugged 0 100 65 stationary False 0 -1 2020-03-01T00:03:14Z10 -1 2020-03-01T00:03:14Z2.397235 48.85334
2.397191 48.85333 other 55 0.49 unplugged 0 100 65 False 0 -1 2020-03-01T00:03:22Z10 -1 2020-03-01T00:03:22Z2.397191 48.85333
2.397147 48.85335 other 55 0.49 unplugged 0 100 65 stationary False 0 -1 2020-03-01T00:03:30Z10 -1 2020-03-01T00:03:30Z2.397147 48.85335
2.397289 48.85334 other 55 0.49 unplugged 0 100 65 stationary False 0 -1 2020-03-01T00:03:44Z10 -1 2020-03-01T00:03:44Z2.397289 48.85334

Getting started with the data processing

Now that we have our fitbit data stored in loc we can start to work with that data. Much of this notebook is adapted code from Shirin Glander's excellent blogpost, which is based on Google Location History data but the same principles apply for our Overland data.

Analyzing the data

Let's install the required ggmap package and load some more data visualization tools:

Attaching package: ‘lubridate’

The following object is masked from ‘package:base’:

    date


Attaching package: ‘zoo’

The following objects are masked from ‘package:base’:

    as.Date, as.Date.numeric

Google's Terms of Service: https://cloud.google.com/maps-platform/terms/.
Please cite ggmap if you use it! See citation("ggmap") for details.

Let's make a map

Source : http://tile.stamen.com/toner/13/4148/2816.png
Source : http://tile.stamen.com/toner/13/4149/2816.png
Source : http://tile.stamen.com/toner/13/4150/2816.png
Source : http://tile.stamen.com/toner/13/4148/2817.png
Source : http://tile.stamen.com/toner/13/4149/2817.png
Source : http://tile.stamen.com/toner/13/4150/2817.png
Source : http://tile.stamen.com/toner/13/4148/2818.png
Source : http://tile.stamen.com/toner/13/4149/2818.png
Source : http://tile.stamen.com/toner/13/4150/2818.png
Source : http://tile.stamen.com/toner/13/4148/2819.png
Source : http://tile.stamen.com/toner/13/4149/2819.png
Source : http://tile.stamen.com/toner/13/4150/2819.png

And now we can plot that data:

Warning message:
“Removed 9407 rows containing missing values (geom_point).”

Notebook
Last updated 3 years, 12 months ago

Analyzing your GPS data from Overland

This notebook requires you to use the Overland connection to get your GPS data from your iPhone into Open Humans.

This notebook then uses the GPS data to plot your personal movement history using R.

For a start let's load our required packages:

In [1]:
library(httr)
library(jsonlite)
library(ggplot2)
library(devtools)
install.packages('ggmap')
library(purrr)
Updating HTML index of packages in '.Library'
Making 'packages.html' ... done

Attaching package: ‘purrr’

The following object is masked from ‘package:jsonlite’:

    flatten

With that out of the way we can access our Overland data from our Open Humans account. As the GPS records are can grow pretty large, each Year-Month will get it's own file. The code below will randomly use one of your files.

If you want to adjust it to a specific month you can change overland-data to something like overland-data-2018-12 following the overland-data-YYYY-MM format. By default it uses 2019-06 as the data points

In [2]:
month1 <- '2020-03'

Now we can start downloading the data:

In [3]:
access_token <- Sys.getenv("OH_ACCESS_TOKEN")
url <- paste("https://www.openhumans.org/api/direct-sharing/project/exchange-member/?access_token=",access_token,sep="")
resp <- GET(url)
user <- content(resp, "parsed")

user_data <- user$data

while (!is.null(user$`next`)) {
    resp <- GET(user$`next`)
    user <- content(resp, "parsed")
    user_data <- append(user_data, user$data)
}

for (data_source in user_data){
    if (grepl(paste('overland-data',month1,sep='-'), data_source$basename)){
        print('read first month')
        loc <- read.csv(url(data_source$download_url))
    }
}
loc$velocity <- loc$speed
loc$date <- loc$timestamp
loc$lon <- loc$longitude
loc$lat <- loc$latitude

head(loc)
[1] "read first month"
longitudelatitudeactivityaltitudebattery_levelbattery_statedeferreddesired_accuracyhorizontal_accuracymotionpausessignificant_changespeedtimestampvertical_accuracywifivelocitydatelonlat
2.397232 48.85328 other 55 0.49 unplugged 0 100 65 stationary False 0 -1 2020-03-01T00:02:55Z10 -1 2020-03-01T00:02:55Z2.397232 48.85328
2.397190 48.85325 other 55 0.49 unplugged 0 100 65 stationary False 0 -1 2020-03-01T00:03:03Z10 -1 2020-03-01T00:03:03Z2.397190 48.85325
2.397235 48.85334 other 55 0.49 unplugged 0 100 65 stationary False 0 -1 2020-03-01T00:03:14Z10 -1 2020-03-01T00:03:14Z2.397235 48.85334
2.397191 48.85333 other 55 0.49 unplugged 0 100 65 False 0 -1 2020-03-01T00:03:22Z10 -1 2020-03-01T00:03:22Z2.397191 48.85333
2.397147 48.85335 other 55 0.49 unplugged 0 100 65 stationary False 0 -1 2020-03-01T00:03:30Z10 -1 2020-03-01T00:03:30Z2.397147 48.85335
2.397289 48.85334 other 55 0.49 unplugged 0 100 65 stationary False 0 -1 2020-03-01T00:03:44Z10 -1 2020-03-01T00:03:44Z2.397289 48.85334

Now we can look into how our data looks like:

In [4]:
loc$velocity <- loc$speed
loc$date <- loc$timestamp
loc$lon <- loc$longitude
loc$lat <- loc$latitude

head(loc)
longitudelatitudeactivityaltitudebattery_levelbattery_statedeferreddesired_accuracyhorizontal_accuracymotionpausessignificant_changespeedtimestampvertical_accuracywifivelocitydatelonlat
2.397232 48.85328 other 55 0.49 unplugged 0 100 65 stationary False 0 -1 2020-03-01T00:02:55Z10 -1 2020-03-01T00:02:55Z2.397232 48.85328
2.397190 48.85325 other 55 0.49 unplugged 0 100 65 stationary False 0 -1 2020-03-01T00:03:03Z10 -1 2020-03-01T00:03:03Z2.397190 48.85325
2.397235 48.85334 other 55 0.49 unplugged 0 100 65 stationary False 0 -1 2020-03-01T00:03:14Z10 -1 2020-03-01T00:03:14Z2.397235 48.85334
2.397191 48.85333 other 55 0.49 unplugged 0 100 65 False 0 -1 2020-03-01T00:03:22Z10 -1 2020-03-01T00:03:22Z2.397191 48.85333
2.397147 48.85335 other 55 0.49 unplugged 0 100 65 stationary False 0 -1 2020-03-01T00:03:30Z10 -1 2020-03-01T00:03:30Z2.397147 48.85335
2.397289 48.85334 other 55 0.49 unplugged 0 100 65 stationary False 0 -1 2020-03-01T00:03:44Z10 -1 2020-03-01T00:03:44Z2.397289 48.85334

Getting started with the data processing

Now that we have our fitbit data stored in loc we can start to work with that data. Much of this notebook is adapted code from Shirin Glander's excellent blogpost, which is based on Google Location History data but the same principles apply for our Overland data.

Analyzing the data

Let's install the required ggmap package and load some more data visualization tools:

In [5]:
library(lubridate)
library(zoo)
# set up plotting theme
library(ggplot2)
library(ggmap)

my_theme <- function(base_size = 12, base_family = "sans"){
  theme_grey(base_size = base_size, base_family = base_family) +
  theme(
    axis.text = element_text(size = 12),
    axis.text.x = element_text(angle = 90, vjust = 0.5, hjust = 1),
    axis.title = element_text(size = 14),
    panel.grid.major = element_line(color = "grey"),
    panel.grid.minor = element_blank(),
    panel.background = element_rect(fill = "aliceblue"),
    strip.background = element_rect(fill = "lightgrey", color = "grey", size = 1),
    strip.text = element_text(face = "bold", size = 12, color = "navy"),
    legend.position = "right",
    legend.background = element_blank(),
    panel.margin = unit(.5, "lines"),
    panel.border = element_rect(color = "grey", fill = NA, size = 0.5)
  )
}
Attaching package: ‘lubridate’

The following object is masked from ‘package:base’:

    date


Attaching package: ‘zoo’

The following objects are masked from ‘package:base’:

    as.Date, as.Date.numeric

Google's Terms of Service: https://cloud.google.com/maps-platform/terms/.
Please cite ggmap if you use it! See citation("ggmap") for details.

Let's make a map

In [6]:
loc$weekday <- wday(loc$timestamp)
loc$weekend <- loc$weekday %in% c('Sunday','Saturday')
loc$weekend <- ifelse(loc$weekend, 'weekend', 'weekday')
loc$hour <- hour(loc$timestamp)
In [11]:
boundary_west=2.3
boundary_east=2.40 # for those I wouldn't have needed to ask Google thanks to the Prime Meridian in Greenwich :D
boundary_south=48.815 # 3
boundary_north=48.9

my_map <- get_stamenmap(bbox=c(boundary_west,
                               boundary_south,
                               boundary_east,
                               boundary_north),zoom=13,maptype='toner',force=TRUE)
Source : http://tile.stamen.com/toner/13/4148/2816.png
Source : http://tile.stamen.com/toner/13/4149/2816.png
Source : http://tile.stamen.com/toner/13/4150/2816.png
Source : http://tile.stamen.com/toner/13/4148/2817.png
Source : http://tile.stamen.com/toner/13/4149/2817.png
Source : http://tile.stamen.com/toner/13/4150/2817.png
Source : http://tile.stamen.com/toner/13/4148/2818.png
Source : http://tile.stamen.com/toner/13/4149/2818.png
Source : http://tile.stamen.com/toner/13/4150/2818.png
Source : http://tile.stamen.com/toner/13/4148/2819.png
Source : http://tile.stamen.com/toner/13/4149/2819.png
Source : http://tile.stamen.com/toner/13/4150/2819.png

And now we can plot that data:

In [29]:
library(repr)
options(repr.plot.width=10, repr.plot.height=10)
ggmap(my_map) + 
    geom_point(data = subset(loc, loc$motion %in% c('stationary', 'walking')), 
                    aes(x = lon, 
                        y = lat), alpha=0.6, size=0.5,color='red') +
  theme(legend.position = "right") + facet_grid(. ~ motion)
Warning message:
“Removed 9407 rows containing missing values (geom_point).”
In [ ]: