Details for plot-overland.ipynb

Published by gedankenstuecke

Description

Use GPS data collected with Overland on an iOS device and plot it on a map. The notebook demonstrates both how to use a full data set and make a global map and gives examples on how to focus on certain regions.

0

Tags & Data Sources

GPS location data location movement maps mapping Overland connection

Comments

Please log in to comment.

Notebook
Last updated 5 months ago

Analyzing your GPS data from Overland

This notebook requires you to use the Overland connection to get your GPS data from your iPhone into Open Humans.

This notebook then uses the GPS data to plot your personal movement history using R.

For a start let's load our required packages:

In [1]:
library(httr)
library(jsonlite)
library(ggplot2)
library(devtools)

With that out of the way we can access our Overland data from our Open Humans account. As the GPS records are can grow pretty large, each Year-Month will get it's own file. The code below will randomly use one of your files.

If you want to adjust it to a specific month you can change overland-data to something like overland-data-2018-12 following the overland-data-YYYY-MM format.

In [2]:
access_token <- Sys.getenv("OH_ACCESS_TOKEN")
url <- paste("https://www.openhumans.org/api/direct-sharing/project/exchange-member/?access_token=",access_token,sep="")
resp <- GET(url)
user <- content(resp, "parsed")

for (data_source in user$data){
    if (data_source$source == "direct-sharing-186" & grepl('overland-data', data_source$basename)){
         gps_data_url <- data_source$download_url
    }
}

temp <- tempfile()
download.file(gps_data_url,temp,method='wget')
json_data <- fromJSON(txt=temp)

Getting started with the data processing

Now that we have our fitbit data stored in json_data we can start to work with that data. Much of this notebook is adapted code from Shirin Glander's excellent blogpost, which is based on Google Location History data but the same principles apply for our Overland data.

Let's read the location data in to the loc variable along with the velocity and the date:

In [3]:
# extracting the locations dataframe
loc = json_data$geometry
properties = json_data$properties
loc$velocity <- properties$speed
loc$date <- properties$timestamp

Now we need to convert the coordinates as they are still in a weird format of having both latitude & longitude in one list. We'll fix this:

In [4]:
library(purrr)
loc$lon <- map(loc$coordinates,1)
loc$lat <- map(loc$coordinates,2)

loc$lat <- as.numeric(loc$lat)
loc$lon <- as.numeric(loc$lon)
Attaching package: ‘purrr’

The following object is masked from ‘package:jsonlite’:

    flatten

Analyzing the data

Let's install the required ggmap package and load some more data visualization tools:

In [5]:
install.packages('ggmap')
library(lubridate)
library(zoo)
# set up plotting theme
library(ggplot2)
library(ggmap)

my_theme <- function(base_size = 12, base_family = "sans"){
  theme_grey(base_size = base_size, base_family = base_family) +
  theme(
    axis.text = element_text(size = 12),
    axis.text.x = element_text(angle = 90, vjust = 0.5, hjust = 1),
    axis.title = element_text(size = 14),
    panel.grid.major = element_line(color = "grey"),
    panel.grid.minor = element_blank(),
    panel.background = element_rect(fill = "aliceblue"),
    strip.background = element_rect(fill = "lightgrey", color = "grey", size = 1),
    strip.text = element_text(face = "bold", size = 12, color = "navy"),
    legend.position = "right",
    legend.background = element_blank(),
    panel.margin = unit(.5, "lines"),
    panel.border = element_rect(color = "grey", fill = NA, size = 0.5)
  )
}
Updating HTML index of packages in '.Library'
Making 'packages.html' ... done

Attaching package: ‘lubridate’

The following object is masked from ‘package:base’:

    date


Attaching package: ‘zoo’

The following objects are masked from ‘package:base’:

    as.Date, as.Date.numeric

Let's make a map

The global view

We will start by looking at a map that will include all GPS points recorded for the data set you are looking at:

Below we define our boundaries that we want to plot by giving the minimum and maximum latitude/longitude of the map we want to see. By default we use the minimum/maximum values observed in our data.

In [6]:
boundary_west=min(loc$lon)
boundary_east=max(loc$lon)
boundary_south=min(loc$lat)
boundary_north=max(loc$lat)
loc_2 <- loc[which(!is.na(loc$velocity)), ]

my_map <- get_map(location=c(boundary_west,
                               boundary_south,
                               boundary_east,
                               boundary_north), source = "stamen",maptype='toner')
Map from URL : http://tile.stamen.com/toner/13/1310/3161.png
Map from URL : http://tile.stamen.com/toner/13/1311/3161.png
Map from URL : http://tile.stamen.com/toner/13/1312/3161.png
Map from URL : http://tile.stamen.com/toner/13/1313/3161.png
Map from URL : http://tile.stamen.com/toner/13/1310/3162.png
Map from URL : http://tile.stamen.com/toner/13/1311/3162.png
Map from URL : http://tile.stamen.com/toner/13/1312/3162.png
Map from URL : http://tile.stamen.com/toner/13/1313/3162.png
Map from URL : http://tile.stamen.com/toner/13/1310/3163.png
Map from URL : http://tile.stamen.com/toner/13/1311/3163.png
Map from URL : http://tile.stamen.com/toner/13/1312/3163.png
Map from URL : http://tile.stamen.com/toner/13/1313/3163.png
Map from URL : http://tile.stamen.com/toner/13/1310/3164.png
Map from URL : http://tile.stamen.com/toner/13/1311/3164.png
Map from URL : http://tile.stamen.com/toner/13/1312/3164.png
Map from URL : http://tile.stamen.com/toner/13/1313/3164.png
Map from URL : http://tile.stamen.com/toner/13/1310/3165.png
Map from URL : http://tile.stamen.com/toner/13/1311/3165.png
Map from URL : http://tile.stamen.com/toner/13/1312/3165.png
Map from URL : http://tile.stamen.com/toner/13/1313/3165.png
Map from URL : http://tile.stamen.com/toner/13/1310/3166.png
Map from URL : http://tile.stamen.com/toner/13/1311/3166.png
Map from URL : http://tile.stamen.com/toner/13/1312/3166.png
Map from URL : http://tile.stamen.com/toner/13/1313/3166.png
Map from URL : http://tile.stamen.com/toner/13/1310/3167.png
Map from URL : http://tile.stamen.com/toner/13/1311/3167.png
Map from URL : http://tile.stamen.com/toner/13/1312/3167.png
Map from URL : http://tile.stamen.com/toner/13/1313/3167.png
In [7]:
ggmap(my_map) + 
    geom_point(data = subset(loc_2, as.Date(loc_2$date) > as.Date('2018-10-10')), 
                aes(x = lon, 
                    y = lat, 
                    color = velocity/1000*60*60,
                    alpha=(velocity/1000*60*60)/max(loc_2$velocity/1000*60*60)),size=0.7) + 
  theme(legend.position = "right") + 
  labs(x = "Longitude", y = "Latitude", 
       title = "My Location History",
       subtitle = "Color scale shows velocity measured for location") +
  scale_colour_gradient(low = "blue", high = "red", guide = guide_legend(title = "Velocity")) + 
    scale_alpha_continuous('Velocity')

As I haven't traveled a lot in this month, the view is focussed pretty well on the Bay Area. But with a different data set the view might also look like this below. First we get another data set, this time for October 2018, where I travelled a lot:

In [8]:
access_token <- Sys.getenv("OH_ACCESS_TOKEN")
url <- paste("https://www.openhumans.org/api/direct-sharing/project/exchange-member/?access_token=",access_token,sep="")
resp <- GET(url)
user <- content(resp, "parsed")

for (data_source in user$data){
    if (data_source$source == "direct-sharing-186" & grepl('overland-data-2018-10', data_source$basename)){
         gps_data_url <- data_source$download_url
    }
}

temp <- tempfile()
download.file(gps_data_url,temp,method='wget')
json_data <- fromJSON(txt=temp)
# extracting the locations dataframe
loc = json_data$geometry
properties = json_data$properties
loc$velocity <- properties$speed
loc$date <- properties$timestamp

loc$lon <- map(loc$coordinates,1)
loc$lat <- map(loc$coordinates,2)

loc$lat <- as.numeric(loc$lat)
loc$lon <- as.numeric(loc$lon)

boundary_west=min(loc$lon)
boundary_east=max(loc$lon)
boundary_south=min(loc$lat)
boundary_north=max(loc$lat)
loc_2 <- loc[which(!is.na(loc$velocity)), ]

my_map <- get_map(location=c(boundary_west,
                               boundary_south,
                               boundary_east,
                               boundary_north), source = "stamen",maptype='toner')
Map from URL : http://tile.stamen.com/toner/5/5/10.png
Map from URL : http://tile.stamen.com/toner/5/6/10.png
Map from URL : http://tile.stamen.com/toner/5/7/10.png
Map from URL : http://tile.stamen.com/toner/5/8/10.png
Map from URL : http://tile.stamen.com/toner/5/9/10.png
Map from URL : http://tile.stamen.com/toner/5/10/10.png
Map from URL : http://tile.stamen.com/toner/5/11/10.png
Map from URL : http://tile.stamen.com/toner/5/12/10.png
Map from URL : http://tile.stamen.com/toner/5/13/10.png
Map from URL : http://tile.stamen.com/toner/5/14/10.png
Map from URL : http://tile.stamen.com/toner/5/15/10.png
Map from URL : http://tile.stamen.com/toner/5/16/10.png
Map from URL : http://tile.stamen.com/toner/5/5/11.png
Map from URL : http://tile.stamen.com/toner/5/6/11.png
Map from URL : http://tile.stamen.com/toner/5/7/11.png
Map from URL : http://tile.stamen.com/toner/5/8/11.png
Map from URL : http://tile.stamen.com/toner/5/9/11.png
Map from URL : http://tile.stamen.com/toner/5/10/11.png
Map from URL : http://tile.stamen.com/toner/5/11/11.png
Map from URL : http://tile.stamen.com/toner/5/12/11.png
Map from URL : http://tile.stamen.com/toner/5/13/11.png
Map from URL : http://tile.stamen.com/toner/5/14/11.png
Map from URL : http://tile.stamen.com/toner/5/15/11.png
Map from URL : http://tile.stamen.com/toner/5/16/11.png
Map from URL : http://tile.stamen.com/toner/5/5/12.png
Map from URL : http://tile.stamen.com/toner/5/6/12.png
Map from URL : http://tile.stamen.com/toner/5/7/12.png
Map from URL : http://tile.stamen.com/toner/5/8/12.png
Map from URL : http://tile.stamen.com/toner/5/9/12.png
Map from URL : http://tile.stamen.com/toner/5/10/12.png
Map from URL : http://tile.stamen.com/toner/5/11/12.png
Map from URL : http://tile.stamen.com/toner/5/12/12.png
Map from URL : http://tile.stamen.com/toner/5/13/12.png
Map from URL : http://tile.stamen.com/toner/5/14/12.png
Map from URL : http://tile.stamen.com/toner/5/15/12.png
Map from URL : http://tile.stamen.com/toner/5/16/12.png

And now we can plot that data:

In [9]:
ggmap(my_map) + 
    geom_point(data = subset(loc_2, as.Date(loc_2$date) > as.Date('2018-10-10')), 
                aes(x = lon, 
                    y = lat, 
                    color = velocity/1000*60*60,
                    alpha=(velocity/1000*60*60)/max(loc_2$velocity/1000*60*60)),size=0.7) + 
  theme(legend.position = "right") + 
  labs(x = "Longitude", y = "Latitude", 
       title = "My Location History",
       subtitle = "Color scale shows velocity measured for location") +
  scale_colour_gradient(low = "blue", high = "red", guide = guide_legend(title = "Velocity")) + 
    scale_alpha_continuous('Velocity')

We see that this map is a lot less useful, as it covers not only my home locations around the Bay Area, but also Los Angeles and London. To zoom in we can adjust our map boundaries manually by editing the numbers below. One easy way to find those is by asking Google for e.g. Coordinates London. This will deliver you the Latitude & Longitude.

For London I entered the following. Adapt these numbers to the place you are interest in!

In [10]:
boundary_west=-0.15
boundary_east=0.015 # for those I wouldn't have needed to ask Google thanks to the Prime Meridian in Greenwich :D
boundary_south=51.47074
boundary_north=51.5274

my_map <- get_map(location=c(boundary_west,
                               boundary_south,
                               boundary_east,
                               boundary_north), source = "stamen",maptype='toner')
Map from URL : http://tile.stamen.com/toner/13/4092/2723.png
Map from URL : http://tile.stamen.com/toner/13/4093/2723.png
Map from URL : http://tile.stamen.com/toner/13/4094/2723.png
Map from URL : http://tile.stamen.com/toner/13/4095/2723.png
Map from URL : http://tile.stamen.com/toner/13/4096/2723.png
Map from URL : http://tile.stamen.com/toner/13/4092/2724.png
Map from URL : http://tile.stamen.com/toner/13/4093/2724.png
Map from URL : http://tile.stamen.com/toner/13/4094/2724.png
Map from URL : http://tile.stamen.com/toner/13/4095/2724.png
Map from URL : http://tile.stamen.com/toner/13/4096/2724.png
Map from URL : http://tile.stamen.com/toner/13/4092/2725.png
Map from URL : http://tile.stamen.com/toner/13/4093/2725.png
Map from URL : http://tile.stamen.com/toner/13/4094/2725.png
Map from URL : http://tile.stamen.com/toner/13/4095/2725.png
Map from URL : http://tile.stamen.com/toner/13/4096/2725.png

With that out of the way we can plot the map again:

In [11]:
ggmap(my_map) + 
    geom_point(data = subset(loc_2, as.Date(loc_2$date) > as.Date('2018-10-10')), 
                aes(x = lon, 
                    y = lat, 
                    color = velocity/1000*60*60,
                    alpha=(velocity/1000*60*60)/max(loc_2$velocity/1000*60*60)),size=0.7) + 
  theme(legend.position = "right") + 
  labs(x = "Longitude", y = "Latitude", 
       title = "My Location History",
       subtitle = "Color scale shows velocity measured for location") +
  scale_colour_gradient(low = "blue", high = "red", guide = guide_legend(title = "Velocity")) + 
    scale_alpha_continuous('Velocity')
Warning message:
“Removed 60174 rows containing missing values (geom_point).”

This makes for a much more useful map, centered around my trip at London. Getting the Latitude / Longitude parameters in the cells above right can take some experimentation, to zoom in on the right point of the map. Give it a try!

In [ ]: