Recent changes in R spatial

class: inverse, center, nonum, clear
background-image: url("figs/cover.jpg")
background-size: cover

.titlestyle[Recent changes in R spatial]
 
.titlestyle[and]
 
.titlestyle[how to be ready for them]

<link rel="stylesheet" type="text/css" href="//fonts.googleapis.com/css?family=Limelight" />
.captionstyle[Jakub Nowosad and Robin Lovelace]

.captionstyle[geocompr.github.io | 2020-04-23]

---

# Contents

.lc[

## What's in the box?

#### R's spatial ecosystem

#### System dependencies (OSGeo Libraries)

#### Coodinate Systems (in PROJ 7)

#### Other changes

##### And then...

#### Computer set-up

#### Running #rspatial in Docker

#### #rspatial and the tidyverse

#### Contributing

]

.rc[
![](https://media0.giphy.com/media/KABPzeh1vQmdi/giphy.gif?cid=ecf05e4798cff6938fc77a0e12691568bee7e3842c0dd169&rid=giphy.gif)
]

---
# R spatial

.lc[
- Ecology
- Earth-Observation
- Economics
- Demography
- Politics
- Journalism
- Archeology
- Transport
- Climatology
- Meteorology
- Geomorphometry
- Hydrology
- Urban-Planning
- Mining
- Marine-Studies
- Soil-Science
- Tourism
- and many more...
]

.rc[
<a href="https://www.tylermw.com/3d-maps-with-rayshader/">
<img src="figs/montbayabove.gif" width="90%" style="display: block; margin: auto;" />
</a>
<a href="https://www.mdpi.com/2306-5729/4/3/92">
<img src="figs/water-detection.png" width="4092" style="display: block; margin: auto;" />
</a>
]

---
# R spatial

.pull-left[

- Spatial aspect can be important in any part of data analysis - data preparation, vizualization, modeling, or communicating the results
- It does not need to be the most important part
- R is great to integrate various aspects of analysis

]

.pull-right[
<a href="https://riatelab.github.io/MDM/">
<img src="figs/cartography-map.png" width="1600" style="display: block; margin: auto;" />
</a>
 
<a href="https://www.pct.bike/">
<img src="figs/pct-example2.png" width="1404" style="display: block; margin: auto;" />
</a>
]

---
# R spatial

.pull-left[
<a href="https://github.com/mtennekes/tmap/tree/master/demo/ClassicMap">
<img src="figs/tmap-map.png" width="2449" style="display: block; margin: auto;" />
</a>
<a href="https://github.com/Z3tt/30DayMapChallenge">
<img src="figs/Zones_TimezonesEarth.png" width="8400" style="display: block; margin: auto;" />
</a>
]

.pull-right[
- https://geocompr.github.io/
- https://www.rspatial.org/
- https://www.r-spatial.org/
- `#rspatial` and `#geocompr` on Twitter

<a href="https://nowosad.github.io/post/maps-distortion/">
<img src="figs/worlds_animate_without_antarctica2.gif" style="display: block; margin: auto;" />
</a>
]

---
# R spatial infrastructure

- **landscapemetrics**: work with categorical raster data

.lc[

```r
library(landscapemetrics)
library(landscapetools)
data("landscape")
show_landscape(landscape, 
             discrete = TRUE)
```

<img src="index_files/figure-html/unnamed-chunk-9-1.png" style="display: block; margin: auto;" />
]

.rc[

```r
calculate_lsm(landscape, level = "landscape")
```

```
## # A tibble: 65 x 6
## layer level class id metric value
## <int> <chr> <int> <int> <chr> <dbl>
## 1 1 landscape NA NA ai 81.1 
## 2 1 landscape NA NA area_cv 273. 
## 3 1 landscape NA NA area_mn 0.00333
## 4 1 landscape NA NA area_sd 0.00911
## 5 1 landscape NA NA cai_cv 171. 
## 6 1 landscape NA NA cai_mn 10.3 
## 7 1 landscape NA NA cai_sd 17.6 
## 8 1 landscape NA NA circle_cv 25.3 
## 9 1 landscape NA NA circle_mn 0.529 
## 10 1 landscape NA NA circle_sd 0.134 
## # … with 55 more rows
```

]

---
# R spatial infrastructure

- [**stplanr**](https://docs.ropensci.org/stplanr/): work with routes and route networks, also see in-development package [**od**](https://github.com/ITSLeeds/od)

.lc[

```r
library(stplanr)
routes_fast_sf$walking =
  flow$On.foot
route_net = overline(
  routes_fast_sf,
  "walking"
  )
plot(route_net["walking"])
```

<img src="index_files/figure-html/unnamed-chunk-11-1.png" style="display: block; margin: auto;" />
]

.rc[

```r
library(dplyr)
l = pct::get_pct_lines(region = "london")
r = route(l = l[l$all > 500, ], route_fun = cyclestreets::journey)
rnet_london = r %>% group_by(name, distances) %>%
  summarise(Cycling = sum(bicycle), Provision = first(provisionName))
tm_shape(rnet_london) + tm_lines("Provision", lwd = "Cycling")
```

]

---
# R spatial infrastructure

.pull-left[
- **landscapemetrics** is based on the raster representation of spatial data 
- **raster**/**terra**
- **stars**

<a href="https://geocompr.github.io/">
<img src="figs/raster-intro-plot2-1.png" width="1792" style="display: block; margin: auto;" />
</a>
]

.pull-right[
- **stplanr** uses the vector representation of spatial data
- **sp**/**sf**

<a href="https://geocompr.github.io/">
<img src="figs/sf-classes.png" width="80%" style="display: block; margin: auto;" />
</a>
]

---
# R spatial infrastructure

---
# OSGeo libraries
.lc[
### [PROJ](https://www.osgeo.org/projects/proj/)

- "conversion of coordinates"
- First versions in [1970s](https://en.wikipedia.org/wiki/PROJ#cite_note-Kresse_Danko_2011_p._948-3)

### [GDAL](https://www.osgeo.org/projects/gdal/)

- Geospatial Data Abstraction Library, first release 2000
- "200+ geospatial data formats"

### [GEOS](https://www.osgeo.org/projects/gdal/)

- Geometry Engine – Open Source
- "spatial predicate functions and spatial operators"
]

.rc[
<a href="https://www.osgeo.org/">
<img src="https://www.osgeo.org/wp-content/themes/roots/assets/img/logo-osgeo.svg" width="60%" style="display: block; margin: auto;" />
</a>

<div class="figure" style="text-align: center">
<img src="figs/GEOS.png" alt="GEOS build matrix" width="90%" />
GEOS build matrix
</div>

.caption[Source: https://trac.osgeo.org/geos]

- Access to these three libraries used to require **sp**, **rgdal** and **rgeos**
- Now **sf** provides a unified access to all three 🎉

]

---
# R spatial infrastructure changes

My (old) views:

---
# R spatial infrastructure changes

Reality (for users):

---

# Open source software evolves

Reality II (for developers software is constantly evolving and usually improving)

.lc[

![](https://media.giphy.com/media/xlK6RwN8nwyjK/giphy.gif)

In theory: an orderly process
]

.rc[

![](https://media.giphy.com/media/xT5LMULHHwE1Ayl7ji/giphy.gif)

In reality: a non-linear process

With 1000s of eyes on it, **open source software tends to outpace and have more supportive communities than proprietary software**

]

---
# Impact of the changes

.lc[

- Changes in **stplanr** impact its users
- Changes in **sf** impacts **stplanr**, which impacts its users
- Changes in PROJ impact **sf**, which impacts **stplanr**, which impacts its users

- Takes [months](https://twitter.com/robinlovelace/status/931514635412484096) + effort to support upstream developments
- Multiply that by 50+ packages...

]

.rc[

]

---
# The largest recent change - PROJ6+

.pull-left[

]

.pull-right[

- PROJ is a library for performing conversions between cartographic projections 
- The history of PROJ dates back to the late 1970s
- PROJ.4 was released in 1994 (version 4.4 in 2000)
- Small(er) changes until 2018
- https://proj.org/

]

---
# PROJ4 (before 2020) - proj4string

.pull-left[

```
+proj=longlat +ellps=WGS84 +datum=WGS84
    +no_defs +towgs84=0,0,0
```

<img src="index_files/figure-html/unnamed-chunk-32-1.png" style="display: block; margin: auto;" />
]

.pull-right[

```
+proj=utm +zone=12 +ellps=GRS80
    +towgs84=0,0,0,0,0,0,0 +units=m +no_defs
```

<img src="index_files/figure-html/unnamed-chunk-33-1.png" style="display: block; margin: auto;" />
]

---
# PROJ4 (before 2020)

.pull-left[
**proj4string**:

- String-based description of a coordinate reference system (CRS)
- It defines projection types and parameter values for particular projections
- Infinite amount of different projections
- For example `+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs +towgs84=0,0,0` or `+proj=utm +zone=12 +ellps=GRS80 +towgs84=0,0,0,0,0,0,0 +units=m +no_defs`
]

.pull-right[
**EPSG**:
- Code description of a coordinate reference system (CRS)
- Finite amount of different projections
- For example `4326` or `2956`

<img src="index_files/figure-html/unnamed-chunk-34-1.png" style="display: block; margin: auto;" />
]

---
# Issues with PROJ4

.lc[
- Static reference frames (no support for time-dependent datums; [on Earth, everything is moving, up to 80mm/year](https://www.openstreetmap.org/user/StephaneP/diary/390290))
- Always transformed first to WSG84 and from there to the target projection (accuracy issue)
- Limited geodetic functionality
- https://www.gaia-gis.it/fossil/libspatialite/wiki?name=PROJ.6
- https://github.com/OSGeo/PROJ/releases/tag/5.0.0

]

.rc[
<a href="https://www.ricsm.gov.au/australian-terrestrial-reference-frame">
<img src="figs/crustal-motion.jpg" width="100%" style="display: block; margin: auto;" />
</a>
]

---
# Changes in PROJ

.pull-left[
- PROJ5, PROJ6, and PROJ7 (GDAL3+)
- Full support for ISO 19162:2018 WKT (also known as **OGC WKT2**)
- Discourage (and limited) use of **proj4string**
- Direct transformations from CRS to CRS
- Capability of time-dependent transformations
- Transformation pipelines 
]

.pull-right[
More details:

- https://proj.org/news.html
- https://gdalbarn.com/
- https://info.crunchydata.com/blog/waiting-for-postgis-3-st_transform-and-proj6

]

---
# Impact of the changes on R spatial

.lc[

- [Need to move from **proj4string** to the **OGC WKT2** representation](https://github.com/r-spatial/sf/issues/1370#issuecomment-615385842)
- Crucial work by Roger Bivand, Edzer Pebesma, and many other package developers
- Important links: [1](https://www.r-spatial.org/r/2020/03/17/wkt.html), [2](http://rgdal.r-forge.r-project.org/articles/PROJ6_GDAL3.html), [3](https://rsbivand.github.io/ECS530_h19/ECS530_III.html)

]

.rc[

**Before PROJ6:**
<a href="https://github.com/r-spatial/sf/issues/1372">
<img src="figs/oldproj.png" width="60%" style="display: block; margin: auto;" />
</a>

**After PROJ6:**
<a href="https://wgithub.com/r-spatial/sf/issues/1372">
<img src="figs/newproj.png" width="60%" style="display: block; margin: auto;" />
</a>

]

---
# New projection representation in R

.lc[

**sf** version <0.9:
- Coordinate reference system is represented by a list with two components: `epsg` and `proj4string`

```r
library(spData)
library(sf) 
st_crs(nz)
```

```
## Coordinate Reference System:
## EPSG: 2193 
## proj4string: "+proj=tmerc 
##     +lat_0=0 +lon_0=173 
##     +k=0.9996 +x_0=1600000
##     +y_0=10000000 +ellps=GRS80
##     +towgs84=0,0,0,0,0,0,0
##     +units=m +no_defs"
```

]

.rc[

**sf** version >0.9 (released 2020-03-24):
- Coordinate reference system is represented by a lists with two components, `input` and `wkt`

```r
library(spData)
library(sf) 
st_crs(nz)
```

```
## Coordinate Reference System:
##   User input: EPSG:2193 
##   wkt:
## PROJCS["NZGD2000 / New Zealand Transverse Mercator 2000",
##     GEOGCS["NZGD2000",
##         DATUM["New_Zealand_Geodetic_Datum_2000",
##             SPHEROID["GRS 1980",6378137,298.257222101,
##                 AUTHORITY["EPSG","7019"]],
##             TOWGS84[0,0,0,0,0,0,0],
##             AUTHORITY["EPSG","6167"]],
##         PRIMEM["Greenwich",0,
##             AUTHORITY["EPSG","8901"]],
##         UNIT["degree",0.0174532925199433,
##             AUTHORITY["EPSG","9122"]],
##         AUTHORITY["EPSG","4167"]],
##     PROJECTION["Transverse_Mercator"],
##     PARAMETER["latitude_of_origin",0],
##     PARAMETER["central_meridian",173],
##     PARAMETER["scale_factor",0.9996],
##     PARAMETER["false_easting",1600000],
##     PARAMETER["false_northing",10000000],
##     UNIT["metre",1,
##         AUTHORITY["EPSG","9001"]],
##     AUTHORITY["EPSG","2193"]]
```

]

---
# New projection representation in R

.pull-left[

Before:

- `nz4326 = st_transform(nz, "+proj=longlat +datum=WGS84 +no_defs")`
- `nz4326 = st_transform(nz, 4326)`
- `nz4326 = st_transform(nz, st_crs(world))`

Now:

- `nz4326 = st_transform(nz, 4326)`
- `nz4326 = st_transform(nz, st_crs(world))`
- Other Spatial Reference System Identifiers (SRIDs) are also possible, for example `nz54030 = st_transform(nz, "ESRI:54030")`
- Use of **proj4string** is still possible, but only in a limited way (it is discouraged)

]

.pull-right[

.tiny[

]

---
# Other changes

.pull-left[
- [Categorical raster data handling](https://github.com/r-spatial/stars/issues/245) in **stars**

]

.pull-right[
- **terra** - a successor of the **raster** package is [on CRAN](https://cran.r-project.org/package=terra). 
It also has an extensive documentation at https://rspatial.org/terra/.

]

---
# Other changes

.pull-left[
- **classInt** - [Head/Tails breaks implementation](https://dieghernan.github.io/classInt/articles/headtailsR.html)

<img src="figs/new-classint.png" width="896" style="display: block; margin: auto;" />
]

.pull-right[

- **tmap** - [major release version 3.0](https://github.com/mtennekes/tmap/issues/406)

<img src="figs/new-tmap.png" width="852" style="display: block; margin: auto;" />
]

---
# Other changes

.pull-left[
- [New rendering platform](https://twitter.com/TimSalabim3/status/1249721111350317060) in the development version of the **mapview** package - `mapviewOptions(platform = "leafgl")`
- It allows for visualization of large number of spatial objects
- Example of rendering more than 400,000 lines:

]

.pull-right[
- **rgee** - [Google Earth Engine for R](https://csaybar.github.io/rgee/)

<img src="figs/new-rgee.png" width="947" style="display: block; margin: auto;" />
<img src="figs/new-rgee3.gif" width="70%" style="display: block; margin: auto;" />
]

---
# Other changes

- R version 4.0.0 tomorrow (2020-04-24)!

---
class: inverse, left, bottom

# Robin's part

---
# Installing #rspatial packages

.pull-left[

### Linux, Mac and Windows ✅

**sf** requires relatively recent versions of R...

```
Depends:     methods, R (≥ 3.3.0)
```

And other packages:

```
Imports: Rcpp (≥ 0.12.18)
```

http://cran.rstudio.com/package=sf

### Solutions

- Maintain your R set-up with care
- `update.packages()` regularly
- Read the NEWS about new and upcoming changes on development repos

]

.pull-right[
![](https://media.giphy.com/media/l378vMO5yhVuap3pe/giphy.gif)
*Artist's impression of installing sf on poorly set-up Mac*. Solution: follow `sf`'s [install instructions]((https://github.com/r-spatial/sf#installing).

]

---
# Computer set-up for sf

.lc[
![](https://raw.githubusercontent.com/geocompr/geocompr.github.io/source/static/img/geocompr-linux.png)
]

.rc[
### On [Ubuntu](https://geocompr.github.io/post/2020/installing-r-spatial-ubuntu/)

```
sudo add-apt-repository ppa:ubuntugis/ubuntugis-unstable
sudo apt install libudunits2-dev libgdal-dev libgeos-dev libproj-dev
```

Other Linux distros: see the **sf** [README](https://github.com/r-spatial/sf#linux)

### On Mac

- See the **sf** [README](https://github.com/r-spatial/sf#macos)

### On Windows

- Binary packages contain OSGeo dependencies

### Managed computers

- #### Check `.libPaths()`, make friends with your system administrator
]

---

# Docker

Try geographic R packages in the browers with [geocompr on binder.org](https://mybinder.org/v2/gh/robinlovelace/geocompr/master?urlpath=rstudio)

![](https://camo.githubusercontent.com/0e31117e59b0f68b809723f2171c3bb252f752d0/687474703a2f2f6d7962696e6465722e6f72672f62616467655f6c6f676f2e737667)

Dozens of geographic R packages and in an up-to-date box from [geocompr/docker](https://github.com/geocompr/docker#geocomputation-with-r-in-docker) repo:

```bash
docker run -e PASSWORD=pw --rm -p 8786:8787 geocompr/geocompr:rstudio_preview
```

![](https://user-images.githubusercontent.com/1825120/79747908-6244b800-8304-11ea-84cc-02b6cc1983e2.png)

Builds on the [Rocker Project](https://www.rocker-project.org/) which maintains a range of images, e.g.:

```bash
# Minimal geographic stack:
docker run -e PASSWORD=pw --rm -p 8786:8787 rockerdev/geospatial:3.6.3
```

![](https://img.shields.io/docker/image-size/rockerdev/geospatial)

---

# Compatibility with the tidyverse

.pull-left[

```r
library(sf)
library(spData)

# Some things work fine...
nz %>% 
  dplyr::select(Population) %>% 
  dplyr::slice(1)
```

```
## Simple feature collection with 1 feature and 1 field
## geometry type:  MULTIPOLYGON
## dimension:      XY
## bbox:           xmin: 1568217 ymin: 5971967 xmax: 1745493 ymax: 6191874
## CRS:            EPSG:2193
##   Population                           geom
## 1     175500 MULTIPOLYGON (((1745493 600...
```

- Some support for `vctrs`

]

.pull-right[

```r
library(ggplot2)
ggplot(nz) + geom_sf()
```

]

---

# Work-in-progress

.pull-left[

- Subset assignment for `vctrs` `sfc` objects 
- See https://github.com/r-spatial/sf/pull/1374
- https://github.com/paleolimbot/geovctrs
- [sfnetworks](https://luukvdmeer.github.io/sfnetworks/articles/intro.html)

![](https://luukvdmeer.github.io/sfnetworks/articles/intro_files/figure-html/unnamed-chunk-9-1.png)

]

.pull-right[

### Issues to iron-out, including
  - `data.table`/spatial integration
  - further integration with tidyverse, e.g.:

```r
nz %>% dplyr::add_row()
```

```
## Error: No common type for `..1` <sf<
## Name : character
## Island : character
## Land_area : double
## Population : double
## Median_income: integer
## Sex_ratio : double
## geom : sfc_MULTIPOLYGON
## >> and `..2` <tbl_df<>>.
```

]

---

# Work-arounds, data.table and a benchmark

.pull-left[

```r
rb = function() rbind(nz, nz)
br = function() {
  dplyr::bind_rows(
    st_drop_geometry(nz),
    st_drop_geometry(nz)
  ) %>% st_sf(geometry = c(
    st_geometry(nz),
    st_geometry(nz)
  ))
}
dt = function() {
  st_as_sf(
    data.table::rbindlist(list(nz, nz))
  )
}
```

]

.pull-right[

```r
bench::mark(rb(), br(), dt(), check = FALSE)
```

```
## # A tibble: 3 x 6
## expression min median `itr/sec` mem_alloc `gc/sec`
## <bch:expr> <bch:tm> <bch:tm> <dbl> <bch:byt> <dbl>
## 1 rb() 2.32ms 2.44ms 340. 11.51KB 2.03
## 2 br() 655.9µs 689.8µs 1433. 88.06KB 0 
## 3 dt() 335.35µs 349.16µs 2839. 3.18MB 4.25
```

```r
identical(rb()$geom, br()$geom)
```

```
## [1] TRUE
```

```r
identical(rb()$geom, dt()$geom)
```

```
## [1] TRUE
```

]

---

# Mapping Covid19

.lc[

- See https://geocompr.github.io/post/2020/mapping-covid-19/
- Starting point

```r
library(sf)
library(dplyr)
library(tmap)
d = COVID19::covid19()
```

]

.rc[

![](https://user-images.githubusercontent.com/1825120/80092918-7f26f880-855b-11ea-8017-c4ec6e382209.gif)

]

---
# How to contribute

## Learn by doing

- [ ] **Communicate** your work by citing R and R packages and simply telling people
- [ ] Ask questions to **colleagues and friends** in person/email/Skype
- [ ] Ask **strangers** (politely) on sites such as [r-sig-geo](https://stat.ethz.ch/mailman/listinfo/r-sig-geo), [stack](https://stackoverflow.com/)/[GIS](https://gis.stackexchange.com/), [r-spatial/discuss](https://github.com/r-spatial/discuss/issues/)/social media 
- [ ] **Confident you've found a problem?** Post a **reproducible** and **constructive** issue on GitHub

![](https://user-images.githubusercontent.com/1825120/79749700-9077c700-8307-11ea-9a6b-1d8356a65741.png)

---

# Plug: sfnetworks hackathon 16th June
- Online hackathon 16th June sign-up [here](https://www.eventbrite.co.uk/e/e-rum2020-satellite-event-hackathon-on-spatial-networks-tickets-90976873277)
- In-development spatial network package hosted at https://github.com/luukvdmeer/sfnetworks
- Questions? Ask at https://github.com/luukvdmeer/sfnetworks/issues

---
class: left, top, clear

.pull-left[
## Next steps

- Getting started
  - Install R 4.0.0 (24th April)
  - `install.packages(c("sf", "raster"))`
  - Check out [*Geocomputation with R*](https://geocompr.robinlovelace.net/) Chapter [2](https://geocompr.robinlovelace.net/spatial-class.html)

- Details of recent changes
  - Check-out the [r-spatial blog](https://www.r-spatial.org/)
  - [Remote Sensing with `terra`](https://rspatial.org/terra/rs/index.html)
  - [Spatial Data Science](https://keen-swartz-3146c4.netlify.app/raster.html#package-stars) chapter on `stars`
  - [Article on changes in GDAL and PROJ](https://www.r-spatial.org/r/2020/03/17/wkt.html)

- Challenges
  - Communicate
  - Test upcoming changes
  - Contribute to development efforts

]

.pull-right[
## Contact:

#geocompr

jakub_nowosad

robinlovelace

## Resources:

https://geocompr.github.io/

https://nowosad.github.io

https://www.robinlovelace.net/

**Slides:** [nowosad.github.io/whyr_webinar004](https://nowosad.github.io/whyr_webinar004)

**Covid-19** worked example: [geocompr.github.io/post](https://geocompr.github.io/post/2020/mapping-covid-19/)

]