---
title: "Data Wrangling R with Answers"
author: "Prof. Eric A. Suess"
output:
  pdf_document: default
  html_document:
    df_print: paged
  html_notebook: default
  word_document: default
---

Some of the code from Chapter 4, Section 1.

In this chapter dplyr is introduced.  We will be using dplyr all year.

The main idea of data wrangling with dplyr are the 5 verbs. 

**select()**  # take a subset of columns

**filter()**  # take a subset of rows

**mutate()**  # add or modify existing columns

**arrange()**  # sort the rows

**summarize()**  # aggregate the data across rows

The dplyr package is part of the tidyverse.  We will install and load the tidyverse.

```{r message=FALSE}
library(mdsr)
library(tidyverse)
```

# Star Wars dataset

```{r}
data("starwars")
glimpse(starwars)
```

# select()

```{r}
starwars %>% select(name, species)
```

# filter()

```{r}
starwars %>% 
  filter(species == "Droid")
```

# select()

```{r}
starwars %>% 
  select(name, ends_with("color"))
```

# mutate()

```{r}
starwars %>% 
  mutate(name, bmi = mass / ((height / 100)  ^ 2)) %>%
  select(name:mass, bmi)
```

# arrange()

```{r}
starwars %>% 
  arrange(desc(mass))
```

# summarize()

```{r}
starwars %>%
  group_by(species) %>%
  summarise(
    n = n(),
    mass = mean(mass, na.rm = TRUE)
  ) %>%
  filter(n > 1)
```

# Questions

Develop the R code to answer the following questions.

1. How many films are in the dataset?

```{r}
starwars %>% select(films) %>%
  unlist() %>%
  unique()
```



2. Are there more Droids or humans in the Star Wars movies?  There are 5 Droids and 35 Humans.  So more Humans.

```{r}
starwars %>% select(species) %>%
  filter(species=="Droid" | species=="Human") %>%
  group_by(species) %>%
  summarize(n=n())
    
```

3. Which of the Star Wars movies was Luke Skywalker in?

```{r}
starwars %>% filter(name=="Luke Skywalker") %>%
  select(films) %>%
  unlist()
```

4. Pose a question and answer it by wrangling the starwars dataset.

What was the distribution of hights?  What was the distribution of hights by species?

```{r}
starwars %>% ggplot(aes(x=height)) +
  geom_histogram()

starwars %>% ggplot(aes(x=height, color=gender)) +
  geom_histogram(aes(y=..density..))

starwars %>% ggplot(aes(x=height, color=gender)) +
  geom_density(aes(y=..density..))

```




# Presidential examples

Try out the code in Chapter 4 Section 1 using the presidential data set.

```{r}
presidential
```

## Star Wars API and R package

More Star Wars stuff you might find interesting.

- Check out the [Star Wars](https://www.starwars.com/) website.  
- Check out the Star Wars API [sawpi](https://swapi.co/).
- And check out the R package [rwars](https://github.com/Ironholds/rwars).

## rwars package

This is a package that connects to the [sawpi](https://swapi.co/) to pull data from the API.

If the package does not install from CRAN you can isntall it from github.

    library(devtools)
    install_github("ironholds/rwars")
    


```{r echo=TRUE}
library(rwars)

planet_schema <- get_planet_schema()
names(planet_schema)

```

## rwars package

Get an individual starship - an X-wing.

Hopefully it won't time out and will actually bring the data back.

```{r echo=TRUE}
x_wing <- get_starship(12)
x_wing
```

## Alternative API that can be accessed via an R package

The [compstatr](https://cran.r-project.org/web/packages/compstatr/vignettes/compstatr.html) R package gives direct access to the St. Louis Metropolitan Police Department's [website](https://www.slmpd.org/Crimereports.shtml).

```{r}
library(compstatr)

cs_last_update()

i <- cs_create_index()

aug19 <- cs_get_data(year = 2019, month = "August", index = i)
aug19
```

The [ukpolice](https://github.com/evanodell/ukpolice) R package to download data from UK Police public data API.

```{r}
library(ukpolice)
library(ggplot2)
library(dplyr)

tv_ss <- ukc_stop_search_force("thames-valley", date = "2018-12")

tv_ss2 <- tv_ss %>% 
  filter(!is.na(officer_defined_ethnicity) & outcome != "" ) %>%
  group_by(officer_defined_ethnicity, outcome) %>%
  summarise(n = n()) %>%
  mutate(perc = n/sum(n))

p1 <- ggplot(tv_ss2, aes(x = outcome, y = perc,
                             group = outcome, fill = outcome)) + 
  geom_col(position = "dodge") + 
  scale_y_continuous(labels = scales::percent,
                     breaks = seq(0.25, 0.8, by = 0.25)) + 
  scale_x_discrete(labels = scales::wrap_format(15)) + 
  theme(legend.position = "none", axis.text.x = element_text(size = 8)) + 
  labs(x = "Outcome", 
       y = "Percentage of stop and searches resulting in outcome",
       title = "Stop and Search Outcomes by Police-Reported Ethnicity",
       subtitle = "Thames Valley Police Department, December 2018",
       caption = "(c) Evan Odell | 2019 | CC-BY-SA") + 
  facet_wrap(~officer_defined_ethnicity)

p1
```

Alternatively you could use the other [ukpolice](https://github.com/njtierney/ukpolice) R package that is available through github.

And here is a nice blog post about crime in SF [Using R for Crime Analysis](https://wetlands.io/maps/Crime-Analysis-Using-R.html.)