---
title: "Stat 652: Midterm: Model 0, Model 1"
author: "Prof. Eric A. Suess"
date: today
format: 
  html:
    embed-resources: true
---

```{r}
library(pacman)
p_load(titanic, tidyverse, janitor, naniar, DataExplorer, tidymodels)
```

Load the data from the titanic R package.  Note that the *titanic_train* dataset contains the labels for Survived and the *titanic_test* dataset does not contain the labels.  So we will build our machine learning model using the *titanic_train* dataset and then make a final classification for the *titanic_test* dataset.  This is how kaggle competitions are done.

I like to clean names so the variables all have names with lowercase letters and underscores.

```{r}
library(titanic)
data(titanic_train)
data(titanic_test)
```


```{r}
titanic_train <- titanic_train |> clean_names()

head(titanic_train)
```

It is always a good idea to check for duplicate records/examples/rows in your dataset.

```{r}
get_dupes(titanic_train)
```

```{r}
titanic_train2 <- titanic_train |> dplyr::select(-passenger_id, -name, -ticket, -cabin) |>
  mutate(
    survived = as_factor(survived),
    pclass = as_factor(pclass),
    sex = as_factor(sex),
    embarked = as_factor(embarked)
  )

head(titanic_train2)
```

```{r}
titanic_test <- titanic_test |> clean_names()

head(titanic_test)
```

It is always a good idea to check for duplicate records/examples/rows in your dataset.

```{r}
get_dupes(titanic_test)
```

```{r}
titanic_test2 <- titanic_test |> select(-passenger_id, -name, -ticket, -cabin) |>
  mutate(
    pclass = as_factor(pclass),
    sex = as_factor(sex),
    embarked = as_factor(embarked)
  )

head(titanic_test2)

```


Start by investigating the missing values and completeness of the features in the data.  Note that the *age* variable contains some missing values.

```{r}
vis_miss(titanic_train2)
gg_miss_var(titanic_train2)
gg_miss_var(titanic_train2, show_pct = TRUE)
```

```{r}
#| eval = FALSE
create_report(titanic_train2, y = "survived", output_file = "report.html", output_dir = getwd())
```

Now try the ML algorithms.


## Model 0:

Summarize the y-variable.  Null Model.

```{r}
titanic_train2 |> group_by(survived) |>
  summarize(n = n()) |>
  mutate(freq = n / sum(n))
```

Make the first split with 80% of the data being in the training data set.

```{r}
titanic_train2_split <- initial_split(titanic_train2, prop = 0.8)
titanic_train2_split
```

Create the recipe for applying the pre-processing.  Note the use of step_nzv(), which removes any columns that have very low variability, and the use of the step_meanimpute() function, which fills in the cells that are missing with the mean of the column.

```{r}
titanic_train2_recipe <- training(titanic_train2_split) |>
  recipe(survived ~ .) |>
  step_rm(pclass, sex, embarked) |> 
  step_nzv(all_predictors()) |>
  step_impute_mean(age) |>
  prep()

summary(titanic_train2_recipe)

tidy(titanic_train2_recipe)
```

Apply the recipe, so the *age* variable should be complete after the imputation.

```{r}
titanic_train2_training <- titanic_train2_recipe |>
  bake(training(titanic_train2_split)) 

titanic_train2_training |> arrange(age)
```

```{r}
titanic_train2_testing <- titanic_train2_recipe |>
  bake(testing(titanic_train2_split)) 

titanic_train2_testing |>  arrange(age)
```

### Model 0: null


```{r}

titanic_train2_null <- null_model()  |>  
  set_engine("parsnip")  |>  
  set_mode("classification") |> 
  fit(survived ~ ., data = titanic_train2_training)

```

```{r}
predict(titanic_train2_null, titanic_train2_training)
```

```{r}
titanic_train2_null |>
  predict(titanic_train2_testing) |>
  bind_cols(titanic_train2_testing) 
```

```{r}
mean(as.numeric(titanic_train$survived), na.rm = TRUE)
```


```{r}
titanic_train2_null |>
  predict(titanic_train2_testing) |>
  bind_cols(titanic_train2_testing) |>
  metrics(truth = survived, estimate = .pred_class)
```



```{r}
titanic_train2_null |>
  predict(titanic_train2_testing) |>
  bind_cols(titanic_train2_testing) |>
  conf_mat(truth = survived, estimate = .pred_class)
```

```{r}
titanic_train2_null |>
  predict(titanic_train2_testing, type = "prob") |>
  bind_cols(titanic_train2_testing) |>
  roc_curve(survived, .pred_0) |>
  autoplot() 
```


### 1 Model: kNN

Setup the models.

```{r}
titanic_train2_knn <- nearest_neighbor(neighbors = 11) |> 
  set_engine("kknn") |>
  set_mode("classification") |>
  fit(survived ~ ., data = titanic_train2_training)

```

What is missing here?  The scaling or normalization has not been applied.


```{r}
predict(titanic_train2_knn, titanic_train2_training)
```

```{r}
titanic_train2_knn |>
  predict(titanic_train2_testing) |>
  bind_cols(titanic_train2_testing) 
```


```{r}
  titanic_train2_knn |>
  predict(titanic_train2_testing) |>
  bind_cols(titanic_train2_testing) |>
  metrics(truth = survived, estimate = .pred_class)
```



```{r}
titanic_train2_knn |>
  predict(titanic_train2_testing) |>
  bind_cols(titanic_train2_testing) |>
  conf_mat(truth = survived, estimate = .pred_class)
```

```{r}
titanic_train2_knn |>
  predict(titanic_train2_testing, type = "prob") |>
  bind_cols(titanic_train2_testing) |>
  roc_curve(survived, .pred_0) |>
  autoplot() 
```
