---
title: "Recode mortality survey data"
output: 
  word_document:
    keep_md: true
---

# Introduction to this template

This is a template which can be used to recode data from non-Kobo formats to be 
able to use the template for creating a report from a retrospective mortality 
survey. 
This will produce a word document which can be kept as a log to show how the data 
was changed. 

- There are sections for reading, reformatting and cleaning data  
- For a more detailed explanation of this template, please visit https://github.com/R4EPI/sitrep/  
- Feedback and suggestions are welcome at the [GitHub issues page](https://github.com/R4EPI/sitrep/issues)

- Text within <! > will not show in your final document. These comments are used
to explain the template. You can delete them if you want.

<!-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This comment will not show up when you knit the document.
A comment with a title with slashes indicates a name of a code chunk.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -->

## Installing and loading required packages 

<!-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/// setup \\\
--------------------------------------------------------------------------------
Several packages are required for different aspects of  analysis with *R*. 
You will need to install these before starting. 
These packages can be quite large and may take a while to download in the
field. If you have access to a USB key with these packages, it makes sense to
copy and paste the packages into your computer's R package library 
(run the command .libPaths() to see the folder path). 
For help installing packages, please visit https://github.com/R4EPI/sitrep/
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -->

```{r setup, include = FALSE, results='hide', message=FALSE, warning=FALSE}


## You need to activate code chunks by changing eval to true below

## hide all code chunks in the output, but show errors
knitr::opts_chunk$set(eval = FALSE,       # YOU NEED TO CHANGE THIS TO TRUE
                      echo = TRUE,        # show all code chunks in output
                      error = TRUE,       # show errors if they appear, but don't stop
                      warning = TRUE,     # show warnings if they appear, but don't stop
                      message = TRUE      # show messages if they appear, but don't stop
                      )


# Ensures the package "pacman" is installed
if (!require("pacman")) {
     install.packages("pacman") }

# Install and load required packages for this template
pacman::p_load(
  knitr,       # create output docs
  here,        # find your files
  rio,         # for importing data
  janitor,     # clean/shape data
  sitrep,      # MSF field epi functions
  summarytools # browse data content
  )

```


<!-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/// read_non_kobo_data \\\
--------------------------------------------------------------------------------
This section is for data not from Kobo.  
If you already have data in Kobo format - use the mortality survey template. 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -->

```{r read_non_kobo_data, warning = FALSE, message = FALSE}

## Excel file ------------------------------------------------------------------
## to read in a specific sheet use "which"
study_data_hh <- rio::import(here::here("mortality_survey.xlsx"), which = "Sheet1")

## Excel file -- Specific range ------------------------------------------------
## you can specify a range in an excel sheet.
# study_data_hh <- rio::import(here::here("mortality_survey.xlsx"), range = "B2:J102")

## Excel file with password ----------------------------------------------------
## use this section if your Excel has a password.
# install.packages(c("excel.link", "askpass"))
# library(excel.link)
# study_data_hh <- xl.read.file(here::here("mortality_survey.xlsx"),
#                              xl.sheet = "Sheet1",
#                              password = askpass::askpass(prompt = "please enter file password"))

## CSV file --------------------------------------------------------------------
# study_data_hh <- rio::import(here::here("mortality_survey.csv"))

## Stata data file -------------------------------------------------------------
# study_data_hh <- rio::import(here::here("mortality_survey.dat"))
```



<!-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/// prep_non_kobo_data \\\
--------------------------------------------------------------------------------
This section is for data not from Kobo. 

It is more difficult to prepare the non Kobo data. You can do it! It will just
take a little more work.

Checklist to update this script to match your data:

[ ] Recode your variable names to match the dictionary
[ ] Recode variable contents to match the dictionary

This step shows you the data dictionary. The Kobo data dictionary has variable 
names in the "name" column of the survey sheet. Possible values for each 
variable are specified in choices sheet. 
In the choices tab, "name" has the shortened value and the "label::english" and 
"label::french" columns have the appropriate long versions.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -->

```{r prep_nonkobo_data}

## MSF Survey Dictionary ----------------------------------------------------------

## generates MSF standard dictionary for Kobo 
study_data_dict <- msf_dict("Mortality")
## generates MSF standard dictionary for Kobo in long format (for recoding)
study_data_dict_long <- msf_dict(dictionary = "Mortality", compact = FALSE)

## look at the standard dictionary by uncommenting the line below
# View(study_data_dict) 
## You will need to recode your variables to match the data dictionary. This is
## addressed below.



## Clean column names ----------------------------------------------------------
## This step fixes the column names so they are easy to use in R.

## make a copy of your orginal dataset and name it study_data_cleaned
study_data_cleaned <- study_data_raw

## define clean variable names using clean_names from the janitor package. 
study_data_cleaned <- janitor::clean_names(study_data_cleaned)

## Match column names ---------------------------------------------------------
## This step helps you match your variables to the standard variables.
## This step will require some patience. Courage!

## Use the function msf_dict_rename_helper() to create a template based on the
## standard dictionary. This will copy a rename command like the one above to your
## clipboard.

# msf_dict_rename_helper("mortality", varnames = "name")

## Paste the result below and your column names to the matching variable.

## Be careful! You still need to be aware of what each variable means and what
## values it takes.

## If there are any variables that are in the MSF dictionary that are not in
## your data set, then you should comment them out, but be aware that some
## analyses may not run because of this. 


## PASTE HERE


## Here is an EXAMPLE for changing a few specific names. function. In this
## example, we have the columns "gender" and "age" that we want to rename as
## "sex" and "age_years". 
## The formula for this is rename(data, NEW_NAME = OLD_NAME).

# study_data_cleaned <- rename(study_data_cleaned, 
#                            sex       = gender, # TEXT
#                            age_years = age     # INTEGER_POSITIVE
# )
```


<!-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/// view_data_dict \\\
--------------------------------------------------------------------------------

Take a look at the standard data dictionary to see what values you need to recode. 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -->

```{r view_data_dict}

## use the long version of the dictionary
## (repeats variable name for each value)
recode_dict <- study_data_dict_long %>% 
  ## only keep variables and values
  select("variable" = name, 
         "values" = option_name)

## browse dictionary
View(recode_dict)
```



<!-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/// browse_data \\\
--------------------------------------------------------------------------------

You'll want to look at your data. Here are a few ways you can explore.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -->

```{r browse_data, eval = FALSE}

## view your whole dataset interactivley (in an excel style format)
View(study_data_raw)

## view unique values contained in variables 
## you can run this for any column -- just replace the column name
unique(study_data_raw$sex)

## use the dfSummary function in combination with view
## note that view is not capitalised with this package
summarytools::dfSummary(study_data_cleaned) %>%
  summarytools::view()
```

<!-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/// recode_factor_vars \\\
--------------------------------------------------------------------------------

This chunk demonstrates how to recode factor variables. You will need to add to 
it based on your dataset by comparing to the variables in the standard data 
dictionary. 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -->

```{r recode_factor_vars}

## Recode character variables
## This step shows how to fix misspellings in the geographic region variable.
## Ideally, you want these values to match and population data!
study_data_cleaned <- study_data_cleaned %>%
  mutate(village_name = case_when(
    village_name == "Valliages 1"       ~ "village 1",
    village_name == "Village1"          ~ "village 1",
    village_name == "Town 3"            ~ "village 3"
    village_name == "Town3"             ~ "village 3",
    TRUE ~ as.character(village_name))
  )


## create a character variable based off groups of a different variable 
study_data_cleaned <- study_data_cleaned %>% 
  mutate(health_district = case_when(
    cluster_number %in% c(1:5) ~ "district_a", 
    TRUE ~ "district_b"
  ))

```


<!-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/// recode_numeric_vars \\\ 
--------------------------------------------------------------------------------

This chunk will help recode numeric variables. You will need to add to 
it based on your dataset by comparing to the variables in the standard data 
dictionary. 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -->
```{r recode_numeric_vars}

## split one age variable in to two 
## the dictionary has all those less than 1 year of age recorded in months
## If you have a numeric variable for age ('age_num') and an associated 
## categorical variable for the unit 'age_unit' (days, month, years), recode like this 
study_data_cleaned <- study_data_cleaned %>% 
  mutate(
    age_years = if_else(age_num >=1 & age_unit == "Year", age_num, NA), 
    age_months = if_else(age_num <1 & age_unit == "Month", age_num, NA)
  )
```


<!-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/// save_recoded_data \\\
--------------------------------------------------------------------------------

Save your recoded dataset as an Excel. 
This automatically names your file "study_data_recoded_DATE", where DATE is the
current date.
You can now use this to use the analysis template. 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -->

```{r save_cleaned_data, eval = FALSE}
rio::export(study_data_cleaned, here::here("data", str_glue("study_data_recoded_{Sys.Date()}.xlsx")))
```

