---
title: "discovr comparing two means"
author: "Andy Field"
output:
  learnr::tutorial:
    progressive: false
    theme: "united"
    highlight: "kate"
    css: ./css/discovr_style_future.css
runtime: shiny_prerendered
description: "Categorical predictors with two categories (comparing two means). Comparing two independent means, comparing two related means, effect sizes, robust comparisons of means (independent and related)."
bibliography: [discovr_09.bib, packages.bib]
---
<html lang="en">

```{r setup, include=FALSE}
knitr::opts_chunk$set(
	echo = TRUE,
	message = FALSE,
	warning = FALSE
)

#necessary to render tutorial correctly
library(learnr) 
library(htmltools)
#easystats
library(datawizard)
library(effectsize)
library(insight)
library(parameters)
#tidyverse
library(dplyr)
library(ggplot2)
library(stringr)
library(tidyr)
#non tidyverse/easystats
library(WRS2)
#students don't use

source("./www/discovr_helpers.R")

#Read dat files needed for the tutorial

cloak_tib <- discovr::invisibility_cloak
cloak_rm_tib <- discovr::invisibility_rm
cloak_messy_tib <- cloak_rm_tib |> 
  pivot_wider(
    id_cols = id,
    values_from = mischief,
    names_from = cloak
    ) |> 
  rename_with(.data = _, ~ tolower(str_replace(.x, pattern = " ", replacement = "_"))) 
```


# discovr comparing two means

```{r, child = "./docs/intro.Rmd"}

```

## Packages {data-progressive=FALSE}

```{r, child = "./docs/packages.Rmd"}

```


## Data

```{r, child = "./docs/data.Rmd"}

```


## Fitting models

```{r, child = "./docs/fit_models.Rmd"}

```


## `r bmu()` Comparing two means: process [(A)]{.lbl}

I imagine a future in which we have some cloaks of invisibility to test out. Given my slightly mischievous streak, the future me is interested in the effect that wearing a cloak of invisibility has on the tendency for mischief. I take 24 participants and placed them in an enclosed community. The community is riddled with hidden cameras so that we can record mischievous acts. Half of the participants are given cloaks of invisibility; they are told not to tell anyone else about their cloak and that they can wear it whenever they liked. I measure how many mischievous acts they performed in a week. We hypothesise:

> H~1~: There will be more mischievous acts, on average, when a cloak is worn than when not.

We can follow our general model fitting process to compare two means by predicting the outcome from the categorical variable that defines the two groups we wish to compare. We will look at this method in detail in `discovr_11`. This tutorial focusses on the historical approach of using a so-called *t*-test, although when the means being compared are independent the results are identical to fitting a model using `lm()`. Figure 4 shows this simplified process, although broadly speaking we can still follow our 5-step process.

<div class="stepbox">
  `r step()` **Step 1 and 2**

 - Step 1: Get your data into `r rproj()` and pre-process using [tidyverse]{.pkg} packages or the [datawizard]{.pkg} package from [easystats]{.pkg}.
 - Step 2: Visualise the data using the [ggplot2]{.pkg} package from [tidyverse]{.pkg}.
</div>

</br>

<figure>
<img src="./images/dsr2_fig_09_04_t_process.svg" alt="See main text for description." style="width:100%">
<figcaption>Figure 4: The general process for comparing means.</figcaption>
</figure>

## `r bmu()` [Step 1]{.alt}: Summarize the data [(A)]{.lbl}

These data are preloaded in this tutorial in a tibble called [cloak_tib]{.alt}.

#### `r alien()` Alien coding challenge

Use the code box to view these data.

```{r cloak_data, exercise = TRUE, exercise.lines = 2}

```

```{r cloak_data-solution}
cloak_tib
```

Note there are three variables:

- `id`: participant id
- `cloak`: whether the participant was assigned a cloak of invisibility
- `mischief`: the number of mischievous acts committed during the week of the study

We can use `describe_distribution()` from [datawizard]{.pkg} to quickly get summary statistics for the variables in the data.

#### `r alien()` Alien coding challenge

Use what you have already learnt about `describe_distribution()` in `discovr_04` to get a table of descriptive statistics for `mischief` for the two cloak groups.

```{r album_whiz, exercise = TRUE, exercise.lines = 5}

```

```{r album_whiz-hint-1}
# pipe the data into group_by() to split the descriptives by group. Replace the xxx
cloak_tib |>
  group_by(xxx)
# now use describe_distribution()
```

```{r album_whiz-hint-2}
#  Use describe_distribution(). Replace the xxx
cloak_tib |>
  group_by(cloak) |> 
  describe_distribution(ci = xxx, iterations = xxx)
# now get the table to regnder nicely
```

```{r album_whiz-solution}
cloak_tib |> 
  group_by(cloak) |> 
  describe_distribution(ci = 0.95, iterations = 1000) |> 
  display()
```


## `r bmu()` [Step 2]{.alt}: Visualizing the data [(A)]{.lbl}

#### `r alien()` Alien coding challenge

We can visualise the data using a violin plot with error bars (see `discovr_05`). Create such a plot below:

```{r exam_plot, exercise = TRUE, exercise.lines = 5}

```

```{r exam_plot-hint-1}
# Set up the plot by replacing the XXXs in this code:
ggplot(xxxxxx, aes(xxxxxx, xxxxxx)) 
```

```{r exam_plot-hint-2}
# Add geom_violin in the usual way
ggplot(cloak_tib, aes(cloak, mischief)) +
  geom_violin()
```

```{r exam_plot-hint-3}
#Add a layer that plots means and confidence intervals using 
# stat_summary()
stat_summary(fun.data = "xxxxxxx")
```

```{r exam_plot-hint-4}
ggplot(cloak_tib, aes(cloak, mischief)) +
  geom_violin() +
  stat_summary(fun.data = "mean_cl_normal")

# Add some labels for the axes
```

```{r exam_plot-hint-5}
ggplot(cloak_tib, aes(cloak, mischief)) +
  geom_violin() +
  stat_summary(fun.data = "mean_cl_normal") +
  labs(x = "Cloak group", y = "Acts of mischief") 

# add theme_minimal()
```

```{r exam_plot-hint-6}
ggplot(cloak_tib, aes(cloak, mischief)) +
  geom_violin() +
  stat_summary(fun.data = "mean_cl_normal") +
  labs(x = "Cloak group", y = "Acts of mischief") +
  theme_minimal()
```


Let's move onto 

<div class="stepbox">
  `r step()` **Step 3**

Fit a statistical model that will answer your research question.
 
</div>


## `r bmu()` Comparing two independent means [(A)]{.lbl}

### `r bmu()` [Step 3]{.alt}: Fitting the model [(A)]{.lbl}

You can do a *t*-test in `r rproj()` using the `t.test()` function, which takes this general form

#### `r robot()` Code example
 
```{r, echo = T, eval = F}
new_model <- t.test(outcome ~ predictor,
                    data = tibble,
                    var.equal = FALSE,
                    conf.level = 0.95,
                    na.action = na.exclude)
```

In which:

* [new_model]{.alt}: an object created that contains information about the model. We can get summary statistics for this model by executing the name of the model.
* [outcome]{.alt}: the variable that contains the scores for the outcome measure (in this case **mischief**).
* [predictor]{.alt}: the variable that contains information about to which group a score belongs (in this case **cloak**).
* [tibble]{.alt}: the name of the tibble containing the data (in this case [cloak_tib]{.alt}) 
* [var.equal]{.alt}: by default the function assumes that variances are unequal ([var.equal = FALSE]{.alt}) and applies Welch's correction (a sensible thing to do). Leave this default alone.
* [conf.level]{.alt}: determines the alpha level for the *p*-value and confidence intervals. By default it is 0.95 (for 95% confidence intervals) and usually you'd exclude this option, but if you want to use a different value, say 99%, you could include [conf.level = 0.99]{.alt}.
* [na.action]{.alt}: If you have complete data (as we have here) exclude this option, but if you have missing values (i.e., [NA]{.alt}s in the data frame) then it can be useful to include [na.action = na.exclude]{.alt}, which will exclude all cases with missing values

#### `r alien()` Alien coding challenge

Using the general code to complete the code to fit a model using `t.test()` and store it as an object named `cloak_mod`. Remember, the data are in a tibble called [cloak_tib]{.alt}, the outcome variable is **mischief** and the grouping variable/predictor is **cloak**.

```{r t1, exercise = TRUE}
cloak_mod <- 
```

```{r t1-solution}
cloak_mod <- t.test(mischief ~ cloak, data = cloak_tib)
```


<div class="bug">
  `r bug()` **De-bug**

It will appear as though nothing has happened because you fitted and stored the model but not yet viewed it.

</div>



<div class="stepbox">
  `r step()` **Step 4**

Evaluate the model using a small number of functions from the [performance]{.pkg} package from [easystats]{.pkg}. Specially, we evaluate

- How well the model fits the data
- Whether the underlying assumptions of the model are met. If not, we refit the model using robust methods (see Figure 3).
 
</div>

We sort of bypass this step using `t.test()` because by default `r rproj()` uses a robust test known as Welch's *t*-test, which corrects for the main assumoption that we need to care about (heteroscedasticity). So, we can move straight onto step 5:

<div class="stepbox">
  `r step()` **Step 5**

Interpret the values that define the model (the [parameter estimates]{.alt}) using the [parameters]{.pkg} package from [easystats]{.pkg}.
 
</div>

### `r bmu()` [Step 5]{.alt}: Interpreting the model [(A)]{.lbl}

#### `r bug()` Exercise

We can view the model - as ever - by feeing it into `model_parameters()` from [parameters]{.alt}. try this in the code box.

```{r, indy_t_setup, echo = F}
cloak_mod <- t.test(mischief ~ cloak, data = cloak_tib)
```


```{r t1b, exercise = TRUE, exercise.lines = 5, exercise.setup = "indy_t_setup"}

```

```{r t1b-hint-1}
# Place your model into model_parameters().
model_parameters(xxx) 
# Now use display so that you get a nice rendered table
```


```{r t1b-solution}
model_parameters(cloak_mod) |> 
  display() 
```

```{r, echo = F}
cloak_mod <- t.test(mischief ~ cloak, data = cloak_tib)
cloak_par <-  model_parameters(cloak_mod)
```



The `t.test()` function calculates Welch's *t*, *t*(`r value_from_ez(cloak_par, value = "df_error")`) = `r value_from_ez(cloak_par, value = "t")`, which does not assume homogeneity of variance but instead adjusts for it. This default behaviour is sensible because, when the assumption is met no adjustment is made, but when it is broken an adjustment is made proportionate to the difference in variances. The resulting (two-tailed) *p*-value is `r value_from_ez(cloak_par, value = "p", p_digits = 3)`, which represents the probability of getting a *t* of `r value_from_ez(cloak_par, value = "t")` or smaller if the null hypothesis were true. Assuming our alpha is 0.05, we'd conclude that there was no significant difference between the means of these two samples because the observed `r value_from_ez(cloak_par, value = "p", p_digits = 3)` is greater than the criterion of 0.05. In terms of the experiment, we can infer that having a cloak of invisibility did not significantly affect the amount of mischief a person got up to.

Finally, the confidence interval gives us an estimate of the range of the true difference between means. If we were to assume that this sample were one of the 95% that yields a confidence interval containing the population value, we'd conclude that the population difference falls between `r value_from_ez(cloak_par, value = "CI_low")` to `r value_from_ez(cloak_par, value = "CI_high")`, but remember our assumption will be wrong 5% of the time.

The [effectsize]{.pkg} package [@effectsize2020; @R-effectsize] that is loaded as part of [easystats]{.pkg}, contains functions for computing effect sizes. In the current context, there are two useful functions:

#### `r robot()` Code example

```{r, eval = F, echo= T}
cohens_d(outcome ~ predictor,
         data = tibble,
         pooled_sd = TRUE,
         adjust = FALSE,
         reference = NULL,
         ci = 0.95)

glass_delta(outcome ~ predictor,
            data = tibble,
            adjust = FALSE,
            reference = NULL,
            ci = 0.95)
```

Both functions estimate Cohen's *d*, but `glass_delta()` uses only the control group standard deviation so should be used when group standard deviations are very different (or you expect your experimental manipulation to affect both the mean and the standard deviation of scores). In both functions you replace [outcome]{.alt} with the name of the outcome variable, [predictor]{.alt} with the name of the variable specifying the two groups to be compared, and [tibble]{.alt} with the name of the tibble containing the variables. Both functions have arguments to determine the width of the confidence interval (by default [ci = 0.95]{.alt}) and whether a small-sample adjustment is made (by default [adjust = FALSE]{.alt} for `cohens_d()` and [adjust = TRUE]{.alt} for `glass_delta()`).  In addition, `cohens_d()` has an argument to specify whether to use the pooled estimate of the standard deviation assuming equal variances ([pooled_sd = TRUE]{.alt}); if set to [FALSE]{.alt} the mean standard deviation from the two groups is used. Both functions also have a reference argument to specify the text label of the group that you want to use as the reference category (by default the second category is used. For both Cohen's *d* and Glass' $\Delta$ this argument affects the direction of the effect because the reference category mean is subtracted from the other group mean to calculate the difference. For Glass' $\Delta$ the reference group standard deviation is used in the denominator

#### `r alien()` Alien coding challenge

Use the code box to calculate Cohens *d* using the No cloak group standard deviation, adjust for the small sample size

```{r g1, exercise = TRUE}

```

```{r g1-solution}
# note we don't need include adjust = TRUE because that is the default value for glass_delta()
glass_delta(mischief ~ cloak, data = cloak_tib, reference = "No cloak") |>
  display()
```


Now use the code box to calculate Cohens *d* using the pooled standard deviation, adjust for the small sample size

```{r d1, exercise = TRUE}

```

```{r d1-solution}
# note we DO need to include adjust = TRUE because the default is FALSE for cohens_d()
cohens_d(mischief ~ cloak, data = cloak_tib, reference = "No cloak", adjust = TRUE) |>
  display()
```



```{r, echo = F, results = "hide"}
cloak_sum <- cloak_tib |> 
  group_by(cloak) |> 
  describe_distribution(ci = 0.95, iterations = 1000)
d_cloak <- cohens_d(mischief ~ cloak, data = cloak_tib, reference = "No cloak", adjust = TRUE)
g_cloak <- glass_delta(mischief ~ cloak, data = cloak_tib, reference = "No cloak")
```



<div class="infobox">
  `r info()` **Hedges' *g* and Cohen's *d***

Note that because we asked for the small-sample adjustment, the effect size is labelled Hedges' *g*, conceptually it's the same as Cohen's *d*, but reporting it as g makes clear that the adjustment was applied.
</div>


Using the pooled estimate, there is `r sprintf("%.2f", d_cloak$Hedges_g)` of a standard deviation difference between the two groups in terms of their mischief making, which is a fairly substantial effect. 


```{r d_quiz, echo = F}
question("Which of these statements about Cohen's *d* is **NOT** correct?",
    answer("The value of *d* cannot exceed 1.", correct = TRUE, message = "This statement is false and so is the correct answer."),
    answer("*d* is the difference between two means expressed in standard deviation units.", message = "This statement is true so is not the correct answer."),
    answer("A *d* of 0.2 would be considered small", message = "This statement is true so is not the correct answer."),
    answer("*d* can be computed using a control group standard deviation, the standard deviation of all scores or a pooled standard deviation.", message = "This statement is true so is not the correct answer."),
    correct = "Correct - well done!",
    random_answer_order = TRUE,
    allow_retry = T
  )
```

<div class="reportbox">
  `r pencil()` **Report**`r rproj()`

On average, participants given a cloak of invisibility engaged in more acts of mischief (*M* = `r value_from_ez(cloak_sum, row = 2, value = "Mean")`, 95% CI[`r value_from_ez(cloak_sum, row = 2, value = "CI_low_mean")`, `r value_from_ez(cloak_sum, row = 2, value = "CI_high_mean")`]), than those not given a cloak (*M* = `r value_from_ez(cloak_sum, value = "Mean")`, 95% CI[`r value_from_ez(cloak_sum, value = "CI_low_mean")`, `r value_from_ez(cloak_sum, value = "CI_high_mean")`]). Having a cloak of invisibility did not significantly affect the amount of mischief a person got up to: the mean difference, *M* = `r value_from_ez(cloak_par, value = "Difference")`, 95% CI [`r value_from_ez(cloak_par, value = "CI_low")`, `r value_from_ez(cloak_par, value = "CI_high")`], was not significantly different from 0, *t*(`r value_from_ez(cloak_par, value = "df_error")`) = `r value_from_ez(cloak_par, value = "t")`, `r value_from_ez(cloak_par, value = "p")`. This effect was very large, `r report_es(d_cloak, col = "Hedges_g")`, but the confidence interval for the effect size contained zero. If this confidence interval is one of the 95% that captures the population effect size then this suggests that a zero effect is plausible. 
</div>


### `r user_astronaut()` Robust models of independent means [(C)]{.lbl}

The [WRS2]{.pkg} package [@R-WRS2; @WRS22020] has several functions for comparing independent means.

#### `r robot()` Code example

The `yuen()` function is based on a timed mean. It takes a similar form to `t.test()`:

```{r, eval = FALSE}
WRS2::yuen(outcome ~ predictor,
           data = tibble,
           tr = .2,
           alpha = .05)
```

There are two arguments that `t.test()` does not have: 

* [tr]{.alt} is the proportion of trimming to be done. The default is .2 or 20% (which is sensible) so you can exclude this argument unless you want to specify an amount other than 20%.
* [alpha]{.alt} sets the alpha level for the test (by default 0.05). You can omit this argument unless you want to use a level other than 0.05.

#### `r alien()` Alien coding challenge

Use the code example, and what you learnt about the [`t.test` function](#itt) to create (and view) an object called [cloak_rob]{.alt} that contains a robust comparison of the 20% trimmed means of the two cloak conditions. As ever we can put our model into `model_parameters()` and `display()` to get nice output.

```{r cloak_rob, exercise = TRUE, exercise.lines = 5}

```

```{r cloak_rob-solution}
cloak_yn <- WRS2::yuen(mischief ~ cloak, data = cloak_tib) 
model_parameters(cloak_yn) |> 
  display()
```

<div class="tip">
  `r cat_space()` **Tip: Andy's naming obsessions**
  
  I tend to use the suffix [_yn]{.alt} to denote a robust model.
</div>

```{r, echo = F}
cloak_yn <- WRS2::yuen(mischief ~ cloak, data = cloak_tib) 
cloak_yn_par <- model_parameters(cloak_yn) 
```

<br />

<div class="reportbox">
  `r pencil()` **Report**`r rproj()`

There was not a significant difference in mischief scores across the two cloak groups, $T_y$(`r  value_from_ez(cloak_yn_par, value = "df_error")`) = `r  value_from_ez(cloak_yn_par, value = "Estimate")`, `r value_from_ez(cloak_yn_par, value = "p")`. On average the no cloak group performed one less mischievous act, *M* = `r value_from_ez(cloak_yn_par, value = "Difference")` with a 95% confidence interval for the trimmed mean difference ranging from `r value_from_ez(cloak_yn_par, value = "Difference_CI_low")` to `r value_from_ez(cloak_yn_par, value = "Difference_CI_high")`.
</div>


We can also compare trimmed means but include a bootstrap by using `yuenbt()`, which takes the same form as `yuen()` but has two additional arguments:

* [nboot = 599]{.alt}: This specifies the number of bootstrap samples to be used. If you exclude this option then the default is 599, which, if anything, you might want to increase (but it's probably not necessary to use more than 2000).
* [side = F]{.alt}: By default the function bootstraps confidence intervals as is, which means that they can be asymmetric. If you want to force the CI to be symmetrical then include [side = T]{.alt} in the function. If you do this you will get a *p*-value, but by default you won't (although you can infer significance from whether the confidence interval crosses zero)

#### `r robot()` Code example

In general for a bootstrap test of independent means based on 20% trimming we execute:

```{r, eval = FALSE}
WRS2::yuenbt(outcome ~ predictor,
           data = tibble,
           tr = .2,
           alpha = .05,
           nboot = 1000,
           side = TRUE)
```

For the default of a 20% trim and alpha of 0.05, this command reduces to:

```{r, eval = FALSE}
WRS2::yuenbt(outcome ~ predictor,
             data = tibble,
             nboot = 1000,
             side = TRUE)
```

#### `r alien()` Alien coding challenge

Using the previous exercise and the code example to do a robust comparison of the 20% trimmed means of the two cloak conditions based on 1000 bootstrap samples. 

<div class="bug">
  `r bug()` **De-bug**

Sadly, `model_parameters()` and `display()` do not work for the `yuenbt()` function.
</div>

```{r cloakbt_rob, exercise = TRUE}

```

```{r cloakbt_rob-solution}
WRS2::yuenbt(mischief ~ cloak, data = cloak_tib, nboot = 1000, side = TRUE)
```

<div class="tip">
  `r cat_space()` **Tip: Bootstrapping**
  
  Because bootstrapping works through sampling the data, confidence intervals and *p*-values from a bootstrap process will differ slightly every time you fit a model. You can see this by rerunning the code in the exercise a few times and observing the output each time.

</div>

Your output should be similar to this:

```{r, echo = F}
cloak_bt_rob <- WRS2::yuenbt(mischief ~ cloak, data = cloak_tib, nboot = 1000, side = TRUE)
cloak_bt_rob
```

Based on this robust test there is not a significant difference (because the confidence interval crosses zero) in mischief scores across the two cloak groups, $T_y$ = `r  round(cloak_bt_rob$test, 2)`, *p* = `r round(cloak_bt_rob$p.value, 3)`. We're also told that on average the no cloak group performed one less mischievous act, *M* = `r cloak_bt_rob$diff` with a 95% confidence interval for the trimmed mean difference ranging from `r round(cloak_bt_rob$conf.int[1], 2)` to `r round(cloak_bt_rob$conf.int[2], 2)`.

## `r bmu()` Comparing two dependent means [(A)]{.lbl}

Let's imagine that we had collected the cloak of invisibility data using a repeated-measures design: we might have recorded each participant's natural level of mischievous acts in a week, then given them an invisibility cloak and counted the number of mischievous acts in the following week.  So, there are 12 participants (not 24) but each participant contributes two mischief scores: one from when they wore a cloak, one from when they didn't. The data are [cloak_rm_tib]{.alt}.

#### `r alien()` Alien coding challenge

Inspect the tibble.

```{r cloak_rm_data, exercise = TRUE, exercise.lines = 2}

```

```{r cloak_rm_data-solution}
cloak_rm_tib
```

Note that the mischief scores themselves are identical to the previous example, for example, the first 'no cloak' score is a 3 and the first 'cloak' score is a 4, the only difference is that both of these scores are now attributable to the same person (Alia). To summarize then, we're using the same mischief scores as before, but we're now imagining that they were generated by a repeated measures design rather than an independent design.

<div class="stepbox">
  `r step()` **Step 1**

Get your data into `r rproj()` and pre-process using [tidyverse]{.pkg} packages or the [datawizard]{.pkg} package from [easystats]{.pkg}.
</div>

To do a paired *t*-test requires the data to be in messy format, which is annoying given how much I go on about using tidy data in the book. Worry not, because in `discovr_06` we used the `pivot_wider()` function to turn tidy data into messy data. No-one can won't remember that far back so to recap we can create a messy version of the repeated measures data using the following code:

```{r, eval = F}
cloak_messy_tib <- cloak_rm_tib |>
  pivot_wider(
    id_cols = id,
    values_from = mischief,
    names_from = cloak
    ) |>
  rename_with(.data = _, ~ tolower(str_replace(.x, pattern = " ", replacement = "_"))) 
```

This code uses the [id]{.alt} variable to pair the scores, uses the variable [cloak]{.alt} to name the new columns, and extracts the scores from the variable [mischief]{.alt} to place in those new columns. The result will be a new tibble ([cloak_messy_tib]{.alt}) with the cloak scores stored in a variable called [Cloak]{.alt} (note the capital C) and the no cloak scores stored in a variable called [No cloak]{.alt} (note the capital and space). These variable names don't conform to good practice because they contain upper case letters and spaces, so the final line of code renames them. 

This final line of code is tricky to unpick, but essentially `rename_with()` from [tidyr]{.pkg} renames any variable based on the rule you place inside it. The [.data = _]{.alt} tells the function to apply the rule to the data coming down the pipe, and what comes after the comma is the rule. We use two functions: `tolower()` converts any text to lower case, and `str_replace()` looks for a pattern within some text and replaces it with whatever you specify. This line of code, therefore, says *rename everything by changing it to lower cases, then search for any spaces and replace them with underscores*. The result is that our variables will be renamed `no_cloak` and `cloak`.


#### `r alien()` Alien coding challenge

Use the code example to create the messy version of the data and store it in [cloak_messy_tib]{.alt}. At first, don't include the final part of the pipe and view the data, then add the final line and view it again to get a feel for what the last lin eo f code does.

```{r cloak_messy_tib, exercise = TRUE, exercise.lines = 10}

```

```{r cloak_messy_tib-hint-1}
# make the data messy:
cloak_messy_tib <- cloak_rm_tib |> 
  pivot_wider(
    id_cols = id,
    values_from = mischief,
    names_from = cloak
    )
# now add code to view it
```

```{r cloak_messy_tib-hint-2}
# make the data messy:
cloak_messy_tib <- cloak_rm_tib |> 
  pivot_wider(
    id_cols = id,
    values_from = mischief,
    names_from = cloak
    )
# view the tibble:
cloak_messy_tib
# Now add the final line that renames the varaibles and view it again
```

```{r cloak_messy_tib-solution}
# make the data messy:
cloak_messy_tib <- cloak_rm_tib |> 
  pivot_wider(
    id_cols = id,
    values_from = mischief,
    names_from = cloak
    ) |> 
  rename_with(.data = _, ~ tolower(str_replace(.x, pattern = " ", replacement = "_"))) 
# view the data:
cloak_messy_tib
```

Now the data are in the form we need them, we can move onto ...

<div class="stepbox">
  `r step()` **Step 2**

Visualise the data using the [ggplot2]{.pkg} package from [tidyverse]{.pkg}.
</div>

Because the scores haven't changed, step two is pretty much the same as what we did earlier, so we can get into 

<div class="stepbox">
  `r step()` **Step 3**

Fit a statistical model that will answer your research question.
</div>



In terms of code, we conduct a paired *t*-test in much the same way as an independent *t*-test except that the structure of the formula changes to reflect the paired scores. For dependent scores, the general form of the function is:
 
```{r, echo = T, eval = F}
new_mod <- t.test(Pair(variable_1, variable_2) ~ 1, data = tibble))
```


Note that in the formula the predictor is now [1]{.alt}, which specifies an intercept (that is, no predictor), and the outcome is [Pair(variable_1, variable_2)]{.alt} in which you replace [variable_1]{.alt} with the name of one of your variables (e.g. cloak) and [variable_2]{.alt} with the name of the other variables (e.g. no_cloak). The [Pair()]{.alt} binds these variables together (note the capital [P]{.alt}).  The other available arguments are the same as for the independent *t*-test. 

#### `r robot()` Code example

For our example, the code is

```{r, eval = F}
cloak_rm_mod  <- t.test(Pair(cloak, no_cloak) ~ 1, data = cloak_messy_tib)
model_parameters(cloak_rm_mod) |> 
  display()
```

As before, this code creates a model called [cloak_rm_mod]{.alt} based on predicting mischief scores (`mischief`) from group membership (`cloak`) and we can use `model_parameters()` to get a nicely rendered table opf output.

#### `r alien()` Alien coding challenge

Use the code box to run a dependent *t*-test and save the object as [cloak_rm_mod]{.alt}

```{r t_text_rm, exercise = TRUE, exercise.lines = 5}

```

```{r t_text_rm-solution}
cloak_rm_mod  <- t.test(Pair(cloak, no_cloak) ~ 1, data = cloak_messy_tib)
model_parameters(cloak_rm_mod) |> 
  display()
```

<div class="stepbox">
  `r step()` **Steps 4 and 5**

- Step 4. Evaluate the model
- Step 5. Interpret the model

</div>

As with the the independent *t*-test, when we use the *t*-test approach we evaluate the model by checking against robust methods (or just use a robust variant in the first place). We'll look at these methods shortly, so for now we'll skip to step 5.

### `r bmu()` [Step 5]{.alt}: Interpret the model [(A)]{.lbl}

```{r echo=FALSE}
cloak_rm_mod  <- t.test(Pair(cloak, no_cloak) ~ 1, data = cloak_messy_tib)
cloak_rm_par <- model_parameters(cloak_rm_mod)
```

On average, participants engaged in more acts of mischief (*M* = `r value_from_ez(cloak_sum, row = 2, value = "Mean")`, 95% CI[`r value_from_ez(cloak_sum, row = 2, value = "CI_low_mean")`, `r value_from_ez(cloak_sum, row = 2, value = "CI_high_mean")`]) when they possessed a cloak of invisibility, than when not (*M* = `r value_from_ez(cloak_sum, value = "Mean")`, 95% CI[`r value_from_ez(cloak_sum, value = "CI_low_mean")`, `r value_from_ez(cloak_sum, value = "CI_high_mean")`]). This difference, `r value_from_ez(cloak_rm_par, value = "Difference")`, 95% CI [`r value_from_ez(cloak_rm_par, value = "CI_low")`, `r value_from_ez(cloak_rm_par, value = "CI_high")`], was significant, *t*(`r value_from_ez(cloak_rm_par, value = "df_error", digits = 0)`) = `r value_from_ez(cloak_rm_par, value = "t")`, `r value_from_ez(cloak_rm_par, value = "p")`. In terms of the experiment, we might conclude that having a cloak of invisibility significantly affected the amount of mischief a person got up to. This doesn't mean the effect is important.

```{r ci_quiz, echo = F}
  question("The confidence interval for the mean difference ranged from 0.53 to 1.97. What does this tell us?",
    answer("If this confidence interval is one of the 95% that contains the population value then the population value of the difference between group means lies between 0.53 to 1.97.", correct = TRUE),
    answer("There is a 95% chance that the population value of the difference between group means lies between 0.53 to 1.97.", message = "You cannot make probability statements from a confidence interval. We don't know whether this particular CI is one of the 95% that contains the population value of the difference between means."),
    answer("The probability of this confidence interval containing the population value is 0.95.", message = "The probability of this confidence interval containing the population value is either 0 (it doesn't) or 1 (it does) but it's impossible to know which."),
    answer("I can be 95% confident that the population value of the difference between group means lies between 0.53 to 1.97.", message = "Confidence intervals do not quantify your subjective confidence."),
    correct = "Correct - well done!",
    random_answer_order = TRUE,
    allow_retry = T
  )
```

### `r user_visor()` Effect sizes for dependent means [(B)]{.lbl}

We could compute Cohen's *d* as we did earlier. However, some argue that you need to factor in the dependency between scores in treatment conditions by factoring in the correlation between the scores. We can do this using `repeated_measures_d()` from the [effectsize]{.pkg} package, in which you specify the data in the same way as for `t.test()` and can use the same arguments as `cohens_d()`.

#### `r robot()` Code example

To get the effect size to accompany our paired *t*-test adjusted for small sample bias, we'd execute

```{r, eval = F}
repeated_measures_d(Pair(cloak, no_cloak) ~ 1, data = cloak_messy_tib, method = "rm") |> 
  display()
```

#### `r alien()` Alien coding challenge

Use the code box to get the effect size for the repeated measures version of the experiment.

```{r d2, exercise = TRUE}

```

```{r d2-solution}
repeated_measures_d(Pair(cloak, no_cloak) ~ 1, data = cloak_messy_tib, method = "rm") |> 
  display()
```


```{r, echo = F}
rm_d <- repeated_measures_d(Pair(cloak, no_cloak) ~ 1, data = cloak_messy_tib, method = "rm") 
```


<div class="reportbox">
  `r pencil()` **Report**`r rproj()`
  
  
On average, participants engaged in more acts of mischief (*M* = `r value_from_ez(cloak_sum, row = 2, value = "Mean")`, 95% CI[`r value_from_ez(cloak_sum, row = 2, value = "CI_low_mean")`, `r value_from_ez(cloak_sum, row = 2, value = "CI_high_mean")`]) when they possessed a cloak of invisibility, than when not (*M* = `r value_from_ez(cloak_sum, value = "Mean")`, 95% CI[`r value_from_ez(cloak_sum, value = "CI_low_mean")`, `r value_from_ez(cloak_sum, value = "CI_high_mean")`]). Having a cloak of invisibility affected the amount of mischief a person got up to: the mean difference, `r value_from_ez(cloak_rm_par, value = "Difference")`, 95% CI [`r value_from_ez(cloak_rm_par, value = "CI_low")`, `r value_from_ez(cloak_rm_par, value = "CI_high")`], was significantly different from 0, *t*(`r value_from_ez(cloak_rm_par, value = "df_error", digits = 0)`) = `r value_from_ez(cloak_rm_par, value = "t")`, `r value_from_ez(cloak_rm_par, value = "p")`. This effect was very large, `r report_es(rm_d, col = "d_rm")`. If this confidence interval is one of the 95% that captures the population effect size then the effect could be as small as `r value_from_ez(rm_d, value = "CI_low")` or as large as `r value_from_ez(rm_d, value = "CI_high")`. 
</div>


### `r user_astronaut()` Robust models of dependent means [(C)]{.lbl}

To compare dependent means with a robust model we can use the `yuend()` function from the [WRS2]{.pkg} package [@R-WRS2; @WRS22020] to get a robust test based on (20% by default) trimmed means. It takes the general form:

```{r, eval = F}
WRS2::yuend(scores_condition_1, scores_condition_2,  tr = .2)
```

In other words, it wants the scores for each condition to be entered as separate variables. For the invisibility data, this means we need the cloak and no cloak scores to be stored as separate variables. Fortunately, this is exactly what we did when we converted the tibble to messy format and stored it as `cloak_messy_tib`. The cloak scores stored in a variable called `cloak` and the no cloak scores stored in a variable called `no_cloak` and we can use `$` to access them (see `discovr_02`. For example, the cloak scores are accessed using `cloak_messy_tib$cloak`.


#### `r robot()` Code example

Putting this together with the `yuend()` function (assuming we don't want to change from a 20% trim) we'd get:

```{r, eval = FALSE}
cloak_rm_rob  <- WRS2::yuend(x = cloak_messy_tib$cloak, y = cloak_messy_tib$no_cloak)
model_parameters(cloak_rm_rob) |> 
  display()
```

#### `r alien()` Alien coding challenge

Use the code example to create (and view) an object called [cloak_rm_rob]{.alt} that contains a robust comparison of the 20% trimmed means of the two cloak conditions.

```{r cloak_rm_rob, exercise = TRUE}

```

```{r cloak_rm_rob-solution}
cloak_rm_rob  <- WRS2::yuend(x = cloak_messy_tib$cloak, y = cloak_messy_tib$no_cloak)
model_parameters(cloak_rm_rob) |> 
  display()
```


```{r, echo = F}
cloak_rm_rob  <- WRS2::yuend(x = cloak_messy_tib$cloak, y = cloak_messy_tib$no_cloak)
cloak_rm_par <- model_parameters(cloak_rm_rob)
```


The output shows that based on this robust test there is a significant difference in mischief scores across the two cloak of invisibility groups, $T_y$(`r value_from_ez(cloak_rm_par, value = "df_error", digits = 0)`) = `r value_from_ez(cloak_rm_par, value = "Estimate")`, `r value_from_ez(cloak_rm_par, value = "p")`. The mean difference is `r value_from_ez(cloak_rm_par, value = "Difference")` with a 95% confidence interval ranging from `r value_from_ez(cloak_rm_par, value = "Difference_CI_low")` to `r value_from_ez(cloak_rm_par, value = "Difference_CI_high")`. Remember that confidence intervals are constructed such that in 95% of samples the intervals contain the true value of the mean difference. So, assuming that this sample's confidence interval is one of the 95 out of 100 that contain the population value, we can say that the true mean difference lies between  `r value_from_ez(cloak_rm_par, value = "Difference_CI_low")` and `r value_from_ez(cloak_rm_par, value = "Difference_CI_high")`. The importance of this interval is that it does not contain zero (both limits are positive), which tells us that the true value of the mean difference is unlikely to be zero (given the assumption we've made). In other words, there is plausibly an effect in the population reflecting more mischievous acts performed when someone is given an invisibility cloak.


<div class="infobox">
  <img src="./images/discovr_hex.png" alt="discovr package hex sticker, female space pirate with gun. Gunsmoke forms the letter R." style="width:100px;height:116px;" class = "img_left">
  
  **A message from Mae Jemstone:**
  
  It's often really useful to compare the mean of two groups. I remember when travelling through *the final frontier* I picked up the *starblind virus*. Luckily, scientists had tested the effect of a drug called *avalon* by comparing symptom levels in people who took the drug against a group who were given a placebo. Those who took avalon had fewer symptoms. Because of that, I knew that I could get back on track by taking some *avalon*! Thanks to your hard work, you'll also now be able to evaluate these sorts of claims - you won't need to rely on people telling you what works, you'll be able to look at the studies, and scrutinise the data. Fantastic work! 
  
</div>

## Resources/References {data-progressive=FALSE}

```{r, child = "./docs/resources.Rmd"}

```


### References
