# Standardized Mean Differences for Repeated Measures

Source:`R/repeated_measures_d.R`

`repeated_measures_d.Rd`

Compute effect size indices for standardized mean differences in repeated
measures data. Pair with any reported `stats::t.test(paired = TRUE)`

.

In a repeated-measures design, the same subjects are measured in multiple
conditions or time points. Unlike the case of independent groups, there are
multiple sources of variation that can be used to standardized the
differences between the means of the conditions / times.

## Usage

```
repeated_measures_d(
x,
y,
data = NULL,
mu = 0,
method = c("rm", "av", "z", "b", "d", "r"),
adjust = TRUE,
ci = 0.95,
alternative = "two.sided",
verbose = TRUE,
...
)
rm_d(
x,
y,
data = NULL,
mu = 0,
method = c("rm", "av", "z", "b", "d", "r"),
adjust = TRUE,
ci = 0.95,
alternative = "two.sided",
verbose = TRUE,
...
)
```

## Arguments

- x, y
Paired numeric vectors, or names of ones in

`data`

.`x`

can also be a formula:`Pair(x,y) ~ 1`

for wide data.`y ~ condition | id`

for long data, possibly with repetitions.

- data
An optional data frame containing the variables.

- mu
a number indicating the true value of the mean (or difference in means if you are performing a two sample test).

- method
Method of repeated measures standardized differences. See details.

- adjust
Apply Hedges' small-sample bias correction? See

`hedges_g()`

.- ci
Confidence Interval (CI) level

- alternative
a character string specifying the alternative hypothesis; Controls the type of CI returned:

`"two.sided"`

(default, two-sided CI),`"greater"`

or`"less"`

(one-sided CI). Partial matching is allowed (e.g.,`"g"`

,`"l"`

,`"two"`

...). See*One-Sided CIs*in effectsize_CIs.- verbose
Toggle warnings and messages on or off.

- ...
Arguments passed to or from other methods. When

`x`

is a formula, these can be`subset`

and`na.action`

.

## Standardized Mean Differences for Repeated Measures

Unlike Cohen's d for independent groups, where standardization naturally is done by the (pooled) population standard deviation (cf. Glass’s \(\Delta\)), when measured across two conditions are dependent, there are many more options for what error term to standardize by. Additionally, some options allow for data to be replicated (many measurements per condition per individual), others require a single observation per condition per individual (aka, paired data; so replications are aggregated).

(It should be noted that all of these have awful and confusing notations.)

Standardize by...

**Difference Score Variance: \(d_{z}\)**(*Requires paired data*) - This is akin to computing difference scores for each individual and then computing a one-sample Cohen's*d*(Cohen, 1988, pp. 48; see examples).**Within-Subject Variance: \(d_{rm}\)**(*Requires paired data*) - Cohen suggested adjusting \(d_{z}\) to estimate the "standard" between-subjects*d*by a factor of \(\sqrt{2(1-r)}\), where*r*is the Pearson correlation between the paired measures (Cohen, 1988, pp. 48).**Control Variance: \(d_{b}\) (aka Becker's**(*d*)*Requires paired data*) - Standardized by the variance of the control condition (or in a pre- post-treatment setting, the pre-treatment condition). This is akin to Glass'*delta*(`glass_delta()`

) (Becker, 1988). Note that this is taken here as the*second*condition (`y`

).**Average Variance: \(d_{av}\)**(*Requires paired data*) - Instead of standardizing by the variance in the of the control (or pre) condition, Cumming suggests standardizing by the average variance of the two paired conditions (Cumming, 2013, pp. 291).**All Variance: Just \(d\)**- This is the same as computing a standard independent-groups Cohen's*d*(Cohen, 1988). Note that CIs*do*account for the dependence, and so are typically more narrow (see examples).**Residual Variance: \(d_{r}\)**(*Requires data with replications*) - Divide by the pooled variance after all individual differences have been partialled out (i.e., the residual/level-1 variance in an ANOVA or MLM setting). In between-subjects designs where each subject contributes a single response, this is equivalent to classical Cohen’s d. Priors in the`BayesFactor`

package are defined on this scale (Rouder et al., 2012).

Note that for paired data, when the two conditions have equal variance, \(d_{rm}\), \(d_{av}\), \(d_{b}\) are equal to \(d\).

## Confidence (Compatibility) Intervals (CIs)

Confidence intervals are estimated using the standard normal parametric method (see Algina & Keselman, 2003; Becker, 1988; Cooper et al., 2009; Hedges & Olkin, 1985; Pustejovsky et al., 2014).

## CIs and Significance Tests

"Confidence intervals on measures of effect size convey all the information
in a hypothesis test, and more." (Steiger, 2004). Confidence (compatibility)
intervals and p values are complementary summaries of parameter uncertainty
given the observed data. A dichotomous hypothesis test could be performed
with either a CI or a p value. The 100 (1 - \(\alpha\))% confidence
interval contains all of the parameter values for which *p* > \(\alpha\)
for the current data and model. For example, a 95% confidence interval
contains all of the values for which p > .05.

Note that a confidence interval including 0 *does not* indicate that the null
(no effect) is true. Rather, it suggests that the observed data together with
the model and its assumptions combined do not provided clear evidence against
a parameter value of 0 (same as with any other value in the interval), with
the level of this evidence defined by the chosen \(\alpha\) level (Rafi &
Greenland, 2020; Schweder & Hjort, 2016; Xie & Singh, 2013). To infer no
effect, additional judgments about what parameter values are "close enough"
to 0 to be negligible are needed ("equivalence testing"; Bauer & Kiesser,
1996).

## Plotting with `see`

The `see`

package contains relevant plotting functions. See the plotting vignette in the `see`

package.

## References

Algina, J., & Keselman, H. J. (2003). Approximate confidence intervals for effect sizes. Educational and Psychological Measurement, 63(4), 537-553.

Becker, B. J. (1988). Synthesizing standardized mean‐change measures. British Journal of Mathematical and Statistical Psychology, 41(2), 257-278.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd Ed.). New York: Routledge.

Cooper, H., Hedges, L., & Valentine, J. (2009). Handbook of research synthesis and meta-analysis. Russell Sage Foundation, New York.

Cumming, G. (2013). Understanding the new statistics: Effect sizes, confidence intervals, and meta-analysis. Routledge.

Hedges, L. V. & Olkin, I. (1985). Statistical methods for meta-analysis. Orlando, FL: Academic Press.

Pustejovsky, J. E., Hedges, L. V., & Shadish, W. R. (2014). Design-comparable effect sizes in multiple baseline designs: A general modeling framework. Journal of Educational and Behavioral Statistics, 39(5), 368-393.

Rouder, J. N., Morey, R. D., Speckman, P. L., & Province, J. M. (2012). Default Bayes factors for ANOVA designs. Journal of mathematical psychology, 56(5), 356-374.

## See also

`cohens_d()`

, and `lmeInfo::g_mlm()`

and `emmeans::effsize()`

for
more flexible methods.

Other standardized differences:
`cohens_d()`

,
`mahalanobis_d()`

,
`means_ratio()`

,
`p_superiority()`

,
`rank_biserial()`

## Examples

```
# Paired data -------
data("sleep")
sleep2 <- reshape(sleep,
direction = "wide",
idvar = "ID", timevar = "group"
)
repeated_measures_d(Pair(extra.1, extra.2) ~ 1, data = sleep2)
#> d (rm) | 95% CI
#> -----------------------
#> -0.75 | [-1.17, -0.33]
#>
#> - Adjusted for small sample bias.
# Same as:
# repeated_measures_d(sleep$extra[sleep$group==1],
# sleep$extra[sleep$group==2])
# repeated_measures_d(extra ~ group | ID, data = sleep)
# More options:
repeated_measures_d(Pair(extra.1, extra.2) ~ 1, data = sleep2, mu = -1)
#> d (rm) | 95% CI
#> ----------------------
#> -0.28 | [-0.65, 0.09]
#>
#> - Adjusted for small sample bias.
#> - Deviation from a difference of -1.
repeated_measures_d(Pair(extra.1, extra.2) ~ 1, data = sleep2, alternative = "less")
#> d (rm) | 95% CI
#> ----------------------
#> -0.75 | [-Inf, -0.40]
#>
#> - Adjusted for small sample bias.
#> - One-sided CIs: lower bound fixed at [-Inf].
# Other methods
repeated_measures_d(Pair(extra.1, extra.2) ~ 1, data = sleep2, method = "av")
#> d (av) | 95% CI
#> -----------------------
#> -0.76 | [-1.13, -0.39]
#>
#> - Adjusted for small sample bias.
repeated_measures_d(Pair(extra.1, extra.2) ~ 1, data = sleep2, method = "b")
#> Becker's d | 95% CI
#> ---------------------------
#> -0.72 | [-1.20, -0.24]
#>
#> - Adjusted for small sample bias.
repeated_measures_d(Pair(extra.1, extra.2) ~ 1, data = sleep2, method = "d")
#> Cohen's d | 95% CI
#> --------------------------
#> -0.80 | [-1.29, -0.30]
#>
#> - Adjusted for small sample bias.
repeated_measures_d(Pair(extra.1, extra.2) ~ 1, data = sleep2, method = "z", adjust = FALSE)
#> d (z) | 95% CI
#> ----------------------
#> -1.28 | [-2.12, -0.45]
# d_z is the same as Cohen's d for one sample (of individual difference):
cohens_d(extra.1 - extra.2 ~ 1, data = sleep2)
#> Cohen's d | 95% CI
#> --------------------------
#> -1.28 | [-2.12, -0.41]
# Repetition data -----------
data("rouder2016")
# For rm, ad, z, b, data is aggregated
repeated_measures_d(rt ~ cond | id, data = rouder2016)
#> The rm standardized difference requires paired data,
#> but data contains more than one observation per design cell.
#> Aggregating data using `mean()`.
#> d (rm) | 95% CI
#> -----------------------
#> -0.80 | [-1.06, -0.53]
#>
#> - Adjusted for small sample bias.
# same as:
rouder2016_wide <- tapply(rouder2016[["rt"]], rouder2016[1:2], mean)
repeated_measures_d(rouder2016_wide[, 1], rouder2016_wide[, 2])
#> d (rm) | 95% CI
#> -----------------------
#> -0.80 | [-1.06, -0.53]
#>
#> - Adjusted for small sample bias.
# For r or d, data is not aggragated:
repeated_measures_d(rt ~ cond | id, data = rouder2016, method = "r")
#> d (r) | 95% CI
#> ----------------------
#> -0.26 | [-0.33, -0.18]
#>
#> - Adjusted for small sample bias.
repeated_measures_d(rt ~ cond | id, data = rouder2016, method = "d", adjust = FALSE)
#> Cohen's d | 95% CI
#> --------------------------
#> -0.25 | [-0.32, -0.18]
# d is the same as Cohen's d for two independent groups:
cohens_d(rt ~ cond, data = rouder2016, ci = NULL)
#> Cohen's d
#> ---------
#> -0.25
#>
#> - Estimated using pooled SD.
```