Compute Cramer's V, phi (\(\phi\)), Cohen's w, normalized Chi (\(\chi\)), Pearson's contingency coefficient, Odds ratios, Risk ratios, Cohen's h and Cohen's g for contingency tables or goodness-of-fit. See details.
Usage
phi(x, y = NULL, ci = 0.95, alternative = "greater", adjust = FALSE, ...)
cohens_w(x, y = NULL, ci = 0.95, alternative = "greater", ...)
cramers_v(x, y = NULL, ci = 0.95, alternative = "greater", adjust = FALSE, ...)
normalized_chi(x, y = NULL, ci = 0.95, alternative = "greater", ...)
pearsons_c(
x,
y = NULL,
ci = 0.95,
alternative = "greater",
adjust = FALSE,
...
)
oddsratio(x, y = NULL, ci = 0.95, alternative = "two.sided", log = FALSE, ...)
riskratio(x, y = NULL, ci = 0.95, alternative = "two.sided", log = FALSE, ...)
cohens_h(x, y = NULL, ci = 0.95, alternative = "two.sided", ...)
cohens_g(x, y = NULL, ci = 0.95, alternative = "two.sided", ...)Arguments
- x
a numeric vector or matrix.
xandycan also both be factors.- y
a numeric vector; ignored if
xis a matrix. Ifxis a factor,yshould be a factor of the same length.- ci
Confidence Interval (CI) level
- alternative
a character string specifying the alternative hypothesis; Controls the type of CI returned:
"greater"(two-sided CI; default for Cramer's V, phi (\(\phi\)), and Cohen's w),"two.sided"(default for OR, RR, Cohen's h and Cohen's g) or"less"(one-sided CI). Partial matching is allowed (e.g.,"g","l","two"...). See One-Sided CIs in effectsize_CIs.- adjust
Should the effect size be bias-corrected? Defaults to
FALSE.- ...
Arguments passed to
stats::chisq.test(), such aspfor goodness-of-fit. Ignored forcohens_g().- log
Take in or output the log of the ratio (such as in logistic models).
Value
A data frame with the effect size (Cramers_v, phi (possibly with
the suffix _adjusted), Cohens_w, normalized_chi, Odds_ratio,
Risk_ratio (possibly with the prefix log_), Cohens_h, or Cohens_g)
and its CIs (CI_low and CI_high).
Details
Cramer's V, phi (\(\phi\)), Cohen's w, and Pearson's C are effect
sizes for tests of independence in 2D contingency tables. For 2-by-2 tables,
Cramer's V, phi and Cohen's w are identical, and are equal to the simple
correlation between two dichotomous variables, ranging between 0 (no
dependence) and 1 (perfect dependence). For larger tables, Cramer's V or
Pearson's C should be used, as they are bounded between 0-1. Cohen's w
can also be used, but since it is not bounded at 1 (can be larger) its
interpretation is more difficult.
For goodness-of-fit in 1D tables Cohen's W, normalized Chi (\(\chi\)) or
Pearson's C can be used. Cohen's w has no upper bound (can be arbitrarily
large, depending on the expected distribution). Normalized Chi is an adjusted
Cohen's w, accounting for the expected distribution, making it bounded
between 0-1. Pearson's C is also bounded between 0-1.
To summarize, for correlation-like effect sizes, we recommend:
For a 2x2 table, use
phi()For larger tables, use
cramers_v()For goodness-of-fit, use
normalized_chi()
For 2-by-2 contingency tables, Odds ratios, Risk ratios and Cohen's h can also be estimated. Note that these are computed with each column representing the different groups, and the first column representing the treatment group and the second column baseline (or control). Effects are given astreatment / control. If you wish you use rows as groups you must pass a transposed table, or switch thexandyarguments.
Cohen's g is an effect size for dependent (paired) contingency tables ranging between 0 (perfect symmetry) and 0.5 (perfect asymmetry) (seestats::mcnemar.test()).
Confidence Intervals for Cohen's g, OR, RR and Cohen's h
For Cohen's g, confidence intervals are based on the proportion (\(P = g
+ 0.5\)) confidence intervals returned by stats::prop.test() (minus 0.5),
which give a good close approximation.
For Odds ratios, Risk ratios and Cohen's h, confidence intervals are
estimated using the standard normal parametric method (see Katz et al., 1978;
Szumilas, 2010).
See Confidence (Compatibility) Intervals (CIs), CIs and Significance Tests,
and One-Sided CIs sections for phi, Cohen's w, Cramer's V,
Pearson's C, and normalized Chi.
Confidence (Compatibility) Intervals (CIs)
Unless stated otherwise, confidence (compatibility) intervals (CIs) are
estimated using the noncentrality parameter method (also called the "pivot
method"). This method finds the noncentrality parameter ("ncp") of a
noncentral t, F, or \(\chi^2\) distribution that places the observed
t, F, or \(\chi^2\) test statistic at the desired probability point of
the distribution. For example, if the observed t statistic is 2.0, with 50
degrees of freedom, for which cumulative noncentral t distribution is t =
2.0 the .025 quantile (answer: the noncentral t distribution with ncp =
.04)? After estimating these confidence bounds on the ncp, they are
converted into the effect size metric to obtain a confidence interval for the
effect size (Steiger, 2004).
For additional details on estimation and troubleshooting, see effectsize_CIs.
CIs and Significance Tests
"Confidence intervals on measures of effect size convey all the information
in a hypothesis test, and more." (Steiger, 2004). Confidence (compatibility)
intervals and p values are complementary summaries of parameter uncertainty
given the observed data. A dichotomous hypothesis test could be performed
with either a CI or a p value. The 100 (1 - \(\alpha\))% confidence
interval contains all of the parameter values for which p > \(\alpha\)
for the current data and model. For example, a 95% confidence interval
contains all of the values for which p > .05.
Note that a confidence interval including 0 does not indicate that the null
(no effect) is true. Rather, it suggests that the observed data together with
the model and its assumptions combined do not provided clear evidence against
a parameter value of 0 (same as with any other value in the interval), with
the level of this evidence defined by the chosen \(\alpha\) level (Rafi &
Greenland, 2020; Schweder & Hjort, 2016; Xie & Singh, 2013). To infer no
effect, additional judgments about what parameter values are "close enough"
to 0 to be negligible are needed ("equivalence testing"; Bauer & Kiesser,
1996).
References
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd Ed.). New York: Routledge.
Katz, D. J. S. M., Baptista, J., Azen, S. P., & Pike, M. C. (1978). Obtaining confidence intervals for the risk ratio in cohort studies. Biometrics, 469-474.
Szumilas, M. (2010). Explaining odds ratios. Journal of the Canadian academy of child and adolescent psychiatry, 19(3), 227.
Johnston, J. E., Berry, K. J., & Mielke Jr, P. W. (2006). Measures of effect size for chi-squared and likelihood-ratio goodness-of-fit tests. Perceptual and motor skills, 103(2), 412-414.
Rosenberg, M. S. (2010). A generalized formula for converting chi-square tests to effect sizes for meta-analysis. PloS one, 5(4), e10059.
See also
chisq_to_phi() for details regarding estimation and CIs.
Other effect size indices:
cles(),
cohens_d(),
effectsize.BFBayesFactor(),
eta_squared(),
rank_biserial()
Examples
## 2-by-2 tables
## -------------
RCT <-
matrix(c(71, 30,
50, 100), nrow = 2, byrow = TRUE,
dimnames = list(
Diagnosis = c("Sick", "Recovered"),
Group = c("Treatment", "Control")))
RCT # note groups are COLUMNS
#> Group
#> Diagnosis Treatment Control
#> Sick 71 30
#> Recovered 50 100
phi(RCT)
#> Phi | 95% CI
#> -------------------
#> 0.36 | [0.26, 1.00]
#>
#> - One-sided CIs: upper bound fixed at [1.00].
pearsons_c(RCT)
#> Pearson's C | 95% CI
#> --------------------------
#> 0.34 | [0.25, 1.00]
#>
#> - One-sided CIs: upper bound fixed at [1.00].
oddsratio(RCT)
#> Odds ratio | 95% CI
#> -------------------------
#> 4.73 | [2.74, 8.17]
oddsratio(RCT, alternative = "greater")
#> Odds ratio | 95% CI
#> ------------------------
#> 4.73 | [3.00, Inf]
#>
#> - One-sided CIs: upper bound fixed at [Inf].
riskratio(RCT)
#> Risk ratio | 95% CI
#> -------------------------
#> 2.54 | [1.87, 3.45]
cohens_h(RCT)
#> Cohen's h | 95% CI
#> ------------------------
#> 0.74 | [0.50, 0.99]
## Larger tables
## -------------
M <-
matrix(c(150, 100, 165,
130, 50, 65,
35, 10, 2,
55, 40, 25), nrow = 4,
dimnames = list(
Music = c("Pop", "Rock", "Jazz", "Classic"),
Study = c("Psych", "Econ", "Law")))
M
#> Study
#> Music Psych Econ Law
#> Pop 150 50 2
#> Rock 100 65 55
#> Jazz 165 35 40
#> Classic 130 10 25
cohens_w(M)
#> Cohen's w | 95% CI
#> ------------------------
#> 0.34 | [0.27, 1.41]
#>
#> - One-sided CIs: upper bound fixed at [1.41~].
cramers_v(M)
#> Cramer's V | 95% CI
#> -------------------------
#> 0.24 | [0.19, 1.00]
#>
#> - One-sided CIs: upper bound fixed at [1.00].
pearsons_c(M)
#> Pearson's C | 95% CI
#> --------------------------
#> 0.32 | [0.26, 1.00]
#>
#> - One-sided CIs: upper bound fixed at [1.00].
## Goodness of fit
## ---------------
Smoking_ASD <- as.table(c(ASD = 17, ASP = 11, TD = 640))
normalized_chi(Smoking_ASD)
#> Norm. Chi | 95% CI
#> ------------------------
#> 0.94 | [0.89, 1.00]
#>
#> - Adjusted for uniform expected probabilities.
#> - One-sided CIs: upper bound fixed at [1.00].
cohens_w(Smoking_ASD)
#> Cohen's w | 95% CI
#> -----------------------
#> 1.33 | [1.26, Inf]
#>
#> - One-sided CIs: upper bound fixed at [Inf].
pearsons_c(Smoking_ASD)
#> Pearson's C | 95% CI
#> --------------------------
#> 0.80 | [0.78, 1.00]
#>
#> - One-sided CIs: upper bound fixed at [1.00].
# Use custom expected values:
normalized_chi(Smoking_ASD, p = c(0.015, 0.010, 0.975))
#> Norm. Chi | 95% CI
#> ------------------------
#> 0.01 | [0.00, 1.00]
#>
#> - Adjusted for non-uniform expected probabilities.
#> - One-sided CIs: upper bound fixed at [1.00].
cohens_w(Smoking_ASD, p = c(0.015, 0.010, 0.975))
#> Cohen's w | 95% CI
#> -----------------------
#> 0.11 | [0.03, Inf]
#>
#> - One-sided CIs: upper bound fixed at [Inf].
pearsons_c(Smoking_ASD, p = c(0.015, 0.010, 0.975))
#> Pearson's C | 95% CI
#> --------------------------
#> 0.11 | [0.03, 1.00]
#>
#> - One-sided CIs: upper bound fixed at [1.00].
## Dependent (Paired) Contingency Tables
## -------------------------------------
Performance <-
matrix(c(794, 150,
86, 570), nrow = 2,
dimnames = list(
"1st Survey" = c("Approve", "Disapprove"),
"2nd Survey" = c("Approve", "Disapprove")))
Performance
#> 2nd Survey
#> 1st Survey Approve Disapprove
#> Approve 794 86
#> Disapprove 150 570
cohens_g(Performance)
#> Cohen's g | 95% CI
#> ------------------------
#> 0.14 | [0.07, 0.19]