`compare_performance()`

computes indices of model
performance for different models at once and hence allows comparison of
indices across models.

compare_performance(..., metrics = "all", rank = FALSE, verbose = TRUE)

... | Multiple model objects (also of different classes). |
---|---|

metrics | Can be |

rank | Logical, if |

verbose | Toggle off warnings. |

A data frame (with one row per model) and one column per "index" (see
`metrics`

).

When `rank = TRUE`

, a new column `Performance_Score`

is returned. This
score ranges from 0% to 100%, higher values indicating better model performance.
Note that all score value do not necessarily sum up to 100%. Rather,
calculation is based on normalizing all indices (i.e. rescaling them to a
range from 0 to 1), and taking the mean value of all indices for each model.
This is a rather quick heuristic, but might be helpful as exploratory index.

In particular when models are of different types (e.g. mixed models, classical
linear models, logistic regression, ...), not all indices will be computed
for each model. In case where an index can't be calculated for a specific
model type, this model gets an `NA`

value. All indices that have any
`NA`

s are excluded from calculating the performance score.

There is a `plot()`

-method for `compare_performance()`

,
which creates a "spiderweb" plot, where the different indices are
normalized and larger values indicate better model performance.
Hence, points closer to the center indicate worse fit indices
(see online-documentation
for more details).

There is also a `plot()`

-method implemented in the see-package.

data(iris) lm1 <- lm(Sepal.Length ~ Species, data = iris) lm2 <- lm(Sepal.Length ~ Species + Petal.Length, data = iris) lm3 <- lm(Sepal.Length ~ Species * Petal.Length, data = iris) compare_performance(lm1, lm2, lm3)#> # Comparison of Model Performance Indices #> #> Name | Model | AIC | BIC | R2 | R2 (adj.) | RMSE | Sigma #> -------------------------------------------------------------------- #> lm1 | lm | 231.452 | 243.494 | 0.619 | 0.614 | 0.510 | 0.515 #> lm2 | lm | 106.233 | 121.286 | 0.837 | 0.833 | 0.333 | 0.338 #> lm3 | lm | 106.767 | 127.842 | 0.840 | 0.835 | 0.330 | 0.336compare_performance(lm1, lm2, lm3, rank = TRUE)#> # Comparison of Model Performance Indices #> #> Name | Model | AIC | BIC | R2 | R2 (adj.) | RMSE | Sigma | Performance-Score #> ---------------------------------------------------------------------------------------- #> lm2 | lm | 106.233 | 121.286 | 0.837 | 0.833 | 0.333 | 0.338 | 99.10% #> lm3 | lm | 106.767 | 127.842 | 0.840 | 0.835 | 0.330 | 0.336 | 99.03% #> lm1 | lm | 231.452 | 243.494 | 0.619 | 0.614 | 0.510 | 0.515 | 0.00%if (require("lme4")) { m1 <- lm(mpg ~ wt + cyl, data = mtcars) m2 <- glm(vs ~ wt + mpg, data = mtcars, family = "binomial") m3 <- lmer(Petal.Length ~ Sepal.Length + (1 | Species), data = iris) compare_performance(m1, m2, m3) }#> Warning: When comparing models, please note that probably not all models were fit from #> same data.#> # Comparison of Model Performance Indices #> #> Name | Model | AIC | BIC | RMSE | Sigma | R2 | R2 (adj.) | Tjur's R2 | Log_loss | Score_log | Score_spherical | PCP | R2 (cond.) | R2 (marg.) | ICC #> --------------------------------------------------------------------------------------------------------------------------------------------------------------------- #> m1 | lm | 156.010 | 161.873 | 2.444 | 2.568 | 0.830 | 0.819 | | | | | | | | #> m2 | glm | 31.298 | 35.695 | 0.359 | 0.934 | | | 0.478 | 0.395 | -14.903 | 0.095 | 0.743 | | | #> m3 | lmerMod | 77.320 | 89.362 | 0.279 | 0.283 | | | | | | | | 0.972 | 0.096 | 0.969