Changelog
Source:NEWS.md
performance 0.11.1
Breaking
 Aliases
posterior_predictive_check()
andcheck_posterior_predictions()
forcheck_predictions()
are deprecated.
General
Improved documentation and new vignettes added.
check_model()
gets abase_size
argument, to set the base font size for plots.check_predictions()
forstanreg
andbrmsfit
models now returns plots in the usual style as for other models and no longer returns plots frombayesplot::pp_check()
.
Bug fixes

check_model()
now falls back on normal QQ plots when a model is not supported by the DHARMa package and simulated residuals cannot be calculated.
performance 0.11.0
CRAN release: 20240322
New functions
simulate_residuals()
andcheck_residuals()
, to simulate and check residuals from generalized linear (mixed) models. Simulating residuals is based on the DHARMa package, and objects returned bysimulate_residuals()
inherit from theDHARMa
class, and thus can be used with any functions from the DHARMa package. However, there are also implementations in the performance package, such ascheck_overdispersion()
,check_zeroinflation()
,check_outliers()
orcheck_model()
.Plots for
check_model()
have been improved. The QQ plots are now based on simulated residuals from the DHARMa package for nonGaussian models, thus providing more accurate and informative plots. The halfnormal QQ plot for generalized linear models can still be obtained by setting the new argumentresidual_type = "normal"
.
Following functions now support simulated residuals (from
simulate_residuals()
) resp. objects returned fromDHARMa::simulateResiduals()
:
General
Improved error messages for
check_model()
when QQplots cannot be created.check_distribution()
is more stable for possibly sparse data.
Bug fixes
Fixed issue in
check_normality()
for ttests.Fixed issue in
check_itemscale()
for data frame inputs, whenfactor_index
was not a named vector.
performance 0.10.9
CRAN release: 20240217
Changes
r2()
for models of classglmmTMB
without random effects now returns the correct rsquared value for nonmixed models.check_itemscale()
now also accepts data frames as input. In this case,factor_index
must be specified, which must be a numeric vector of same length as number of columns inx
, where each element is the index of the factor to which the respective column inx
.check_itemscale()
gets aprint_html()
method.Clarification in the documentation of the
estimator
argument forperformance_aic()
.Improved plots for overdispersionchecks for negativebinomial models from package glmmTMB (affects
check_overdispersion()
andcheck_model()
).Improved detection rates for singularity in
check_singularity()
for models from package glmmTMB.For model of class
glmmTMB
, deviance residuals are now used in thecheck_model()
plot.Improved (better to understand) error messages for
check_model()
,check_collinearity()
andcheck_outliers()
for models with nonnumeric response variables.r2_kullback()
now gives an informative error for nonsupported models.
Bug fixes
Fixed issue in
binned_residuals()
for models with binary outcome, where in rare occasions empty bins could occur.performance_score()
should no longer fail for models where scoring rules can’t be calculated. Instead, an informative message is returned.check_outliers()
now properly accept thepercentage_central
argument when using the"mcd"
method.Fixed edge cases in
check_collinearity()
andcheck_outliers()
for models with response variables of classesDate
,POSIXct
,POSIXlt
ordifftime
.Fixed issue with
check_model()
for models of package quantreg.
performance 0.10.8
CRAN release: 20231030
Changes
 Changed behaviour of
check_predictions()
for models from binomial family, to get comparable plots for different ways of outcome specification. Now, if the outcome is a proportion, or defined as matrix of trials and successes, the produced plots are the same (because the models should be the same, too).
Bug fixes
Fixed CRAN check errors.
Fixed issue with
binned_residuals()
for models with binomial family, where the outcome was a proportion.
performance 0.10.7
CRAN release: 20231027
Breaking changes

binned_residuals()
gains a few new arguments to control the residuals used for the test, as well as different options to calculate confidence intervals (namely,ci_type
,residuals
,ci
anditerations
). The default values to compute binned residuals have changed. Default residuals are now “deviance” residuals (and no longer “response” residuals). Default confidence intervals are now “exact” intervals (and no longer based on Gaussian approximation). Useci_type = "gaussian"
andresiduals = "response"
to get the old defaults.
Changes to functions

binned_residuals()
 likecheck_model()
 gains ashow_dots
argument to show or hide data points that lie inside error bounds. This is particular useful for models with many observations, where generating the plot would be very slow.
performance 0.10.6
Changes to functions

check_outliers()
for method"ics"
now detects number of available cores for parallel computing via the"mc.cores"
option. This is more robust than the previous method, which usedparallel::detectCores()
. Now you should set the number of cores viaoptions(mc.cores = 4)
.
Bug fixes
 Fixed issues is
check_model()
for models that used data sets with variables of class"haven_labelled"
.
performance 0.10.5
CRAN release: 20230912
Changes to functions
More informative message for
test_*()
functions that “nesting” only refers to fixed effects parameters and currently ignores random effects when detecting nested models.check_outliers()
for"ICS"
method is now more stable and less likely to fail.check_convergence()
now works for parsnip_glm
models.
Bug fixes

check_collinearity()
did not work for hurdle or zeroinflated models of package pscl when model had no explicitly defined formula for the zeroinflation model.
performance 0.10.4
CRAN release: 20230602
Changes to functions
icc()
andr2_nakagawa()
gain aci_method
argument, to either calculate confidence intervals usingboot::boot()
(instead oflmer::bootMer()
) whenci_method = "boot"
or analytical confidence intervals (ci_method = "analytical"
). Useci_method = "boot"
when the default method fails to compute confidence intervals and useci_method = "analytical"
if bootstrapped intervals cannot be calculated at all. Note that the default computation method is preferred.check_predictions()
accepts abandwidth
argument (smoothing bandwidth), which is passed down to theplot()
methods densityestimation.check_predictions()
gains atype
argument, which is passed down to theplot()
method to change plottype (density or discrete dots/intervals). By default,type
is set to"default"
for models without discrete outcomes, and elsetype = "discrete_interval"
.performance_accuracy()
now includes confidence intervals, and reports those by default (the standard error is no longer reported, but still included).
Bug fixes
 Fixed issue in
check_collinearity()
for fixest models that usedi()
to create interactions in formulas.
performance 0.10.3
CRAN release: 20230407
New functions

item_discrimination()
, to calculate the discrimination of a scale’s items.
Support for new models
model_performance()
,check_overdispersion()
,check_outliers()
andr2()
now work with objects of classfixest_multi
(@etiennebacher, #554).model_performance()
can now return the “Weak instruments” statistic and pvalue for models of classivreg
withmetrics = "weak_instruments"
(@etiennebacher, #560).Support for
mclogit
models.
Changes to functions
test_*()
functions now automatically fit a nullmodel when only one model objects was provided for testing multiple models.Warnings in
model_performance()
for unsupported objects of classBFBayesFactor
can now be suppressed withverbose = FALSE
.check_predictions()
no longer fails with issues whenre_formula = NULL
for mixed models, but instead gives a warning and tries to compute posterior predictive checks withre_formuka = NA
.check_outliers()
now also works for metaanalysis models from packages metafor and meta.plot()
forperformance::check_model()
no longer produces a normal QQ plot for GLMs. Instead, it now shows a halfnormal QQ plot of the absolute value of the standardized deviance residuals.
Bug fixes
 Fixed issue in
print()
method forcheck_collinearity()
, which could mix up the correct order of parameters.
performance 0.10.2
CRAN release: 20230112
General
 Revised usage of
insight::get_data()
to meet forthcoming changes in the insight package.
Changes to functions

check_collinearity()
now acceptsNULL
for theci
argument.
Bug fixes
 Fixed issue in
item_difficulty()
with detecting the maximum values of an item set. Furthermore,item_difficulty()
gets amaximum_value
argument in case no item contains the maximum value due to missings.
performance 0.10.1
CRAN release: 20221125
Changes to functions
icc()
andr2_nakagawa()
getci
anditerations
arguments, to compute confidence intervals for the ICC resp. R2, based on bootstrapped sampling.r2()
getsci
, to compute (analytical) confidence intervals for the R2.The model underlying
check_distribution()
was now also trained to detect cauchy, halfcauchy and inversegamma distributions.model_performance()
now allows to include the ICC for Bayesian models.
Bug fixes
verbose
didn’t work forr2_bayes()
withBFBayesFactor
objects.Fixed issues in
check_model()
for models with convergence issues that lead toNA
values in residuals.Fixed bug in
check_outliers
whereby passing multiple elements to the threshold list generated an error (#496).test_wald()
now warns the user about inappropriate F test and callstest_likelihoodratio()
for binomial models.Fixed edge case for usage of
parellel::detectCores()
incheck_outliers()
.
performance 0.10.0
CRAN release: 20221003
Breaking Change
The minimum needed R version has been bumped to
3.6
.The alias
performance_lrt()
was removed. Usetest_lrt()
resp.test_likelihoodratio()
.
New functions
 Following functions were moved from package parameters to performance:
check_sphericity_bartlett()
,check_kmo()
,check_factorstructure()
andcheck_clusterstructure()
.
Changes to functions
check_normality()
,check_homogeneity()
andcheck_symmetry()
now works forhtest
objects.Print method for
check_outliers()
changed significantly: now states the methods, thresholds, and variables used, reports outliers per variable (for univariate methods) as well as any observation flagged for several variables/methods. Includes a new optional ID argument to add along the row number in the output (@rempsyc #443).check_outliers()
now uses more conventional outlier thresholds. TheIQR
and confidence interval methods now gain improved distance scores that are continuous instead of discrete.
Bug Fixes
Fixed wrong zscore values when using a vector instead of a data frame in
check_outliers()
(#476).Fixed
cronbachs_alpha()
for objects fromparameters::principal_component()
.
performance 0.9.2
CRAN release: 20220810
General
print()
methods formodel_performance()
andcompare_performance()
get alayout
argument, which can be"horizontal"
(default) or"vertical"
, to switch the layout of the printed table.Improved speed performance for
check_model()
and some otherperformance_*()
functions.Improved support for models of class
geeglm
.
Changes to functions

check_model()
gains ashow_dots
argument, to show or hide data points. This is particular useful for models with many observations, where generating the plot would be very slow.
Bug Fixes
 Fixes wrong column names in
model_performance()
output forkmeans
objects (#453)
performance 0.9.1
CRAN release: 20220620
Breaking
 The formerly “conditional” ICC in
icc()
is now named “unadjusted” ICC.
New functions

performance_cv()
for crossvalidated model performance.
Changes to functions
check_overdispersion()
gets aplot()
method.check_outliers()
now also works for models of classesgls
andlme
. As a consequence,check_model()
will no longer fail for these models.check_collinearity()
now includes the confidence intervals for the VIFs and tolerance values.model_performance()
now also includes withinsubject R2 measures, where applicable.Improved handling of random effects in
check_normality()
(i.e. when argumenteffects = "random"
).
Bug fixes
check_predictions()
did not work for GLMs with matrixresponse.check_predictions()
did not work for logistic regression models (i.e. models with binary response) from package glmmTMBitem_split_half()
did not work when the input data frame or matrix only contained two columns.Fixed wrong computation of
BIC
inmodel_performance()
when models had transformed response values.Fixed issues in
check_model()
for GLMs with matrixresponse.
performance 0.9.0
CRAN release: 20220330
New functions

check_concurvity()
, which returns GAM concurvity measures (comparable to collinearity checks).
Changes to functions
Check functions
check_predictions()
,check_collinearity()
andcheck_outliers()
now support (mixed) regression models fromBayesFactor
.check_zeroinflation()
now also works forlme4::glmer.nb()
models.check_collinearity()
better supports GAM models.
Test functions
test_performance()
now callstest_lrt()
ortest_wald()
instead oftest_vuong()
when package CompQuadForm is missing.test_performance()
andtest_lrt()
now compute the corrected loglikelihood when models with transformed response variables (such as log or sqrttransformations) are passed to the functions.
Model performance functions
performance_aic()
now corrects the AIC value for models with transformed response variables. This also means that comparing models usingcompare_performance()
allows comparisons of AIC values for models with and without transformed response variables.Also,
model_performance()
now corrects both AIC and BIC values for models with transformed response variables.
Plotting and printing
The
print()
method forbinned_residuals()
now prints a short summary of the results (and no longer generates a plot). Aplot()
method was added to generate plots.
The
plot()
output forcheck_model()
was revised:For binomial models, the constant variance plot was omitted, and a binned residuals plot included.
The densityplot that showed normality of residuals was replaced by the posterior predictive check plot.
Bug fixes
model_performance()
for models from lme4 did not report AICc when requested.r2_nakagawa()
messed up order of group levels whenby_group
wasTRUE
.
performance 0.8.0
CRAN release: 20211001
Breaking Changes
The
ci
level inr2()
for Bayesian models now defaults to0.95
, to be in line with the latest changes in the bayestestR package.S3method dispatch for
pp_check()
was revised, to avoid problems with the bayesplot package, where the generic is located.
General
Minor revisions to wording for messages from some of the checkfunctions.
posterior_predictive_check()
andcheck_predictions()
were added as aliases forpp_check()
.
New functions

check_multimodal()
andcheck_heterogeneity_bias()
. These functions will be removed from the parameters packages in the future.
Changes to functions

r2()
for linear models can now compute confidence intervals, via theci
argument.
Bug fixes
Fixed issues in
check_model()
for Bayesian models.Fixed issue in
pp_check()
for models with transformed response variables, so now predictions and observed response values are on the same (transformed) scale.
performance 0.7.3
CRAN release: 20210721
Changes to functions
check_outliers()
has newci
(orhdi
,eti
) method to filter based on Confidence/Credible intervals.compare_performance()
now also accepts a list of model objects.performance_roc()
now also works for binomial models from other classes than glm.Several functions, like
icc()
orr2_nakagawa()
, now have anas.data.frame()
method.check_collinearity()
now correctly handles objects from forthcoming afex update.
performance 0.7.2
CRAN release: 20210517
New functions

performance_mae()
to calculate the mean absolute error.
Bug fixes
Fixed issue with
"data length differs from size of matrix"
warnings in examples in forthcoming R 4.2.Fixed issue in
check_normality()
for models with sample size larger than
5.000 observations.
Fixed issue in
check_model()
for glmmTMB models.Fixed issue in
check_collinearity()
for glmmTMB models with zeroinflation, where the zeroinflated model was an interceptonly model.
performance 0.7.1
CRAN release: 20210409
New supported models
Add support for
model_fit
(tidymodels).model_performance
supports kmeans models.
General
Give more informative warning when
r2_bayes()
for BFBayesFactor objects can’t be calculated.Several
check_*()
functions now return informative messages for invalid model types as input.r2()
supportsmhurdle
(mhurdle) models.The
performance_roc()
andperformance_accuracy()
functions unfortunately had spelling mistakes in the output columns: Sensitivity was called Sensivity and Specificity was called Specifity. We think these are understandable mistakes :)
Changes to functions
check_model()
check_model()
gains more arguments, to customize plot appearance.Added option to detrend QQ/PP plots in
check_model()
.
model_performance()
The
metrics
argument frommodel_performance()
andcompare_performance()
gains a"AICc"
option, to also compute the 2nd order AIC."R2_adj"
is now an explicit option in themetrics
argument frommodel_performance()
andcompare_performance()
.
Other functions
The defaultmethod for
r2()
now tries to compute an rsquared for all models that have no specificr2()
method yet, by using following formula:1sum((yy_hat)^2)/sum((yy_bar)^2))
The column name
Parameter
incheck_collinearity()
is now more appropriately namedTerm
.
Bug fixes
test_likelihoodratio()
now correctly sorts models with identical fixed effects part, but different other model parts (like zeroinflation).Fixed incorrect computation of models from inverseGaussian families, or Gaussian families fitted with
glm()
.Fixed issue in
performance_roc()
for models where outcome was not 0/1 coded.Fixed issue in
performance_accuracy()
for logistic regression models whenmethod = "boot"
.cronbachs_alpha()
did not work formatrix
objects, as stated in the docs. It now does.
performance 0.7.0
CRAN release: 20210203
Breaking Changes

compare_performance()
doesn’t return the models’ Bayes Factors, now returned bytest_performance()
andtest_bf()
.
New functions to test or compare models
test_vuong()
, to compare models using Vuong’s (1989) Test.test_bf()
, to compare models using Bayes factors.test_likelihoodratio()
as an alias forperformance_lrt()
.test_wald()
, as a rough approximation for the LRT.test_performance()
, to run the most relevant and appropriate tests based on the input.
Changes to functions
performance_lrt()
performance_lrt()
get an aliastest_likelihoodratio()
.Does not return AIC/BIC now (as they are not related to LRT per se and can be easily obtained with other functions).
Now contains a column with the difference in degrees of freedom between models.
Fixed column names for consistency.
Other functions
Revised computation of
performance_mse()
, to ensure that it’s always based on response residuals.performance_aic()
is now more robust.
Bug fixes
Fixed issue in
icc()
andvariance_decomposition()
for multivariate response models, where not all model parts contained random effects.Fixed issue in
compare_performance()
with duplicated rows.check_collinearity()
no longer breaks for models with rank deficient model matrix, but gives a warning instead.Fixed issue in
check_homogeneity()
formethod = "auto"
, which wrongly tested the response variable, not the residuals.Fixed issue in
check_homogeneity()
for edge cases where predictor had nonsyntactic names.
performance 0.6.1
CRAN release: 20201209
General

check_collinearity()
gains averbose
argument, to toggle warnings and messages.
performance 0.6.0
CRAN release: 20201201
General

model_performance()
now supportsmargins
,gamlss
,stanmvreg
andsemLme
.
New functions
r2_somers()
, to compute Somers’ Dxy rankcorrelation as R2measure for logistic regression models.display()
, to print output from packagefunctions into different formats.print_md()
is an alias fordisplay(format = "markdown")
.
Changes to functions
model_performance()
model_performance()
is now more robust and doesn’t fail if an index could not be computed. Instead, it returns all indices that were possible to calculate.model_performance()
gains a defaultmethod that catches all model objects not previously supported. If model object is also not supported by the defaultmethod, a warning is given.model_performance()
for metaformodels now includes the degrees of freedom for Cochran’s Q.
Other functions
performance_mse()
andperformance_rmse()
now always try to return the (R)MSE on the response scale.performance_accuracy()
now accepts all types of linear or logistic regression models, even if these are not of classlm
orglm
.performance_roc()
now accepts all types of logistic regression models, even if these are not of classglm
.r2()
for mixed models andr2_nakagawa()
gain atolerance
argument, to set the tolerance level for singularity checks when computing random effect variances for the conditional rsquared.
Bug fixes
Fixed issue in
icc()
introduced in the last update that makelme
models fail.Fixed issue in
performance_roc()
for models with factors as response.
performance 0.5.1
CRAN release: 20201029
Breaking changes
 Column names for
model_performance()
andcompare_performance()
were changed to be in line with the easystats naming convention:LOGLOSS
is nowLog_loss
,SCORE_LOG
isScore_log
andSCORE_SPHERICAL
is nowScore_spherical
.
New functions

r2_posterior()
for Bayesian models to obtain posterior distributions of Rsquared.
Changes to functions
r2_bayes()
works with Bayesian models fromBayesFactor
( #143 ).model_performance()
works with Bayesian models fromBayesFactor
( #150 ).model_performance()
now also includes the residual standard deviation.Improved formatting for Bayes factors in
compare_performance()
.compare_performance()
withrank = TRUE
doesn’t use theBF
values whenBIC
are present, to prevent “doubledipping” of the BIC values (#144).The
method
argument incheck_homogeneity()
gains a"levene"
option, to use Levene’s Test for homogeneity.
Bug fixes
 Fix bug in
compare_performance()
when...
arguments were function calls to regression objects, instead of direct function calls.
performance 0.5.0
CRAN release: 20200912
General
check_heteroscedasticity()
should now also work with zeroinflated mixed models from glmmTMB and GLMMadpative.check_outliers()
now returns a logical vector. Original numerical vector is still accessible viaas.numeric()
.
New functions

pp_check()
to compute posterior predictive checks for frequentist models.
Bug fixes
Fixed issue with incorrect labeling of groups from
icc()
whenby_group = TRUE
.Fixed issue in
check_heteroscedasticity()
for mixed models where sigma could not be calculated in a straightforward way.Fixed issues in
check_zeroinflation()
forMASS::glm.nb()
.Fixed CRAN check issues.
performance 0.4.8
CRAN release: 20200727
Changes to functions

icc()
now also computes a “classical” ICC forbrmsfit
models. The former way of calculating an “ICC” forbrmsfit
models is now available as new function calledvariance_decomposition()
.
Bug fixes
Fix issue with new version of bigutilsr for
check_outliers()
.Fix issue with model order in
performance_lrt()
.
performance 0.4.7
CRAN release: 20200614
Changes to functions
model_performance.rma()
now includes results from heterogeneity test for metaanalysis objects.check_normality()
now also works for mixed models (with the limitation that studentized residuals are used).check_normality()
gets aneffects
argument for mixed models, to check random effects for normality.
Bug fixes
Fixed issue in
performance_accuracy()
for binomial models when response variable had nonnumeric factor levels.Fixed issues in
performance_roc()
, which printed 1  AUC instead of AUC.
performance 0.4.6
CRAN release: 20200503
General
Minor revisions to
model_performance()
to meet changes in mlogit package.Support for
bayesx
models.
Changes to functions
icc()
gains aby_group
argument, to compute ICCs per different group factors in mixed models with multiple levels or crossclassified design.r2_nakagawa()
gains aby_group
argument, to compute explained variance at different levels (following the variancereduction approach by Hox 2010).performance_lrt()
now works on lavaan objects.
Bug fixes
Fix issues in some functions for models with logical dependent variable.
Fix bug in
check_itemscale()
, which caused multiple computations of skewness statistics.Fix issues in
r2()
for gam models.
performance 0.4.5
CRAN release: 20200328
General

model_performance()
andr2()
now support rmaobjects from package metafor, mlm and bife models.
Changes to functions
compare_performance()
gets abayesfactor
argument, to include or exclude the Bayes factor for model comparisons in the output.Added
r2.aov()
.
Bug fixes
Fixed issue in
performance_aic()
for models from package survey, which returned three different AIC values. Now only the AIC value is returned.Fixed issue in
check_collinearity()
for glmmTMB models when zeroinflated formula only had one predictor.Fixed issue in
check_model()
for lme models.Fixed issue in
check_distribution()
for brmsfit models.Fixed issue in
check_heteroscedasticity()
for aov objects.Fixed issues for lmrob and glmrob objects.