Cross-validated model performanceSource:
This function cross-validates regression models in a user-supplied new sample or by using holdout (train-test), k-fold, or leave-one-out cross-validation.
performance_cv( model, data = NULL, method = c("holdout", "k_fold", "loo"), metrics = "all", prop = 0.3, k = 5, stack = TRUE, verbose = TRUE, ... )
A regression model.
Optional. A data frame containing the same variables as
modelthat will be used as the cross-validation sample.
Character string, indicating the cross-validation method to use: whether holdout (
"holdout", aka train-test), k-fold (
"k_fold"), or leave-one-out (
datais supplied, this argument is ignored.
"common"or a character vector of metrics to be computed (some of
c("ELPD", "Deviance", "MSE", "RMSE", "R2")). "common" will compute R2 and RMSE.
method = "holdout", what proportion of the sample to hold out as the test sample?
method = "k_fold", the number of folds to use.
method = "k_fold", should performance be computed by stacking residuals from each holdout fold and calculating each metric on the stacked data (
TRUE, default) or should performance be computed by calculating metrics within each holdout fold and averaging performance across each fold (
A data frame with columns for each metric requested, as well as
method = "holdout" and the
Method used for cross-validation. If
method = "holdout" and
stack = TRUE, the standard error (standard
deviation across holdout folds) for each metric is also included.
model <- lm(mpg ~ wt + cyl, data = mtcars) performance_cv(model) #> # Cross-validation performance (30% holdout method) #> #> MSE | RMSE | R2 #> ----------------- #> 6.9 | 2.6 | 0.83