This post covers the parsnip package, which provides a common API to make model building in R easier.

Setup

Packages

The following packages are required:

Code

library(parsnip)
library(rsample)
library(recipes)

library(dplyr)

library(caret)
library(nnet)
library(agua)

tidymodels Packages
tidyverse Packages
- dplyr
Model Source Packages
- caret
- nnet
- agua

Unlike the previous entries in this series, the parsnip package has several expansion packages that provide additional algorithms and features (e.g., agua).

Additionally, like its predecessor caret, other R packages are leveraged for their modeling algorithms (e.g., nnet). Functions in these packages are called by the functions in parsnip, so these soure packages must also be installed.

Data

Throughout this series, I utilized the penguins data set from the modeldata package, which has a categorical outcome. Additionally, to demonstrate building models with a continuous outcome, I also utilized the car_prices data set from the modeldata package in a few examples.

Referencing previous entries in this series on the rsample and recipes packages, a 70/30 train/test initial_split() on both data sets is taken, a few pre-processing steps are applied on the training data sets to create recipe() objects, and those objects are passed to the prep() and juice() functions. This creates a processed training data set for each original data set.

Categorical
Continuous

Code

data("penguins", package = "modeldata")

set.seed(1914)
penguins_split_obj <- initial_split(penguins, prop = 0.7)

penguins_recipe_obj <- recipe(species ~ ., training(penguins_split_obj)) %>% 
  step_select(species, island, bill_length_mm, body_mass_g) %>% 
  step_naomit(all_predictors()) %>% 
  step_filter(species %in% c("Adelie", "Gentoo")) %>% 
  step_mutate(
    species = factor(species, levels = c("Adelie", "Gentoo")), 
    island = as.factor(island)
  )

penguins_train_tbl <- penguins_recipe_obj %>% 
  prep() %>% 
  juice()

penguins_train_tbl

# A tibble: 194 × 4
   species island    bill_length_mm body_mass_g
   <fct>   <fct>              <dbl>       <int>
 1 Adelie  Biscoe              36.5        2850
 2 Gentoo  Biscoe              52.5        5450
 3 Adelie  Biscoe              40.6        3550
 4 Gentoo  Biscoe              44.9        4750
 5 Adelie  Biscoe              39.6        3500
 6 Gentoo  Biscoe              45.8        4700
 7 Gentoo  Biscoe              46.1        4500
 8 Adelie  Biscoe              37.7        3075
 9 Gentoo  Biscoe              45.7        4400
10 Adelie  Torgersen           37.3        3775
# … with 184 more rows

Code

data("car_prices", package = "modeldata")

set.seed(1915)
car_prices_split_obj <- initial_split(car_prices, prop = 0.7)

car_prices_recipe_obj <- recipe(Price ~ ., training(car_prices_split_obj)) %>% 
  step_select(Price, Mileage, Doors, Leather) %>% 
  step_mutate(Doors = as.factor(Doors), Leather = as.factor(Leather))

car_prices_train_tbl <- car_prices_recipe_obj %>% 
  prep() %>% 
  juice()

car_prices_train_tbl

# A tibble: 562 × 4
    Price Mileage Doors Leather
    <dbl>   <int> <fct> <fct>  
 1 17978.   10986 4     0      
 2 20512.   16633 4     0      
 3 16507.   17451 4     1      
 4 25997.   21433 4     1      
 5 15129.   13828 4     1      
 6 12965.   29707 4     1      
 7 30575.   22298 4     1      
 8 21383.    7287 4     1      
 9 32053.    5144 4     0      
10 12208.   23512 2     1      
# … with 552 more rows

Background

Modeling in R can be a difficult process depending on the desired algorithm and which package’s implementation is used. The blessing and curse of open source is that anyone can create a package. While this creates a much richer ecosystem and allows for rapid development and expansion of the language, it also leads to inconsistent (and sometimes confusing) implementations of the same idea. As the number of R packages on CRAN (and elsewhere) continues to rapidly increase each year, this only exacerbates the issue.

The desire for a predictable, consistent, and uniform interface to model building is not a new concept. The caret package was first developed in 2005 and later published to CRAN in 2007. Also in 2007, the scikit-learn python library started development and was published in 2010.

The goal of the caret package was to provide a single interface to build many different models. It was written as wrapper around dozens of R packages that implemented hundreds of model algorithms. It provided a single function called caret::train() with uniform, predictable input arguments to select a model algorithm and build it. As it matured it attempted to be a do-it-all package when it came to modeling, attempting to resample, pre-process, model, tune, and post-process all in one function. As more models were added to the package, this quickly became untenable and it to become difficult to maintain. As a result, its developer, Max Kuhn realized that a new approach was needed. He and a team at Posit, formerly RStudio, started developing the tidymodels package ecosystem. Creating an ecosystem of packages, rather than a single package, allowed each package to focus on doing one part of the model building process well, while also being designed in such a way that all the packages work together and have consistent and predictable use patterns.

The model building package in tidymodels is the parsnip package. In the examples below, three modeling algorithms will be used to demonstrate how they can each be built in their underlying source package, using caret, and using parsnip. The key takeaway is how uniform, consistent, and user-friendly the parsnip API is, and how it has improved upon the caret interface, but still carries a similar philosophy.

Linear Regression

Perhaps the most well known modeling method is the linear regression. While there are several implementations in R, the lm() function from the stats package is commonly used. The function takes, at a minimum, two arguments: a formula() of the desired model and a data set provided to the data argument.

mod_lm_1_fit <- lm(Price ~ ., data = car_prices_train_tbl)

The lm() function is wrapped by caret. As shown below, the caret::train() function uses its code by setting method = "lm". The formula() and data arguments are passed the same way as before. Finally, the trControl argument is used to control the details of how a model is trained. The trainControl() function is used to define those details. By default, it instructs caret::train() to use a resampling method like boostrapping or cross-validation. Since the goal here is to simply demonstrate how to fit models using the caret package, no resampling is selected by setting the argument method = "none".

mod_lm_2_fit <- train(
  Price ~ ., data = car_prices_train_tbl, 
  method    = "lm",
  trControl = trainControl(method = "none")
)

The parsnip package decomposes the process of defining a model into several pipe-able steps. The first step is to instantiate a specification using the linear_reg() function. This is then fed into the set_engine() function, which specifies from which package (or in some cases, from which function) the model building code will be sourced. As shown below, lm() is once again used (since there are multiple packages in R that implement a linear regression, there are multiple engines that can be chosen). The result is then further fed into the parsnip::fit() function, which takes in the model formula and the data.

mod_lm_3_fit <- linear_reg() %>% 
  set_engine("lm") %>% 
  fit(Price ~ ., data = car_prices_train_tbl)

This sequential process of

Method Specification (e.g., linear_reg())
Engine Specification set_engine())
Model Fit (parsnip::fit())

creates the foundation of the uniform API the parsnip package offers and will be repeated for other models.

As shown below, all three methods result in the creation of the same model.

Code

summary(mod_lm_1_fit)


Call:
lm(formula = Price ~ ., data = car_prices_train_tbl)

Residuals:
   Min     1Q Median     3Q    Max 
-14422  -7451  -2214   6615  42245 

Coefficients:
              Estimate Std. Error t value Pr(>|t|)    
(Intercept)  2.543e+04  1.521e+03  16.718  < 2e-16 ***
Mileage     -1.592e-01  5.045e-02  -3.157 0.001683 ** 
Doors4      -3.971e+03  9.731e+02  -4.081 5.13e-05 ***
Leather1     3.173e+03  9.476e+02   3.349 0.000866 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 9788 on 558 degrees of freedom
Multiple R-squared:  0.0669,    Adjusted R-squared:  0.06188 
F-statistic: 13.33 on 3 and 558 DF,  p-value: 2.039e-08

Code

summary(mod_lm_2_fit)


Call:
lm(formula = .outcome ~ ., data = dat)

Residuals:
   Min     1Q Median     3Q    Max 
-14422  -7451  -2214   6615  42245 

Coefficients:
              Estimate Std. Error t value Pr(>|t|)    
(Intercept)  2.543e+04  1.521e+03  16.718  < 2e-16 ***
Mileage     -1.592e-01  5.045e-02  -3.157 0.001683 ** 
Doors4      -3.971e+03  9.731e+02  -4.081 5.13e-05 ***
Leather1     3.173e+03  9.476e+02   3.349 0.000866 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 9788 on 558 degrees of freedom
Multiple R-squared:  0.0669,    Adjusted R-squared:  0.06188 
F-statistic: 13.33 on 3 and 558 DF,  p-value: 2.039e-08

Code

summary(mod_lm_3_fit$fit)


Call:
stats::lm(formula = Price ~ ., data = data)

Residuals:
   Min     1Q Median     3Q    Max 
-14422  -7451  -2214   6615  42245 

Coefficients:
              Estimate Std. Error t value Pr(>|t|)    
(Intercept)  2.543e+04  1.521e+03  16.718  < 2e-16 ***
Mileage     -1.592e-01  5.045e-02  -3.157 0.001683 ** 
Doors4      -3.971e+03  9.731e+02  -4.081 5.13e-05 ***
Leather1     3.173e+03  9.476e+02   3.349 0.000866 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 9788 on 558 degrees of freedom
Multiple R-squared:  0.0669,    Adjusted R-squared:  0.06188 
F-statistic: 13.33 on 3 and 558 DF,  p-value: 2.039e-08

Logistic Regression

While linear regression is used for a continuous outcome, logistic regression is used for a categorical outcome (typically, binary). There are, again, several implementations in R. The glm() function from the stats package is commonly used. As with lm(), it takes a formula of the desired model and a data set provided to the data argument. Additionally, it requires a third argument: family = "binomial".

mod_glm_1_fit <- glm(
  species ~ ., 
  data = penguins_train_tbl, 
  family = "binomial"
)

The glm() function is also wrapped by caret. As shown below, the caret::train function uses its code by setting method = "glm" and additionally passing in family = "binomial". The formula, data, and trControl arguments are passed the same way as shown previously.

mod_glm_2_fit <- train(
  species ~ ., data = penguins_train_tbl, 
  method = "glm", 
  family = "binomial",
  trControl = trainControl(method = "none")
)

To create a logistic regression using the parsnip package, follow the same pattern as before. The first step is to instantiate a specification using the logistic_reg() function. This is then fed into the set_engine() function, specifying glm() as the engine (since there are multiple packages in R that implement a linear regression, there are multiple engines that can be chosen). Note that family = "binomial" is not used here. The result is then further fed into the parsnip::fit() function, which takes in the model formula and the data.

mod_glm_3_fit <- logistic_reg() %>% 
  set_engine("glm") %>% 
  fit(species ~ ., data = penguins_train_tbl)

As shown below, all three methods result in the creation of the same model.

Code

summary(mod_glm_1_fit)


Call:
glm(formula = species ~ ., family = "binomial", data = penguins_train_tbl)

Deviance Residuals: 
     Min        1Q    Median        3Q       Max  
-2.98422  -0.00003   0.00000   0.03693   1.32808  

Coefficients:
                  Estimate Std. Error z value Pr(>|z|)    
(Intercept)     -5.254e+01  1.516e+01  -3.466 0.000528 ***
islandDream     -1.915e+01  4.069e+03  -0.005 0.996245    
islandTorgersen -2.125e+01  3.666e+03  -0.006 0.995375    
bill_length_mm   9.209e-01  3.682e-01   2.501 0.012385 *  
body_mass_g      3.258e-03  1.915e-03   1.701 0.088920 .  
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 268.198  on 193  degrees of freedom
Residual deviance:  20.787  on 189  degrees of freedom
AIC: 30.787

Number of Fisher Scoring iterations: 20

Code

summary(mod_glm_2_fit)


Call:
NULL

Deviance Residuals: 
     Min        1Q    Median        3Q       Max  
-2.98422  -0.00003   0.00000   0.03693   1.32808  

Coefficients:
                  Estimate Std. Error z value Pr(>|z|)    
(Intercept)     -5.254e+01  1.516e+01  -3.466 0.000528 ***
islandDream     -1.915e+01  4.069e+03  -0.005 0.996245    
islandTorgersen -2.125e+01  3.666e+03  -0.006 0.995375    
bill_length_mm   9.209e-01  3.682e-01   2.501 0.012385 *  
body_mass_g      3.258e-03  1.915e-03   1.701 0.088920 .  
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 268.198  on 193  degrees of freedom
Residual deviance:  20.787  on 189  degrees of freedom
AIC: 30.787

Number of Fisher Scoring iterations: 20

Code

summary(mod_glm_3_fit$fit)


Call:
stats::glm(formula = species ~ ., family = stats::binomial, data = data)

Deviance Residuals: 
     Min        1Q    Median        3Q       Max  
-2.98422  -0.00003   0.00000   0.03693   1.32808  

Coefficients:
                  Estimate Std. Error z value Pr(>|z|)    
(Intercept)     -5.254e+01  1.516e+01  -3.466 0.000528 ***
islandDream     -1.915e+01  4.069e+03  -0.005 0.996245    
islandTorgersen -2.125e+01  3.666e+03  -0.006 0.995375    
bill_length_mm   9.209e-01  3.682e-01   2.501 0.012385 *  
body_mass_g      3.258e-03  1.915e-03   1.701 0.088920 .  
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 268.198  on 193  degrees of freedom
Residual deviance:  20.787  on 189  degrees of freedom
AIC: 30.787

Number of Fisher Scoring iterations: 20

Neural Network

Both linear regression and logistic regression are classic modeling methods that do not have any hyperparameters. To further expand upon the models that the parsnip package, the final model that will be examined is a neural network. Specifically, a multilayer perceptron (MLP). These types of models can handle both continuous and categorical outcomes, so both data sets are used below as examples.

Continuous
Categorical

The car_prices data set from the modeldata package is once again used. To prepare the data for modeling, the recipes package was used to one-hot encode the categorical features. The resulting pre-processed data set is shown below.

Code

car_prices_recipe_num_obj <- car_prices_recipe_obj %>% 
  step_dummy(all_nominal_predictors(), one_hot = TRUE)

car_prices_train_num_tbl <- car_prices_recipe_num_obj %>% 
  prep() %>% 
  juice()

car_prices_train_num_tbl

# A tibble: 562 × 6
    Price Mileage Doors_X2 Doors_X4 Leather_X0 Leather_X1
    <dbl>   <int>    <dbl>    <dbl>      <dbl>      <dbl>
 1 17978.   10986        0        1          1          0
 2 20512.   16633        0        1          1          0
 3 16507.   17451        0        1          0          1
 4 25997.   21433        0        1          0          1
 5 15129.   13828        0        1          0          1
 6 12965.   29707        0        1          0          1
 7 30575.   22298        0        1          0          1
 8 21383.    7287        0        1          0          1
 9 32053.    5144        0        1          1          0
10 12208.   23512        1        0          0          1
# … with 552 more rows

The penguins data set from the modeldata package is once again used. To prepare the data for modeling, the recipes package was used to one-hot encode the categorical features. The resulting pre-processed data set is shown below.s

Code

penguins_recipe_num_obj <- penguins_recipe_obj %>% 
  step_dummy(all_nominal_predictors(), one_hot = TRUE)

penguins_train_num_tbl <- penguins_recipe_num_obj %>% 
  prep() %>% 
  juice()

penguins_train_num_tbl

# A tibble: 194 × 6
   species bill_length_mm body_mass_g island_Biscoe island_Dream island_Torger…¹
   <fct>            <dbl>       <int>         <dbl>        <dbl>           <dbl>
 1 Adelie            36.5        2850             1            0               0
 2 Gentoo            52.5        5450             1            0               0
 3 Adelie            40.6        3550             1            0               0
 4 Gentoo            44.9        4750             1            0               0
 5 Adelie            39.6        3500             1            0               0
 6 Gentoo            45.8        4700             1            0               0
 7 Gentoo            46.1        4500             1            0               0
 8 Adelie            37.7        3075             1            0               0
 9 Gentoo            45.7        4400             1            0               0
10 Adelie            37.3        3775             0            0               1
# … with 184 more rows, and abbreviated variable name ¹island_Torgersen

There are several implementations in R. One common one is the nnet::nnet() function from the nnet package. As with previous algorithms, the function takes a formula of the desired model and a data set provided to the data argument. Additional arguments are also used to set the values of the hyperparameters and other options for the model:

lineout: set to TRUE to indicate that the output units are continuous, FALSE (the default) if categorical
size: the number of units in the hidden layer (hyperparameter)
decay: the weight decay (hyperparameter)
maxit: the maximum number of iterations (hyperparameter)
trace: set to FALSE to silence messages to the console

Continuous
Categorical

set.seed(1916)

mod_nnet_cont_1_fit <- nnet(
  Price ~ ., data = car_prices_train_num_tbl,
  linout = TRUE,
  size   = 1,
  decay  = 0,
  maxit  = 100,
  trace  = FALSE
)

set.seed(1917)

mod_nnet_cat_1_fit <- nnet(
  species ~ ., data = penguins_train_num_tbl,
  linout = FALSE,
  size  = 1,
  decay = 0,
  maxit = 100,
  trace = FALSE
)

The nnet package is also wrapped by caret. As shown below, the caret::train function leverages it by setting method = "nnet" and additionally passing in some of the options (linout and trace). Notice that the maxit hyperparameter is passed directly into the function, while the other hyperparameters size and decay were passed to the tuneGrid argument as a tibble(). This is a requirement of caret. If resampling was being used, multiple possible values of size and decay would be passed to iterate through and see which combination performs the best. Here, only single values are passed to demonstrate basic functionality. Additionally, the formula, data, and trControl arguments are passed the same way as shown previously.

Continuous
Categorical

set.seed(1916)

mod_nnet_cont_2_fit <- train(
  Price ~ ., data = car_prices_train_num_tbl,
  method    = "nnet",
  linout    = TRUE,
  maxit     = 100,
  trace     = FALSE,
  tuneGrid  = tibble(size = 1, decay = 0),
  trControl = trainControl(method = "none")
)

set.seed(1917)

mod_nnet_cat_2_fit <- train(
  species ~ ., data = penguins_train_num_tbl,
  method    = "nnet",
  linout    = FALSE,
  maxit     = 100,
  trace     = FALSE,
  tuneGrid  = tibble(size = 1, decay = 0),
  trControl = trainControl(method = "none")
)

To create an MLP using the parsnip package, follow the same pattern as before. The first step is to instantiate a specification using the mlp() function. This is then fed into the set_mode() function, which informs the model that the outcome is continuous (“regression”) or categorical (“classification”). This is passed to the set_engine() function, specifying the nnet package as the engine. To pass the hyperparameters, the set_args() function is then used. Note that the names of the arguments are slightly different in some cases than what they are called in the source package. This is because different packages (“engines”) may be available for models, so standardized names are used so that the functions work across all engines. For engine = "nnet", the argument translations are:

size -> hidden_units
decay -> penalty
maxit -> epochs

The result is then further fed into the parsnip::fit() function, which takes in the model formula and the data.

Continuous
Categorical

set.seed(1916)

mod_nnet_cont_3_fit <- mlp() %>% 
  set_mode("regression") %>% 
  set_engine("nnet") %>% 
  set_args(
    hidden_units = 1,
    penalty      = 0,
    epochs       = 100
  ) %>% 
  fit(Price ~ ., data = car_prices_train_num_tbl)

set.seed(1917)

mod_nnet_cat_3_fit <- mlp() %>% 
  set_mode("classification") %>% 
  set_engine("nnet") %>% 
  set_args(
    hidden_units = 1,
    penalty      = 0,
    epochs       = 100
  ) %>% 
  fit(species ~ ., data = penguins_train_num_tbl)

As shown below, all three methods for both continuous and categorical data sets result in the creation of the same models.

Continuous
Categorical

Code

mod_nnet_cont_1_fit

a 5-1-1 network with 8 weights
inputs: Mileage Doors_X2 Doors_X4 Leather_X0 Leather_X1 
output(s): Price 
options were - linear output units

Code

mod_nnet_cont_2_fit$finalModel

a 5-1-1 network with 8 weights
inputs: Mileage Doors_X2 Doors_X4 Leather_X0 Leather_X1 
output(s): .outcome 
options were - linear output units

Code

mod_nnet_cont_3_fit

parsnip model object

a 5-1-1 network with 8 weights
inputs: Mileage Doors_X2 Doors_X4 Leather_X0 Leather_X1 
output(s): Price 
options were - linear output units

Code

mod_nnet_cat_1_fit

a 5-1-1 network with 8 weights
inputs: bill_length_mm body_mass_g island_Biscoe island_Dream island_Torgersen 
output(s): species 
options were - entropy fitting

Code

mod_nnet_cat_2_fit$finalModel

a 5-1-1 network with 8 weights
inputs: bill_length_mm body_mass_g island_Biscoe island_Dream island_Torgersen 
output(s): .outcome 
options were - entropy fitting

Code

mod_nnet_cat_3_fit

parsnip model object

a 5-1-1 network with 8 weights
inputs: bill_length_mm body_mass_g island_Biscoe island_Dream island_Torgersen 
output(s): species 
options were - entropy fitting

Automated Machine Learning

The previous examples have consisted of fitting one model at a time. In situations where trying multiple models quickly is desired, one option is to use automated machine learning (AutoML). There are several options to automatically fit and tune many models at once, but one optional available in the tidymodels ecosystem is the agua package, which is a wrapper around h2o.

Before beginning model building, the h2o_start() function must be called to start an instance of an h2o server. From there, the auto_ml() function is used to instantiate an instance. The set_mode() function is then used to declare that the outcome is continuous (the car_prices data set from the modeldata package is used in this example). This is then passed to the set_engine() function, where the h2o R package is used as the underlying engine. From there, the set_args() function is used to declare that only 10 models will be examined using the max_models argument (there are additional arguments that can be used). The result is then further fed into the parsnip::fit() function, which takes in the model formula and the data.

h2o_start()

set.seed(1918)

mod_h2o_fit <- auto_ml() %>% 
  set_mode("regression") %>% 
  set_engine("h2o") %>% 
  set_args(max_models = 10) %>% 
  fit(Price ~ ., data = car_prices_train_num_tbl)

h2o_end()

mod_h2o_fit

parsnip model object

═════════════════════════ H2O AutoML Summary: 12 models ════════════════════════ 
 
 
══════════════════════════════════ Leaderboard ═════════════════════════════════ 
                                                 model_id      rmse       mse
1                          GBM_1_AutoML_1_20230921_201419  9830.895  96646498
2 StackedEnsemble_BestOfFamily_1_AutoML_1_20230921_201419  9903.569  98080681
3    StackedEnsemble_AllModels_1_AutoML_1_20230921_201419  9940.484  98813224
4                          DRF_1_AutoML_1_20230921_201419  9951.271  99027787
5                          XRT_1_AutoML_1_20230921_201419 10006.285 100125740
6                 DeepLearning_1_AutoML_1_20230921_201419 10019.250 100385371
       mae     rmsle mean_residual_deviance
1 7690.348 0.4131071               96646498
2 7739.664 0.4160158               98080681
3 7761.562 0.4175968               98813224
4 7772.377 0.4190103               99027787
5 7754.525 0.4188401              100125740
6 7807.799 0.4217025              100385371

Notes

In addition to the parsnip package website, another excellent resource for modeling algorithms is the Applied Predictive Modeling book by Max Kuhn and Kjell Johnson.

This post is based on a presentation that was given on the date listed. It may be updated from time to time to fix errors, detail new functions, and/or remove deprecated functions so the packages and R version will likely be newer than what was available at the time.

The R session information used for this post:

sessionInfo()

R version 4.2.1 (2022-06-23)
Platform: aarch64-apple-darwin20 (64-bit)
Running under: macOS 14.0

Matrix products: default
BLAS:   /Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices datasets  utils     methods   base     

other attached packages:
[1] agua_0.1.1      nnet_7.3-17     caret_6.0-93    lattice_0.20-45
[5] ggplot2_3.4.0   recipes_1.0.4   dplyr_1.0.10    rsample_1.1.1  
[9] parsnip_1.0.3  

loaded via a namespace (and not attached):
 [1] tidyr_1.2.1          jsonlite_1.8.0       splines_4.2.1       
 [4] foreach_1.5.2        prodlim_2019.11.13   stats4_4.2.1        
 [7] GPfit_1.0-8          renv_0.16.0          yaml_2.3.5          
[10] globals_0.16.2       ipred_0.9-13         pillar_1.8.1        
[13] glue_1.6.2           pROC_1.18.0          digest_0.6.29       
[16] hardhat_1.2.0        colorspace_2.0-3     htmltools_0.5.3     
[19] Matrix_1.4-1         plyr_1.8.8           timeDate_4022.108   
[22] pkgconfig_2.0.3      DiceDesign_1.9       lhs_1.1.6           
[25] listenv_0.8.0        purrr_0.3.5          scales_1.2.1        
[28] gower_1.0.1          lava_1.7.1           proxy_0.4-27        
[31] timechange_0.1.1     tibble_3.1.8         generics_0.1.3      
[34] ellipsis_0.3.2       withr_2.5.0          furrr_0.3.1         
[37] cli_3.6.1            survival_3.3-1       magrittr_2.0.3      
[40] evaluate_0.16        future_1.29.0        fansi_1.0.3         
[43] parallelly_1.32.1    nlme_3.1-157         MASS_7.3-57         
[46] dials_1.1.0          class_7.3-20         tools_4.2.1         
[49] data.table_1.14.6    tune_1.0.1           lifecycle_1.0.3     
[52] stringr_1.5.0        munsell_0.5.0        e1071_1.7-13        
[55] compiler_4.2.1       rlang_1.1.1          grid_4.2.1          
[58] yardstick_1.1.0      iterators_1.0.14     rstudioapi_0.14     
[61] rmarkdown_2.16       gtable_0.3.1         ModelMetrics_1.2.2.2
[64] codetools_0.2-18     reshape2_1.4.4       R6_2.5.1            
[67] lubridate_1.9.0      knitr_1.40           fastmap_1.1.0       
[70] future.apply_1.10.0  utf8_1.2.2           workflows_1.1.2     
[73] stringi_1.7.12       parallel_4.2.1       Rcpp_1.0.9          
[76] vctrs_0.6.3          rpart_4.1.16         tidyselect_1.2.0    
[79] xfun_0.40