Package index • nestedtune

Nested resampling

Build the design the outer loop runs on: one outer fold per row, with that fold’s inner resamples beside it, and no copy of the data per fold.

nested_resamples(): Build a nested resampling design

Running the loop

Tune on each outer fold’s inner resamples, select a candidate, then fit and score it on the outer split, keeping what every fold chose. A workflow with nothing to tune and a whole workflow set run on the same design.

nested_tune_grid(): Nested cross-validation with a grid search inside
nested_tune_bayes(): Nested cross-validation with Bayesian optimization inside
nested_tune_race_anova() nested_tune_race_win_loss(): Nested cross-validation with racing inside
nested_tune_sim_anneal(): Nested cross-validation with simulated annealing inside
nested_fit_resamples(): Score a workflow with nothing to tune on a nested design
nested_workflow_map(): Run every workflow of a workflow set through one nested design
collect_metrics(<nested_results>): Collect the metrics from a nested resampling run
collect_selections() collect_inner_metrics() collect_notes(<nested_results>): Collect each outer fold's selections, inner metrics or notes
collect_predictions(<nested_results>) collect_extracts(<nested_results>): Collect the predictions or extracts of a nested run's outer fits
compute_metrics(<nested_results>): Score the saved predictions of a nested run with a metric set
augment(<nested_results>): Join the held-out predictions of a nested run onto its data
print(<nested_results>): Print a nested cross-validation result
summary(<nested_results>) print(<summary.nested_results>): Summarize a nested cross-validation result
predict(<nested_results>) predict(<nested_results_set>): Predicting from a nested run
autoplot(<nested_results>): Plot a nested cross-validation result
agreement(): Tabulate how often each parameter setting was selected across the outer folds
extract_procedure(): Extract the procedure a nested run used
selection_rule(): Choose how each fold selects its candidate
collect_metrics(<nested_results_set>) collect_selections(<nested_results_set>) collect_inner_metrics(<nested_results_set>) collect_notes(<nested_results_set>) collect_predictions(<nested_results_set>) collect_extracts(<nested_results_set>) compute_metrics(<nested_results_set>) augment(<nested_results_set>): Collect the results of a workflow-set run
print(<nested_results_set>): Print a workflow-set run
extract_workflow(<nested_results_set>): Extract one workflow of a workflow-set run
agreement(<nested_results_set>) autoplot(<nested_results_set>) summary(<nested_results_set>) print(<summary.nested_results_set>): Summarize, plot and tabulate a workflow-set run

The final model

Run the recorded tuning procedure once more with the whole dataset in hand and get back the model to deploy. It is its own object, never a field on the results.

nested_final_fit(): Fit the final model after nested cross-validation
print(<nested_final_fit>): Print a final fit
summary(<nested_final_fit>) print(<summary.nested_final_fit>): Summarize a final fit
predict(<nested_final_fit>) augment(<nested_final_fit>): Predict with the final model
extract_fit_parsnip(<nested_final_fit>) extract_fit_engine(<nested_final_fit>) extract_recipe(<nested_final_fit>) extract_mold(<nested_final_fit>) extract_preprocessor(<nested_final_fit>) extract_spec_parsnip(<nested_final_fit>) outcome_names(<nested_final_fit>) extract_parameter_set_dials(<nested_final_fit>): Extract the parts of a final fit's workflow
extract_tune_results(): Extract the tuning run of a final fit
extract_scored_candidates(): Extract the candidates a final fit scored

Re-exports

reexports collect_metrics extract_workflow extract_fit_parsnip extract_fit_engine extract_recipe extract_mold extract_preprocessor extract_spec_parsnip outcome_names extract_parameter_set_dials autoplot augment compute_metrics collect_notes collect_predictions collect_extracts: Objects exported from other packages