# Integration Workflows: Combining Supervised and Unsupervised Learning

    library(tidylearn)
    library(dplyr)
    library(ggplot2)
    #> Warning: package 'ggplot2' was built under R version 4.5.3

## Introduction

Five functions put an unsupervised step in front of a supervised one: reduce the features, add cluster membership as a feature, propagate labels you do not have, handle outliers explicitly, or fit one model per cluster.

They coordinate the wrapped packages rather than implementing anything new. Each step is an ordinary `tl_model()` object, so `$fit` still reaches what the underlying package returned — `$fit$model` for the unsupervised step.

One rule governs all five, and it is the thing that goes wrong most often: the unsupervised step is fitted on training data and must then be applied to the test set, never refitted on it. Every example below carries the transformation across explicitly, because refitting leaks the test set into the model.

## Dimensionality Reduction as Preprocessing

PCA or MDS collapses correlated predictors into fewer components before the supervised fit. That costs some information and buys a smaller, less collinear feature space.

### Basic Usage

    # Reduce dimensions before classification
    reduced <- tl_reduce_dimensions(iris,
                                    response = "Species",
                                    method = "pca",
                                    n_components = 3)

    # Inspect reduced data
    head(reduced$data)
    #> # A tibble: 6 × 4
    #>     PC1    PC2     PC3 Species
    #>   <dbl>  <dbl>   <dbl> <fct>
    #> 1 -2.26 -0.478  0.127  setosa
    #> 2 -2.07  0.672  0.234  setosa
    #> 3 -2.36  0.341 -0.0441 setosa
    #> 4 -2.29  0.595 -0.0910 setosa
    #> 5 -2.38 -0.645 -0.0157 setosa
    #> 6 -2.07 -1.48  -0.0269 setosa

    # Train classifier on the reduced features. iris has three species, so this
    # needs a multiclass-capable method -- logistic regression is binary only.
    model_reduced <- tl_model(reduced$data, Species ~ ., method = "forest")
    print(model_reduced)
    #> tidylearn Model
    #> ===============
    #> Paradigm: supervised
    #> Method: forest
    #> Task: Classification
    #> Formula: Species ~ .
    #>
    #> Training observations: 150

    # Training-set accuracy. Use tl_evaluate() rather than comparing $.pred by
    # hand: what $.pred holds depends on the method and prediction type.
    tl_evaluate(model_reduced)
    #> # A tibble: 1 × 2
    #>   metric   value
    #>   <chr>    <dbl>
    #> 1 accuracy     1

### Comparison: Original vs Reduced Features

    # Split data for fair comparison
    split <- tl_split(iris, prop = 0.7, stratify = "Species", seed = 123)

    # Model with original features
    model_original <- tl_model(split$train, Species ~ ., method = "forest")
    eval_original <- tl_evaluate(model_original, new_data = split$test)

    # Model with PCA features
    reduced_train <- tl_reduce_dimensions(split$train,
                                          response = "Species",
                                          method = "pca",
                                          n_components = 3)

    model_pca <- tl_model(reduced_train$data, Species ~ ., method = "forest")

    # The test set must be projected through the PCA fitted on the training
    # data -- refitting PCA on the test set would leak information
    test_transformed <- predict(
      reduced_train$reduction_model,
      new_data = split$test %>% select(-Species)
    )
    test_transformed$Species <- split$test$Species

    eval_pca <- tl_evaluate(model_pca, new_data = test_transformed)

    # Compare results
    acc <- function(x) round(x$value[x$metric == "accuracy"] * 100, 1)
    n_original <- ncol(split$train) - 1
    n_reduced <- sum(grepl("^PC", names(reduced_train$data)))
    cat("Original features:", n_original, "->", acc(eval_original), "%\n")
    #> Original features: 4 -> 93.3 %
    cat("PCA features:", n_reduced, "->", acc(eval_pca), "%\n")
    #> PCA features: 3 -> 91.1 %
    cat("Feature reduction:",
        round((1 - n_reduced / n_original) * 100, 1), "%\n")
    #> Feature reduction: 25 %

## Cluster-Based Feature Engineering

Add cluster assignments as a feature, so a model that cannot express group structure directly gets a column that encodes it. The cluster model is kept on the result as a `"cluster_model"` attribute, which is what you need to assign test rows to the same clusters.

### Adding Cluster Features

    # Add cluster features
    data_clustered <- tl_add_cluster_features(iris,
                                              response = "Species",
                                              method = "kmeans",
                                              k = 3)

    # Check new features
    names(data_clustered)
    #> [1] "Sepal.Length"   "Sepal.Width"    "Petal.Length"   "Petal.Width"
    #> [5] "Species"        "cluster_kmeans"

    # Train model with cluster features
    model_cluster <- tl_model(data_clustered, Species ~ ., method = "forest")
    print(model_cluster)
    #> tidylearn Model
    #> ===============
    #> Paradigm: supervised
    #> Method: forest
    #> Task: Classification
    #> Formula: Species ~ .
    #>
    #> Training observations: 150

### Performance Comparison

    # Compare models with and without cluster features
    split_comp <- tl_split(iris, prop = 0.7, stratify = "Species", seed = 42)

    # Without cluster features
    model_no_cluster <- tl_model(split_comp$train, Species ~ ., method = "forest")
    preds_no_cluster <- predict(model_no_cluster, new_data = split_comp$test)
    acc_no_cluster <- mean(preds_no_cluster$.pred == split_comp$test$Species)

    # With cluster features
    train_clustered <- tl_add_cluster_features(split_comp$train,
                                               response = "Species",
                                               method = "kmeans",
                                               k = 3)
    model_with_cluster <- tl_model(train_clustered, Species ~ ., method = "forest")

    # Need to get cluster model for test data
    cluster_model <- attr(train_clustered, "cluster_model")
    test_clusters <- predict(cluster_model, new_data = split_comp$test[, -5])
    test_clustered <- split_comp$test
    test_clustered$cluster_kmeans <- as.factor(test_clusters$cluster)

    preds_with_cluster <- predict(model_with_cluster, new_data = test_clustered)
    acc_with_cluster <- mean(preds_with_cluster$.pred == split_comp$test$Species)

    cat("Without cluster features:", round(acc_no_cluster * 100, 1), "%\n")
    #> Without cluster features: 93.3 %
    cat("With cluster features:", round(acc_with_cluster * 100, 1), "%\n")
    #> With cluster features: 93.3 %

## Semi-Supervised Learning

When labels are expensive and unlabelled rows are plentiful, cluster the full dataset and give each cluster the majority label of whatever labelled rows fall in it. The propagated labels are guesses, and the fit is only as good as the assumption that clusters line up with classes.

### Training with Limited Labels

    # Use only 10% of labels
    set.seed(123)
    labeled_indices <- sample(nrow(iris), size = 15)  # 15 of 150 labelled

    # supervised_method defaults to "tree". Named here because a forest is
    # the better fit for propagated labels: the propagation step introduces
    # noise, and averaging over trees absorbs more of it than one tree does.
    model_semi <- tl_semisupervised(iris, Species ~ .,
                                    labeled_indices = labeled_indices,
                                    cluster_method = "kmeans",
                                    supervised_method = "forest")

    print(model_semi)
    #> tidylearn Model
    #> ===============
    #> Paradigm: supervised
    #> Method: forest
    #> Task: Classification
    #> Formula: Species ~ .
    #>
    #> Training observations: 150

    # Check how labels were propagated
    label_mapping <- model_semi$semisupervised_info$label_mapping
    print(label_mapping)
    #> # A tibble: 3 × 2
    #>   cluster cluster_label
    #>     <int> <chr>
    #> 1       1 virginica
    #> 2       2 versicolor
    #> 3       3 setosa

    # Evaluate against the true labels
    preds_semi <- predict(model_semi, new_data = iris, type = "class")
    accuracy_semi <- mean(preds_semi$.pred == iris$Species)

    cat("Accuracy with only", length(labeled_indices), "labels:",
        round(accuracy_semi * 100, 1), "%\n")
    #> Accuracy with only 15 labels: 90 %
    labeled_pct <- round(
      length(labeled_indices) / nrow(iris) * 100, 1
    )
    cat("Proportion of data labeled:", labeled_pct, "%\n")
    #> Proportion of data labeled: 10 %

### Comparison: Semi-Supervised vs Fully Supervised

    # Fully supervised with same amount of data
    labeled_data <- iris[labeled_indices, ]
    model_full <- tl_model(labeled_data, Species ~ ., method = "forest")
    preds_full <- predict(model_full, new_data = iris, type = "class")
    accuracy_full <- mean(preds_full$.pred == iris$Species)

    cat("Fully supervised (15 samples):", round(accuracy_full * 100, 1), "%\n")
    #> Fully supervised (15 samples): 94.7 %
    cat("Semi-supervised (15 labels + propagation):",
        round(accuracy_semi * 100, 1), "%\n")
    #> Semi-supervised (15 labels + propagation): 90 %

## Anomaly-Aware Modeling

`tl_anomaly_aware()` runs outlier detection first and then does one of two things with what it finds: `action = "flag"` adds an indicator column and keeps the rows, `action = "remove"` drops them.

### Flagging Anomalies

    # Flag anomalies as a feature
    model_anomaly_flag <- tl_anomaly_aware(iris, Species ~ .,
                                           response = "Species",
                                           anomaly_method = "dbscan",
                                           action = "flag",
                                           supervised_method = "forest")

    # Check anomaly info
    cat("Anomalies detected:", model_anomaly_flag$anomaly_info$n_anomalies, "\n")

### Removing Anomalies

    # Remove anomalies before training
    model_anomaly_remove <- tl_anomaly_aware(iris, Species ~ .,
                                             response = "Species",
                                             anomaly_method = "dbscan",
                                             action = "remove",
                                             supervised_method = "forest")

    cat("Anomalies removed:", model_anomaly_remove$anomalies_removed, "\n")

## Stratified Models

One model per cluster, for data where the relationship differs between groups. It reads more easily than a single model carrying many interaction terms, and it needs enough rows in every cluster for each fit to be estimable.

### Training Stratified Models

    # Train separate models for different clusters
    stratified_models <- tl_stratified_models(mtcars, mpg ~ .,
                                              cluster_method = "kmeans",
                                              k = 3,
                                              supervised_method = "linear")
    #> Note: Response 'mpg' has 6 unique numeric values. Treating as regression. Convert to factor for classification.
    #> Note: Response 'mpg' has 8 unique numeric values. Treating as regression. Convert to factor for classification.

    # Check structure
    names(stratified_models)
    #> [1] "cluster_model"     "supervised_models" "formula"
    #> [4] "data"
    length(stratified_models$supervised_models)
    #> [1] 3

    # Predictions using stratified models
    preds_stratified <- predict(stratified_models)
    head(preds_stratified)
    #> # A tibble: 6 × 2
    #>   .pred .cluster
    #>   <dbl>    <int>
    #> 1  20.7        3
    #> 2  20.5        3
    #> 3  24.7        3
    #> 4  21.4        1
    #> 5  19.2        2
    #> 6  18.1        1

    # Calculate RMSE
    rmse_stratified <- sqrt(mean((preds_stratified$.pred - mtcars$mpg)^2))
    cat("Stratified Model RMSE:", round(rmse_stratified, 2), "\n")
    #> Stratified Model RMSE: 1.06

    # Compare with single model
    model_single <- tl_model(mtcars, mpg ~ ., method = "linear")
    preds_single <- predict(model_single)
    rmse_single <- sqrt(mean((preds_single$.pred - mtcars$mpg)^2))
    cat("Single Model RMSE:", round(rmse_single, 2), "\n")
    #> Single Model RMSE: 2.15

## Complete Integration Workflow

Combining multiple integration techniques:

    # Step 1: Split data
    workflow_split <- tl_split(iris, prop = 0.7, stratify = "Species", seed = 42)

    # Step 2: Reduce dimensions
    workflow_reduced <- tl_reduce_dimensions(workflow_split$train,
                                             response = "Species",
                                             method = "pca",
                                             n_components = 3)

    # Step 3: Add cluster features to reduced data
    workflow_clustered <- tl_add_cluster_features(workflow_reduced$data,
                                                  response = "Species",
                                                  method = "kmeans",
                                                  k = 3)

    # Step 4: Train final model
    workflow_model <- tl_model(workflow_clustered, Species ~ ., method = "forest")

    print(workflow_model)
    #> tidylearn Model
    #> ===============
    #> Paradigm: supervised
    #> Method: forest
    #> Task: Classification
    #> Formula: Species ~ .
    #>
    #> Training observations: 105

    # Transform test data through same pipeline
    # 1. Apply PCA transformation
    test_pca <- predict(workflow_reduced$reduction_model,
                        new_data = workflow_split$test[, -5])
    test_pca$Species <- workflow_split$test$Species

    # 2. Get cluster assignments. The cluster model was fitted on the PC
    # columns; predict() matches new_data to those columns by name and
    # errors on a mismatch rather than assigning against the wrong ones.
    cluster_model_wf <- attr(workflow_clustered, "cluster_model")
    test_clusters_wf <- predict(cluster_model_wf, new_data = test_pca)
    test_pca$cluster_kmeans <- as.factor(test_clusters_wf$cluster)

    # 3. Predict
    workflow_preds <- predict(workflow_model, new_data = test_pca)
    workflow_accuracy <- mean(workflow_preds$.pred == workflow_split$test$Species)

    cat("Complete Workflow Accuracy:", round(workflow_accuracy * 100, 1), "%\n")
    #> Complete Workflow Accuracy: 91.1 %

## Practical Example: Credit Risk Assessment

    # Simulate credit data
    set.seed(42)
    n <- 500
    credit_data <- data.frame(
      age = rnorm(n, 40, 12),
      income = rnorm(n, 50000, 20000),
      debt_ratio = runif(n, 0, 0.5),
      credit_score = rnorm(n, 700, 100),
      years_employed = rpois(n, 5)
    )

    # Create target variable (default risk)
    credit_data$default <- factor(
      ifelse(
        credit_data$debt_ratio > 0.4 & credit_data$credit_score < 650,
        "Yes", "No"
      )
    )

    # Split data
    credit_split <- tl_split(
      credit_data, prop = 0.7, stratify = "default", seed = 123
    )

    # Strategy 1: Add customer segments as features
    credit_clustered <- tl_add_cluster_features(credit_split$train,
                                                response = "default",
                                                method = "kmeans",
                                                k = 4)

    model_credit <- tl_model(credit_clustered, default ~ ., method = "forest")

    # Transform test data
    cluster_model_credit <- attr(credit_clustered, "cluster_model")
    test_clusters_credit <- predict(cluster_model_credit,
                                    new_data = credit_split$test[, -6])
    test_credit <- credit_split$test
    test_credit$cluster_kmeans <- as.factor(test_clusters_credit$cluster)

    preds_credit <- predict(model_credit, new_data = test_credit)
    accuracy_credit <- mean(preds_credit$.pred == credit_split$test$default)

    cat("Credit Risk Model Accuracy:", round(accuracy_credit * 100, 1), "%\n")
    #> Credit Risk Model Accuracy: 100 %

## What Each Combination Buys You

- Dimensionality reduction before fitting trades some accuracy for a smaller feature space. Whether the trade is worth making is empirical – run the comparison above on your own data.
- Cluster features give a model a handle on group structure it cannot otherwise express. On data with no group structure they add noise.
- Semi-supervised learning is worth reaching for when labels are expensive and unlabelled observations are plentiful.
- Anomaly-aware modelling decides what happens to outliers explicitly rather than leaving it to the loss function.
- Stratified models fit one model per cluster, which reads more easily than a single model with many interaction terms, and needs enough observations in every cluster to be estimable.

## Function Reference

Function | Puts this in front of the supervised fit
--- | ---
`tl_reduce_dimensions()` | PCA or MDS
`tl_add_cluster_features()` | Cluster membership as a column
`tl_semisupervised()` | Label propagation from clusters
`tl_anomaly_aware()` | Outlier detection, flagged or removed
`tl_stratified_models()` | One model per cluster

## Where to Go Next

- `vignette("unsupervised-learning")` — the clustering and ordination steps above, driven directly
- `vignette("tuning-and-pipelines")` — `tl_pipeline()` does the train-then-replay bookkeeping for you
- `vignette("automl")` — `tl_auto_ml()` applies the PCA and clustering variants automatically