Title:	Feature Importance Methods for Global Explanations
Version:	1.0.0
Description:	Provides a consistent interface for common feature importance methods as described in Ewald et al. (2024) <doi:10.1007/978-3-031-63797-1_22>, including permutation feature importance (PFI), conditional and relative feature importance (CFI, RFI), leave one covariate out (LOCO), and Shapley additive global importance (SAGE), as well as feature sampling mechanisms to support conditional importance methods.
License:	LGPL (≥ 3)
URL:	https://mlr-org.github.io/xplainfi/, https://github.com/mlr-org/xplainfi
BugReports:	https://github.com/mlr-org/xplainfi/issues
Depends:	R (≥ 4.1.0)
Imports:	checkmate, cli, data.table (≥ 1.15.0), mirai, mlr3 (≥ 1.1.0), mvtnorm, paradox (≥ 1.0.0), R6, stats, utils
Suggests:	arf, DiagrammeR, foreach, future, future.apply, ggplot2, glue, gower, knitr, knockoff, lgr, mlr3data, mlr3fselect, mlr3learners, partykit, patchwork, ranger, rmarkdown, rpart, testthat (≥ 3.0.0), withr
VignetteBuilder:	knitr
Config/Needs/website:	rmarkdown, here, jsonlite, cpi
Config/testthat/edition:	3
Config/testthat/parallel:	true
Encoding:	UTF-8
RoxygenNote:	7.3.3
NeedsCompilation:	no
Packaged:	2026-01-26 16:53:28 UTC; Lukas
Author:	Lukas Burk [aut, cre, cph]
Maintainer:	Lukas Burk <cran@lukasburk.de>
Repository:	CRAN
Date/Publication:	2026-01-30 11:00:02 UTC

xplainfi: Feature Importance Methods for Global Explanations

Description

Provides a consistent interface for common feature importance methods as described in Ewald et al. (2024) doi:10.1007/978-3-031-63797-1_22, including permutation feature importance (PFI), conditional and relative feature importance (CFI, RFI), leave one covariate out (LOCO), and Shapley additive global importance (SAGE), as well as feature sampling mechanisms to support conditional importance methods.

Author(s)

Maintainer: Lukas Burk cran@lukasburk.de (ORCID) [copyright holder]

See Also

Useful links:

• https://mlr-org.github.io/xplainfi/

• https://github.com/mlr-org/xplainfi

• Report bugs at https://github.com/mlr-org/xplainfi/issues

Conditional Feature Importance

Description

Implementation of CFI using modular sampling approach

Super classes

xplainfi::FeatureImportanceMethod -> xplainfi::PerturbationImportance -> CFI

Methods

Public methods

• CFI$new()

• CFI$compute()

• CFI$clone()

Method new()

Creates a new instance of the CFI class

Usage

CFI$new(
  task,
  learner,
  measure = NULL,
  resampling = NULL,
  features = NULL,
  groups = NULL,
  relation = "difference",
  n_repeats = 1L,
  batch_size = NULL,
  sampler = NULL
)

Arguments

Method compute()

Compute CFI scores

Usage

CFI$compute(
  n_repeats = NULL,
  batch_size = NULL,
  store_models = TRUE,
  store_backends = TRUE
)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

CFI$clone(deep = FALSE)

Arguments

References

Blesch K, Koenen N, Kapar J, Golchian P, Burk L, Loecher M, Wright M (2025). “Conditional Feature Importance with Generative Modeling Using Adversarial Random Forests.” Proceedings of the AAAI Conference on Artificial Intelligence, 39(15), 15596–15604. doi:10.1609/aaai.v39i15.33712.

Examples

library(mlr3)
library(mlr3learners)

task <- sim_dgp_correlated(n = 500)

# Using default ConditionalARFSampler
cfi <- CFI$new(
  task = task,
  learner = lrn("regr.ranger", num.trees = 10),
  measure = msr("regr.mse")
)
cfi$compute()
cfi$importance()

ARF-based Conditional Sampler

Description

Implements conditional sampling using Adversarial Random Forests (ARF). ARF can handle mixed data types (continuous and categorical) and provides flexible conditional sampling by modeling the joint distribution.

Details

The ConditionalARFSampler fits an Adversarial Random Forest model on the task data, then uses it to generate samples from P(X_j | X_{-j}) where X_j is the feature of interest and X_{-j} are the conditioning features.

Super classes

xplainfi::FeatureSampler -> xplainfi::ConditionalSampler -> ConditionalARFSampler

Public fields

Methods

Public methods

• ConditionalARFSampler$new()

• ConditionalARFSampler$sample()

• ConditionalARFSampler$sample_newdata()

• ConditionalARFSampler$clone()

Method new()

Creates a new instance of the ConditionalARFSampler class. To fit the ARF in parallel, register a parallel backend first (see arf::arf) and set parallel = TRUE.

Usage

ConditionalARFSampler$new(
  task,
  conditioning_set = NULL,
  num_trees = 10L,
  min_node_size = 20L,
  finite_bounds = "no",
  epsilon = 1e-15,
  round = TRUE,
  stepsize = 0,
  verbose = FALSE,
  parallel = FALSE,
  ...
)

Arguments

Method sample()

Sample from stored task. Parameters use hierarchical resolution: function argument > stored param_set value > hard-coded default.

Usage

ConditionalARFSampler$sample(
  feature,
  row_ids = NULL,
  conditioning_set = NULL,
  round = NULL,
  stepsize = NULL,
  verbose = NULL,
  parallel = NULL
)

Arguments

Returns

Modified copy with sampled feature(s).

Method sample_newdata()

Sample from external data. See ⁠$sample()⁠ for parameter details.

Usage

ConditionalARFSampler$sample_newdata(
  feature,
  newdata,
  conditioning_set = NULL,
  round = NULL,
  stepsize = NULL,
  verbose = NULL,
  parallel = NULL
)

Arguments

Returns

Modified copy with sampled feature(s).

Method clone()

The objects of this class are cloneable with this method.

Usage

ConditionalARFSampler$clone(deep = FALSE)

Arguments

References

Watson D, Blesch K, Kapar J, Wright M (2023). “Adversarial Random Forests for Density Estimation and Generative Modeling.” In Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, 5357–5375. https://proceedings.mlr.press/v206/watson23a.html.

Blesch K, Koenen N, Kapar J, Golchian P, Burk L, Loecher M, Wright M (2025). “Conditional Feature Importance with Generative Modeling Using Adversarial Random Forests.” Proceedings of the AAAI Conference on Artificial Intelligence, 39(15), 15596–15604. doi:10.1609/aaai.v39i15.33712.

Examples

library(mlr3)
task = tgen("2dnormals")$generate(n = 100)
# Create sampler with default parameters
sampler = ConditionalARFSampler$new(task, conditioning_set = "x2", verbose = FALSE)
# Sample using row_ids from stored task
sampled_data = sampler$sample("x1", row_ids = 1:10)
# Or use external data
data = task$data()
sampled_data_ext = sampler$sample_newdata("x1", newdata = data, conditioning_set = "x2")

# Example with custom ARF parameters
sampler_custom = ConditionalARFSampler$new(
  task,
  min_node_size = 10L,
  finite_bounds = "local",
  verbose = FALSE
)
sampled_custom = sampler_custom$sample("x1", conditioning_set = "x2")

(experimental) Conditional Inference Tree Conditional Sampler

Description

Implements conditional sampling using conditional inference trees (ctree). Builds a tree predicting target features from conditioning features, then samples from the terminal node corresponding to each test observation.

Details

This sampler approximates the conditional distribution P(X_B | X_A = x_A) by:

1. Building a conditional inference tree with X_B as response and X_A as predictors

2. For each test observation, finding its terminal (leaf) node in the tree

3. Sampling uniformly from training observations in that same terminal node

Conditional inference trees (ctree) use permutation tests to determine splits, which helps avoid overfitting and handles mixed feature types naturally. The tree partitions the feature space based on the conditioning variables, creating local neighborhoods that respect the conditional distribution structure.

Key advantages over other samplers:

• Handles mixed feature types (continuous and categorical)

• Non-parametric (no distributional assumptions)

• Automatic feature selection (splits only on informative features)

• Can capture non-linear conditional relationships

• Statistically principled splitting criteria

Hyperparameters control tree complexity:

• mincriterion: Significance level for splits (higher = fewer splits)

• minsplit: Minimum observations required for a split

• minbucket: Minimum observations in terminal nodes

This implementation is inspired by shapr's ctree approach but simplified for our use case (we build trees on-demand rather than pre-computing all subsets).

Advantages:

• Works with any feature types

• Robust to outliers

• Interpretable tree structure

• Handles high-dimensional conditioning

Limitations:

• Requires model fitting (slower than kNN)

• Can produce duplicates if terminal nodes are small

• Tree building time increases with data size

Super classes

xplainfi::FeatureSampler -> xplainfi::ConditionalSampler -> ConditionalCtreeSampler

Public fields

Methods

Public methods

• ConditionalCtreeSampler$new()

• ConditionalCtreeSampler$clone()

Method new()

Creates a new ConditionalCtreeSampler.

Usage

ConditionalCtreeSampler$new(
  task,
  conditioning_set = NULL,
  mincriterion = 0.95,
  minsplit = 20L,
  minbucket = 7L,
  use_cache = TRUE
)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

ConditionalCtreeSampler$clone(deep = FALSE)

Arguments

References

Hothorn T, Hornik K, Zeileis A (2006). “Unbiased Recursive Partitioning: A Conditional Inference Framework.” Journal of Computational and Graphical Statistics, 15(3), 651–674. doi:10.1198/106186006X133933.

Aas K, Jullum M, Løland A (2021). “Explaining Individual Predictions When Features Are Dependent: More Accurate Approximations to Shapley Values.” Artificial Intelligence, 298, 103502. doi:10.1016/j.artint.2021.103502.

Examples

library(mlr3)
task = tgen("friedman1")$generate(n = 100)

# Create sampler with default parameters
sampler = ConditionalCtreeSampler$new(task)

# Sample features conditioned on others
test_data = task$data(rows = 1:5)
sampled = sampler$sample_newdata(
  feature = c("important2", "important3"),
  newdata = test_data,
  conditioning_set = "important1"
)

Gaussian Conditional Sampler

Description

Implements conditional sampling assuming features follow a multivariate Gaussian distribution. Computes conditional distributions analytically using standard formulas for multivariate normal distributions.

Details

For a joint Gaussian distribution X \sim N(\mu, \Sigma), partitioned as X = (X_A, X_B), the conditional distribution is:

X_B | X_A = x_A \sim N(\mu_{B|A}, \Sigma_{B|A})

where:

\mu_{B|A} = \mu_B + \Sigma_{BA} \Sigma_{AA}^{-1} (x_A - \mu_A)

\Sigma_{B|A} = \Sigma_{BB} - \Sigma_{BA} \Sigma_{AA}^{-1} \Sigma_{AB}

This is equivalent to the regression formulation used by fippy:

\beta = \Sigma_{BA} \Sigma_{AA}^{-1}

\mu_{B|A} = \mu_B + \beta (x_A - \mu_A)

\Sigma_{B|A} = \Sigma_{BB} - \beta \Sigma_{AB}

Assumptions:

• Features are approximately multivariate normal

• Only continuous features are supported

Advantages:

• Very fast (closed-form solution)

• Deterministic (given seed)

• No hyperparameters

• Memory efficient

Limitations:

• Strong distributional assumption

• May produce out-of-range values for bounded features

• Cannot handle categorical features

• Integer features are treated as continuous and rounded back to integers

Super classes

xplainfi::FeatureSampler -> xplainfi::ConditionalSampler -> ConditionalGaussianSampler

Public fields

Methods

Public methods

• ConditionalGaussianSampler$new()

• ConditionalGaussianSampler$clone()

Method new()

Creates a new ConditionalGaussianSampler.

Usage

ConditionalGaussianSampler$new(task, conditioning_set = NULL)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

ConditionalGaussianSampler$clone(deep = FALSE)

Arguments

References

Anderson T (2003). An Introduction to Multivariate Statistical Analysis, 3rd edition. Wiley-Interscience, Hoboken, NJ. ISBN 9780471360919.

Examples

library(mlr3)
task = tgen("friedman1")$generate(n = 100)
sampler = ConditionalGaussianSampler$new(task)

# Sample x2, x3 conditioned on x1
test_data = task$data(rows = 1:5)
sampled = sampler$sample_newdata(
  feature = c("important2", "important3"),
  newdata = test_data,
  conditioning_set = "important1"
)

k-Nearest Neighbors Conditional Sampler

Description

Implements conditional sampling using k-nearest neighbors (kNN). For each observation, finds the k most similar observations based on conditioning features, then samples the target features from these neighbors.

Details

This sampler approximates the conditional distribution P(X_B | X_A = x_A) by:

1. Finding the k nearest neighbors of x_A in the training data

2. Sampling uniformly from the target feature values X_B of these k neighbors

This is a simple, non-parametric approach that:

• Requires no distributional assumptions

• Handles mixed feature types (numeric, integer, factor, ordered, logical)

• Is computationally efficient (no model fitting required)

• Adapts locally to the data structure

The method is related to hot-deck imputation and kNN imputation techniques used in missing data problems. As k \to \infty and k/n \to 0, the kNN conditional distribution converges to the true conditional distribution under mild regularity conditions (Lipschitz continuity).

Distance Metrics:

The sampler supports two distance metrics:

• Euclidean: For numeric/integer features only. Standardizes features before computing distances.

• Gower: For mixed feature types. Handles numeric, factor, ordered, and logical features. Numeric features are range-normalized, categorical features use exact matching (0/1).

The distance parameter controls which metric to use:

• "auto" (default): Automatically selects Euclidean for all-numeric features, Gower otherwise

• "euclidean": Forces Euclidean distance (errors if non-numeric features present)

• "gower": Forces Gower distance (works with any feature types)

Advantages:

• Very fast (no model training)

• Works with any feature types

• Automatic distance metric selection

• Naturally respects local data structure

Limitations:

• Sensitive to choice of k

• The full task data is required for prediction

• Can produce duplicates if k is small

• May not extrapolate well to new regions

Super classes

xplainfi::FeatureSampler -> xplainfi::ConditionalSampler -> ConditionalKNNSampler

Public fields

Methods

Public methods

• ConditionalKNNSampler$new()

• ConditionalKNNSampler$sample()

• ConditionalKNNSampler$sample_newdata()

• ConditionalKNNSampler$clone()

Method new()

Creates a new ConditionalKNNSampler.

Usage

ConditionalKNNSampler$new(task, conditioning_set = NULL, k = 5L)

Arguments

Method sample()

Sample features from their kNN-based conditional distribution.

Usage

ConditionalKNNSampler$sample(
  feature,
  row_ids = NULL,
  conditioning_set = NULL,
  k = NULL
)

Arguments

Returns

Modified copy with sampled feature(s).

Method sample_newdata()

Sample from external data conditionally.

Usage

ConditionalKNNSampler$sample_newdata(
  feature,
  newdata,
  conditioning_set = NULL,
  k = NULL
)

Arguments

Returns

Modified copy with sampled feature(s).

Method clone()

The objects of this class are cloneable with this method.

Usage

ConditionalKNNSampler$clone(deep = FALSE)

Arguments

References

Little R, Rubin D (2019). Statistical Analysis with Missing Data, 3rd edition. John Wiley & Sons, Hoboken, NJ. ISBN 9780470526798.

Troyanskaya O, Cantor M, Sherlock G, Brown P, Hastie T, Tibshirani R, Botstein D, Altman R (2001). “Missing Value Estimation Methods for DNA Microarrays.” Bioinformatics, 17(6), 520–525. doi:10.1093/bioinformatics/17.6.520.

Examples

library(mlr3)
task = tgen("friedman1")$generate(n = 100)
sampler = ConditionalKNNSampler$new(task, k = 5)

# Sample features conditioned on others
test_data = task$data(rows = 1:5)
sampled = sampler$sample_newdata(
  feature = c("important2", "important3"),
  newdata = test_data,
  conditioning_set = "important1"
)

Conditional SAGE

Description

SAGE with conditional sampling (features are "marginalized" conditionally). Uses ConditionalARFSampler as default ConditionalSampler.

Super classes

xplainfi::FeatureImportanceMethod -> xplainfi::SAGE -> ConditionalSAGE

Public fields

Methods

Public methods

• ConditionalSAGE$new()

• ConditionalSAGE$clone()

Method new()

Creates a new instance of the ConditionalSAGE class.

Usage

ConditionalSAGE$new(
  task,
  learner,
  measure = NULL,
  resampling = NULL,
  features = NULL,
  n_permutations = 10L,
  sampler = NULL,
  batch_size = 5000L,
  n_samples = 100L,
  early_stopping = FALSE,
  se_threshold = 0.01,
  min_permutations = 10L,
  check_interval = 1L
)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

ConditionalSAGE$clone(deep = FALSE)

Arguments

See Also

MarginalSAGE

Examples

library(mlr3)
task = tgen("friedman1")$generate(n = 100)


# Using default ConditionalARFSampler (also handles all mixed data)
sage = ConditionalSAGE$new(
  task = task,
  learner = lrn("regr.ranger", num.trees = 50),
  measure = msr("regr.mse"),
  n_permutations = 3L,
  n_samples = 20
)
sage$compute()


# For alternative conditional samplers:
custom_sampler = ConditionalGaussianSampler$new(
  task = task
)
sage_custom = ConditionalSAGE$new(
  task = task,
  learner = lrn("regr.ranger", num.trees = 50),
  measure = msr("regr.mse"),
  n_permutations = 5L,
  n_samples = 20,
  sampler = custom_sampler
)
sage_custom$compute()

Conditional Feature Sampler

Description

Base class for conditional sampling methods where features are sampled conditionally on other features. This is an abstract class that should be extended by concrete implementations.

Super class

xplainfi::FeatureSampler -> ConditionalSampler

Methods

Public methods

• ConditionalSampler$new()

• ConditionalSampler$sample()

• ConditionalSampler$sample_newdata()

• ConditionalSampler$clone()

Method new()

Creates a new instance of the ConditionalSampler class

Usage

ConditionalSampler$new(task, conditioning_set = NULL)

Arguments

Method sample()

Sample from stored task conditionally on other features.

Usage

ConditionalSampler$sample(
  feature,
  row_ids = NULL,
  conditioning_set = NULL,
  ...
)

Arguments

Returns

Modified copy with sampled feature(s).

Method sample_newdata()

Sample from external data conditionally.

Usage

ConditionalSampler$sample_newdata(
  feature,
  newdata,
  conditioning_set = NULL,
  ...
)

Arguments

Returns

Modified copy with sampled feature(s).

Method clone()

The objects of this class are cloneable with this method.

Usage

ConditionalSampler$clone(deep = FALSE)

Arguments

Feature Importance Method Class

Description

Feature Importance Method Class

Feature Importance Method Class

Public fields

Methods

Public methods

• FeatureImportanceMethod$new()

• FeatureImportanceMethod$compute()

• FeatureImportanceMethod$importance()

• FeatureImportanceMethod$obs_loss()

• FeatureImportanceMethod$reset()

• FeatureImportanceMethod$print()

• FeatureImportanceMethod$scores()

• FeatureImportanceMethod$clone()

Method new()

Creates a new instance of this R6 class. This is typically intended for use by derived classes.

Usage

FeatureImportanceMethod$new(
  task,
  learner,
  measure = NULL,
  resampling = NULL,
  features = NULL,
  groups = NULL,
  param_set = paradox::ps(),
  label
)

Arguments

Method compute()

Compute feature importance scores

Usage

FeatureImportanceMethod$compute(store_backends = TRUE)

Arguments

Method importance()

Get aggregated importance scores. The stored measure object's aggregator (default: mean) will be used to aggregated importance scores across resampling iterations and, depending on the method use, permutations (PerturbationImportance or refits LOCO).

Usage

FeatureImportanceMethod$importance(
  relation = NULL,
  standardize = FALSE,
  ci_method = c("none", "raw", "nadeau_bengio", "quantile"),
  conf_level = 0.95,
  ...
)

Arguments

Details

Variance estimates for importance scores are biased due to the resampling procedure. Molnar et al. (2023) suggest to use the variance correction factor proposed by Nadeau & Bengio (2003) of n2/n1, where n2 and n1 are the sizes of the test- and train set, respectively. This should then be combined with approx. 15 iterations of either bootstrapping or subsampling.

The use of bootstrapping in this context can lead to problematic information leakage when combined with learners that perform bootstrapping themselves, e.g., Random Forest learners. In such cases, observations may be used as train- and test instances simultaneously, leading to erroneous performance estimates.

An approach leading to still imperfect, but improved variance estimates could be:

PFI$new(
  task = sim_dgp_interactions(n = 1000),
  learner = lrn("regr.ranger", num.trees = 100),
  measure = msr("regr.mse"),
  # Subsampling instead of bootstrapping due to RF
  resampling = rsmp("subsampling", repeats = 15),
  n_repeats = 5
)

n_repeats = 5 in this context only improves the stability of the PFI estimate within the resampling iteration, whereas rsmp("subsampling", repeats = 15) is used to account for learner variance and necessitates variance correction.

This approach can in principle also be applied to CFI and RFI, but beware that a conditional sample such as ConditionalARFSampler also needs to be trained on data, which would need to be taken account by the variance estimation method. Analogously, the "nadeau_bengio" correction was recommended for the use with PFI by Molnar et al., so its use with other methods like LOCO or SAGE is experimental.

Note that even if measure uses an aggregator function that is not the mean, variance estimation currently will always use mean() and var().

Returns

(data.table) Aggregated importance scores with columns "feature", "importance", and depending on ci_method also "se", "conf_lower", "conf_upper".

Method obs_loss()

Calculate observation-wise importance scores.

Requires that ⁠$compute()⁠ was run and that measure is decomposable and has an observation-wise loss (Measure$obs_loss()) associated with it. This is not the case for measure like classif.auc, which is not decomposable.

Usage

FeatureImportanceMethod$obs_loss(relation = NULL)

Arguments

Returns

(data.table) Observation-wise losses and importance scores with columns "feature", "iter_rsmp", "iter_repeat" (if applicable), "row_ids", "loss_baseline", "loss_post", and "obs_importance".

Method reset()

Resets all stored fields populated by ⁠$compute⁠: ⁠$resample_result⁠, ⁠$scores⁠, ⁠$obs_losses⁠, and ⁠$predictions⁠.

Usage

FeatureImportanceMethod$reset()

Method print()

Print importance scores

Usage

FeatureImportanceMethod$print(...)

Arguments

Method scores()

Calculate importance scores for each resampling iteration and sub-iterations (iter_rsmp in PFI for example).

Iteration-wise importance are computed on the fly depending on the chosen relation (difference or ratio) to avoid re-computation if only a different relation is needed.

Usage

FeatureImportanceMethod$scores(relation = NULL)

Arguments

Returns

(data.table) Iteration-wise importance scores with columns for "feature", iteration indices, baseline and post-modification scores, and "importance".

Method clone()

The objects of this class are cloneable with this method.

Usage

FeatureImportanceMethod$clone(deep = FALSE)

Arguments

References

Nadeau C, Bengio Y (2003). “Inference for the Generalization Error.” Machine Learning, 52(3), 239–281. doi:10.1023/A:1024068626366. Molnar C, Freiesleben T, König G, Herbinger J, Reisinger T, Casalicchio G, Wright M, Bischl B (2023). “Relating the Partial Dependence Plot and Permutation Feature Importance to the Data Generating Process.” In Longo L (ed.), Explainable Artificial Intelligence, 456–479. ISBN 978-3-031-44064-9, doi:10.1007/978-3-031-44064-9_24.

Feature Sampler Class

Description

Base class for implementing different sampling strategies for feature importance methods like PFI and CFI

Public fields

Methods

Public methods

• FeatureSampler$new()

• FeatureSampler$sample()

• FeatureSampler$sample_newdata()

• FeatureSampler$print()

• FeatureSampler$clone()

Method new()

Creates a new instance of the FeatureSampler class

Usage

FeatureSampler$new(task)

Arguments

Method sample()

Sample values for feature(s) from stored task

Usage

FeatureSampler$sample(feature, row_ids = NULL)

Arguments

Returns

Modified copy of the input features with the feature(s) sampled: A data.table with same number of columns and one row matching the supplied row_ids

Method sample_newdata()

Sample values for feature(s) using external data

Usage

FeatureSampler$sample_newdata(feature, newdata)

Arguments

Method print()

Print sampler

Usage

FeatureSampler$print(...)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

FeatureSampler$clone(deep = FALSE)

Arguments

Gaussian Knockoff Conditional Sampler

Description

A KnockoffSampler defaulting to second-order Gaussian knockoffs as created by knockoff::create.second_order.

Details

This is equivalent to KnockoffSampler using the default knockoff_fun.

Super classes

xplainfi::FeatureSampler -> xplainfi::KnockoffSampler -> KnockoffGaussianSampler

Public fields

Methods

Public methods

• KnockoffGaussianSampler$new()

• KnockoffGaussianSampler$clone()

Method new()

Creates a new instance using Gaussian knockoffs via knockoff::create.second_order.

Usage

KnockoffGaussianSampler$new(task, iters = 1)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

KnockoffGaussianSampler$clone(deep = FALSE)

Arguments

References

Watson D, Wright M (2021). “Testing Conditional Independence in Supervised Learning Algorithms.” Machine Learning, 110(8), 2107–2129. doi:10.1007/s10994-021-06030-6.

Blesch K, Watson D, Wright M (2023). “Conditional Feature Importance for Mixed Data.” AStA Advances in Statistical Analysis, 108(2), 259–278. doi:10.1007/s10182-023-00477-9.

Examples

library(mlr3)
task = tgen("2dnormals")$generate(n = 100)
# Create sampler
sampler = KnockoffGaussianSampler$new(task)
# Sample using row_ids from stored task
sampled_data = sampler$sample("x1")

Knockoff Sampler

Description

Implements conditional sampling using Knockoffs.

Details

The KnockoffSampler samples Knockoffs based on the task data. This class allows arbitrary knockoff_fun, which also means that no input checking against supported feature types can be done. Use KnockoffGaussianSampler for the Gaussian knockoff sampler for numeric features. Alternative knockoff samplers include knockoff_seq() from the seqknockoff package available on GitHub: https://github.com/kormama1/seqknockoff.

Knockoffs are related to the ConditionalSampler familty, with key differences: They do not allow specifying a conditioning_set

Super class

xplainfi::FeatureSampler -> KnockoffSampler

Public fields

Methods

Public methods

• KnockoffSampler$new()

• KnockoffSampler$sample()

• KnockoffSampler$clone()

Method new()

Creates a new instance of the KnockoffSampler class.

Usage

KnockoffSampler$new(
  task,
  knockoff_fun = function(x) knockoff::create.second_order(as.matrix(x)),
  iters = 1
)

Arguments

Method sample()

Sample from stored task using knockoff values. Replaces specified feature(s) with their knockoff counterparts from the pre-generated knockoff matrix.

Usage

KnockoffSampler$sample(feature, row_ids = NULL)

Arguments

Returns

Modified copy with knockoff feature(s).

Method clone()

The objects of this class are cloneable with this method.

Usage

KnockoffSampler$clone(deep = FALSE)

Arguments

References

Watson D, Wright M (2021). “Testing Conditional Independence in Supervised Learning Algorithms.” Machine Learning, 110(8), 2107–2129. doi:10.1007/s10994-021-06030-6.

Blesch K, Watson D, Wright M (2023). “Conditional Feature Importance for Mixed Data.” AStA Advances in Statistical Analysis, 108(2), 259–278. doi:10.1007/s10182-023-00477-9.

Examples

library(mlr3)
task = tgen("2dnormals")$generate(n = 100)
# Create sampler with default parameters
sampler = KnockoffSampler$new(task)
# Sample using row_ids from stored task
sampled_data = sampler$sample("x1")

Leave-One-Covariate-Out (LOCO)

Description

Calculates Leave-One-Covariate-Out (LOCO) scores.

Details

LOCO measures feature importance by comparing model performance with and without each feature. For each feature, the model is retrained without that feature and the performance difference (reduced_model_loss - full_model_loss) indicates the feature's importance. Higher values indicate more important features.

Super classes

xplainfi::FeatureImportanceMethod -> xplainfi::WVIM -> LOCO

Methods

Public methods

• LOCO$new()

• LOCO$compute()

• LOCO$clone()

Method new()

Creates a new instance of this R6 class.

Usage

LOCO$new(
  task,
  learner,
  measure = NULL,
  resampling = NULL,
  features = NULL,
  n_repeats = 1L
)

Arguments

Method compute()

Compute LOCO importances.

Usage

LOCO$compute(store_models = TRUE, store_backends = TRUE)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

LOCO$clone(deep = FALSE)

Arguments

References

Lei J, G'Sell M, Rinaldo A, Tibshirani R, Wasserman L (2018). “Distribution-Free Predictive Inference for Regression.” Journal of the American Statistical Association, 113(523), 1094–1111. doi:10.1080/01621459.2017.1307116.

Examples

library(mlr3)
library(mlr3learners)

task <- sim_dgp_correlated(n = 500)

loco <- LOCO$new(
  task = task,
  learner = lrn("regr.ranger", num.trees = 10),
  measure = msr("regr.mse")
)
loco$compute()
loco$importance()

Marginal Permutation Sampler

Description

Implements marginal permutation-based sampling for Permutation Feature Importance (PFI). Each specified feature is randomly shuffled (permuted) independently, breaking the relationship between the feature and the target as well as between rows.

Details

The permutation sampler randomly shuffles feature values across observations:

• Each feature is permuted independently within its column

• The association between feature values and target values is broken

• The association between feature values across rows is broken

• The marginal distribution of each feature is preserved

Important distinction from SAGE's "marginal" approach:

• MarginalPermutationSampler: Shuffles features independently, breaking row structure

• MarginalSAGE: Uses reference data but keeps rows intact (features in coalition stay together)

This is the classic approach used in Permutation Feature Importance (PFI) and assumes features are independent.

Super classes

xplainfi::FeatureSampler -> xplainfi::MarginalSampler -> MarginalPermutationSampler

Methods

Public methods

• MarginalPermutationSampler$new()

• MarginalPermutationSampler$clone()

Method new()

Creates a new instance of the MarginalPermutationSampler class.

Usage

MarginalPermutationSampler$new(task)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

MarginalPermutationSampler$clone(deep = FALSE)

Arguments

Examples

library(mlr3)
task = tgen("2dnormals")$generate(n = 10)
task$data()
sampler = MarginalPermutationSampler$new(task)

# Sample using row_ids from stored task
sampler$sample("x1")

# Or use external data
data = task$data()
sampler$sample_newdata("x1", newdata = data)

Marginal Reference Sampler

Description

Samples complete observations from reference data to replace feature values. This approach samples from the marginal distribution while preserving within-row feature dependencies.

Details

This sampler implements what is called "marginal imputation" in the SAGE literature (Covert et al. 2020). For each observation, it samples a complete row from reference data and takes the specified feature values from that row. This approach:

• Samples from the marginal distribution P(X_S) where S is the set of features

• Preserves dependencies within the sampled reference row

• Breaks dependencies between test and reference data

Terminology note: In SAGE literature, this is called "marginal imputation" because features outside the coalition are "imputed" by sampling from their marginal distribution. We use MarginalReferenceSampler to avoid confusion with missing data imputation and to clarify that it samples from reference data.

Comparison with other samplers:

• MarginalPermutationSampler: Shuffles each feature independently, breaking all row structure

• MarginalReferenceSampler: Samples complete rows, preserving within-row dependencies

• ConditionalSampler: Samples from P(X_S | X_{-S}), conditioning on other features

Use in SAGE:

This is the default approach for MarginalSAGE. For a test observation x and features to marginalize S, it samples a reference row x_ref and creates a "hybrid" observation combining x's coalition features with x_ref's marginalized features.

Super classes

xplainfi::FeatureSampler -> xplainfi::MarginalSampler -> MarginalReferenceSampler

Public fields

Methods

Public methods

• MarginalReferenceSampler$new()

• MarginalReferenceSampler$clone()

Method new()

Creates a new instance of the MarginalReferenceSampler class.

Usage

MarginalReferenceSampler$new(task, n_samples = NULL)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

MarginalReferenceSampler$clone(deep = FALSE)

Arguments

References

Covert I, Lundberg S, Lee S (2020). “Understanding Global Feature Contributions With Additive Importance Measures.” In Advances in Neural Information Processing Systems, volume 33, 17212–17223. https://proceedings.neurips.cc/paper/2020/hash/c7bf0b7c1a86d5eb3be2c722cf2cf746-Abstract.html.

Examples

library(mlr3)
task = tgen("friedman1")$generate(n = 100)

# Default: uses all task data as reference
sampler = MarginalReferenceSampler$new(task)
sampled = sampler$sample("important1", row_ids = 1:10)

# Subsample reference data to 50 rows
sampler_subsampled = MarginalReferenceSampler$new(task, n_samples = 50L)
sampled2 = sampler_subsampled$sample("important1", row_ids = 1:10)

Marginal SAGE

Description

SAGE with marginal sampling (features are marginalized independently). This is the standard SAGE implementation.

Super classes

xplainfi::FeatureImportanceMethod -> xplainfi::SAGE -> MarginalSAGE

Methods

Public methods

• MarginalSAGE$new()

• MarginalSAGE$clone()

Method new()

Creates a new instance of the MarginalSAGE class.

Usage

MarginalSAGE$new(
  task,
  learner,
  measure = NULL,
  resampling = NULL,
  features = NULL,
  n_permutations = 10L,
  batch_size = 5000L,
  n_samples = 100L,
  early_stopping = FALSE,
  se_threshold = 0.01,
  min_permutations = 10L,
  check_interval = 1L
)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

MarginalSAGE$clone(deep = FALSE)

Arguments

See Also

ConditionalSAGE

Examples

library(mlr3)
task = tgen("friedman1")$generate(n = 100)
sage = MarginalSAGE$new(
  task = task,
  learner = lrn("regr.ranger", num.trees = 50),
  measure = msr("regr.mse"),
  n_permutations = 3L,
  n_samples = 20
)
sage$compute()

Marginal Sampler Base Class

Description

Abstract base class for marginal sampling strategies that do not condition on other features. Marginal samplers sample from P(X_S), the marginal distribution of features S, ignoring any dependencies with other features.

Details

This class provides a common interface for different marginal sampling approaches:

• MarginalPermutationSampler: Shuffles features independently within the dataset

• MarginalReferenceSampler: Samples complete rows from reference data

Both approaches sample from the marginal distribution P(X_S), but differ in how they preserve or break within-row dependencies:

• Permutation breaks ALL dependencies (both with target and between features)

• Reference sampling preserves WITHIN-row dependencies but breaks dependencies with test data

Comparison with ConditionalSampler:

• MarginalSampler: Samples from P(X_S) - no conditioning

• ConditionalSampler: Samples from P(X_S | X_{-S})- conditions on other features

This base class implements the public ⁠$sample()⁠ and ⁠$sample_newdata()⁠ methods, delegating to private .sample_marginal() which subclasses must implement.

Super class

xplainfi::FeatureSampler -> MarginalSampler

Methods

Public methods

• MarginalSampler$sample()

• MarginalSampler$sample_newdata()

• MarginalSampler$clone()

Method sample()

Sample features from their marginal distribution.

Usage

MarginalSampler$sample(feature, row_ids = NULL)

Arguments

Returns

Modified copy with sampled feature(s).

Method sample_newdata()

Sample from external data.

Usage

MarginalSampler$sample_newdata(feature, newdata)

Arguments

Returns

Modified copy with sampled feature(s).

Method clone()

The objects of this class are cloneable with this method.

Usage

MarginalSampler$clone(deep = FALSE)

Arguments

Permutation Feature Importance

Description

Implementation of Permutation Feature Importance (PFI) using modular sampling approach. PFI measures the importance of a feature by calculating the increase in model error when the feature's values are randomly permuted, breaking the relationship between the feature and the target variable.

Details

Permutation Feature Importance was originally introduced by Breiman (2001) as part of the Random Forest algorithm. The method works by:

1. Computing baseline model performance on the original dataset

2. For each feature, randomly permuting its values while keeping other features unchanged

3. Computing model performance on the permuted dataset

4. Calculating importance as the difference (or ratio) between permuted and original performance

Super classes

xplainfi::FeatureImportanceMethod -> xplainfi::PerturbationImportance -> PFI

Methods

Public methods

• PFI$new()

• PFI$compute()

• PFI$clone()

Method new()

Creates a new instance of the PFI class

Usage

PFI$new(
  task,
  learner,
  measure = NULL,
  resampling = NULL,
  features = NULL,
  groups = NULL,
  relation = "difference",
  n_repeats = 1L,
  batch_size = NULL
)

Arguments

Method compute()

Compute PFI scores

Usage

PFI$compute(
  n_repeats = NULL,
  batch_size = NULL,
  store_models = TRUE,
  store_backends = TRUE
)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

PFI$clone(deep = FALSE)

Arguments

References

Breiman L (2001). “Random Forests.” Machine Learning, 45(1), 5–32. doi:10.1023/A:1010933404324. Fisher A, Rudin C, Dominici F (2019). “All Models Are Wrong, but Many Are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously.” Journal of Machine Learning Research, 20, 177. https://pmc.ncbi.nlm.nih.gov/articles/PMC8323609/. Strobl C, Boulesteix A, Kneib T, Augustin T, Zeileis A (2008). “Conditional Variable Importance for Random Forests.” BMC Bioinformatics, 9(1), 307. doi:10.1186/1471-2105-9-307.

Examples

library(mlr3)
library(mlr3learners)

task <- sim_dgp_correlated(n = 500)

pfi <- PFI$new(
  task = task,
  learner = lrn("regr.ranger", num.trees = 10),
  measure = msr("regr.mse")
)
pfi$compute()
pfi$importance()

Perturbation Feature Importance Base Class

Description

Abstract base class for perturbation-based importance methods PFI, CFI, and RFI

Super class

xplainfi::FeatureImportanceMethod -> PerturbationImportance

Public fields

Methods

Public methods

• PerturbationImportance$new()

• PerturbationImportance$importance()

• PerturbationImportance$clone()

Method new()

Creates a new instance of the PerturbationImportance class

Usage

PerturbationImportance$new(
  task,
  learner,
  measure = NULL,
  resampling = NULL,
  features = NULL,
  groups = NULL,
  sampler = NULL,
  relation = "difference",
  n_repeats = 1L,
  batch_size = NULL
)

Arguments

Method importance()

Get aggregated importance scores. Extends the base ⁠$importance()⁠ method to support the additional "cpi" ci_method.

Usage

PerturbationImportance$importance(
  relation = NULL,
  standardize = FALSE,
  ci_method = c("none", "raw", "nadeau_bengio", "quantile", "cpi"),
  conf_level = 0.95,
  test = c("t", "wilcoxon", "fisher", "binomial"),
  B = 1999,
  ...
)

Arguments

Returns

(data.table) Aggregated importance scores.

Method clone()

The objects of this class are cloneable with this method.

Usage

PerturbationImportance$clone(deep = FALSE)

Arguments

Relative Feature Importance

Description

RFI generalizes CFI and PFI with arbitrary conditioning sets and samplers.

Super classes

xplainfi::FeatureImportanceMethod -> xplainfi::PerturbationImportance -> RFI

Methods

Public methods

• RFI$new()

• RFI$compute()

• RFI$clone()

Method new()

Creates a new instance of the RFI class

Usage

RFI$new(
  task,
  learner,
  measure = NULL,
  resampling = NULL,
  features = NULL,
  groups = NULL,
  conditioning_set = NULL,
  relation = "difference",
  n_repeats = 1L,
  batch_size = NULL,
  sampler = NULL
)

Arguments

Method compute()

Compute RFI scores

Usage

RFI$compute(
  conditioning_set = NULL,
  n_repeats = NULL,
  batch_size = NULL,
  store_models = TRUE,
  store_backends = TRUE
)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

RFI$clone(deep = FALSE)

Arguments

References

König G, Molnar C, Bischl B, Grosse-Wentrup M (2021). “Relative Feature Importance.” In 2020 25th International Conference on Pattern Recognition (ICPR), 9318–9325. doi:10.1109/ICPR48806.2021.9413090.

Examples

library(mlr3)
task = tgen("friedman1")$generate(n = 200)
rfi = RFI$new(
  task = task,
  learner = lrn("regr.ranger", num.trees = 50),
  measure = msr("regr.mse"),
  conditioning_set = c("important1")
)
rfi$compute()
rfi$importance()

Shapley Additive Global Importance (SAGE) Base Class

Description

Base class for SAGE (Shapley Additive Global Importance) feature importance based on Shapley values with marginalization. This is an abstract class - use MarginalSAGE or ConditionalSAGE.

Details

SAGE uses Shapley values to fairly distribute the total prediction performance among all features. Unlike perturbation-based methods, SAGE marginalizes features by integrating over their distribution. This is approximated by averaging predictions over a reference dataset.

Standard Error Calculation: The standard errors (SE) reported in ⁠$convergence_history⁠ reflect the uncertainty in Shapley value estimation across different random permutations within a single resampling iteration. These SEs quantify the Monte Carlo sampling error for a fixed trained model and are only valid for inference about the importance of features for that specific model. They do not capture broader uncertainty from model variability across different train/test splits or resampling iterations.

Super class

xplainfi::FeatureImportanceMethod -> SAGE

Public fields

Methods

Public methods

• SAGE$new()

• SAGE$compute()

• SAGE$plot_convergence()

• SAGE$clone()

Method new()

Creates a new instance of the SAGE class.

Usage

SAGE$new(
  task,
  learner,
  measure = NULL,
  resampling = NULL,
  features = NULL,
  n_permutations = 10L,
  batch_size = 5000L,
  n_samples = 100L,
  early_stopping = TRUE,
  se_threshold = 0.01,
  min_permutations = 10L,
  check_interval = 1L
)

Arguments

Method compute()

Compute SAGE values.

Usage

SAGE$compute(
  store_backends = TRUE,
  batch_size = NULL,
  early_stopping = NULL,
  se_threshold = NULL,
  min_permutations = NULL,
  check_interval = NULL
)

Arguments

Method plot_convergence()

Plot convergence history of SAGE values.

Usage

SAGE$plot_convergence(features = NULL)

Arguments

Returns

A ggplot2 object

Method clone()

The objects of this class are cloneable with this method.

Usage

SAGE$clone(deep = FALSE)

Arguments

References

Covert I, Lundberg S, Lee S (2020). “Understanding Global Feature Contributions With Additive Importance Measures.” In Advances in Neural Information Processing Systems, volume 33, 17212–17223. https://proceedings.neurips.cc/paper/2020/hash/c7bf0b7c1a86d5eb3be2c722cf2cf746-Abstract.html.

See Also

MarginalSAGE ConditionalSAGE

Williamson's Variable Importance Measure (WVIM)

Description

Base class generalizing refit-based variable importance measures. Default corresponds to leaving out each feature n_repeats times, which corresponds to LOCO (Leave One Covariate Out).

Super class

xplainfi::FeatureImportanceMethod -> WVIM

Public fields

Methods

Public methods

• WVIM$new()

• WVIM$compute()

• WVIM$clone()

Method new()

Creates a new instance of this R6 class.

Usage

WVIM$new(
  task,
  learner,
  measure = NULL,
  resampling = NULL,
  features = NULL,
  groups = NULL,
  direction = c("leave-out", "leave-in"),
  label = "Williamson's Variable Importance Measure (WVIM)",
  n_repeats = 1L
)

Arguments

Method compute()

Computes leave-out or leave-in feature importance. wvim_design_matrix(task$feature_names, "leave-out") corresponds to LOCO.

Usage

WVIM$compute(
  store_models = TRUE,
  store_backends = TRUE,
  store_instance = FALSE
)

Arguments

Method clone()

The objects of this class are cloneable with this method.

Usage

WVIM$clone(deep = FALSE)

Arguments

Examples

library(mlr3)
library(mlr3learners)

task <- sim_dgp_correlated(n = 500)

# Group correlated features together, independent features separately
groups <- list(
  correlated = c("x1", "x2"),
  independent = c("x3", "x4")
)

wvim <- WVIM$new(
  task = task,
  learner = lrn("regr.ranger", num.trees = 10),
  groups = groups
)
wvim$compute()
wvim$importance()

Check group specification

Description

Check group specification

Usage

check_groups(groups, all_features)

Arguments

Value

The input list group, with each element now named.

Examples

task <- sim_dgp_interactions(n = 100)
task$feature_names

# Intended use
groups1 = list(effects = c("x1", "x2", "x3"), noise = c("noise1", "noise2"))
check_groups(groups1, task$feature_names)

# Names are auto-generated where needed
check_groups(list(a = "x1",  c("x2", "x1")), task$feature_names)

# Examples for cases that throw errors:

# Unexpected features
groups2 = list(effects = c("x1", "foo", "bar", "x1"))
try(check_groups(groups2, task$feature_names))
# Too deeply nested
groups3 = list(effects = c("x1", "x2", "x3"), noise = c("noise1", list(c("noise2"))))
try(check_groups(groups2, task$feature_names))

Default value for NULL

Description

A backport of ⁠%||%⁠ available in R versions from 4.4.0.

Usage

x %||% y

Arguments

Examples

1 %||% 2
NULL %||% 2

Print an Rd-formatted bib entry

Description

Print an Rd-formatted bib entry

Usage

print_bib(...)

Arguments

Aggregate Predictions by Coalition and Test Instance

Description

Averages predictions across multiple samples (reference data or conditional samples) for each unique combination of coalition and test instance.

Usage

sage_aggregate_predictions(
  combined_data,
  predictions,
  task_type,
  class_names = NULL
)

Arguments

Value

data.table with columns:

• .coalition_id: Coalition identifier (integer)

• .test_instance_id: Test instance identifier (integer)

• For classification: One column per class with averaged probabilities (numeric)

• For regression: avg_pred column with averaged predictions (numeric)

Batch Predict for SAGE

Description

Performs batched prediction on combined data to manage memory usage. Supports both classification (probability predictions) and regression.

Usage

sage_batch_predict(learner, combined_data, task, batch_size, task_type)

Arguments

Value

For classification: matrix of class probabilities (n_rows x n_classes). For regression: numeric vector of predictions (length n_rows).

Simulate data as in Ewald et al. (2024)

Description

Reproduces the data generating process from Ewald et al. (2024) for benchmarking feature importance methods. Includes correlated features and interaction effects.

Usage

sim_dgp_ewald(n = 500)

Arguments

Details

Mathematical Model:

X_1, X_3, X_5 \sim \text{Uniform}(0,1)

X_2 = X_1 + \varepsilon_2, \quad \varepsilon_2 \sim N(0, 0.001)

X_4 = X_3 + \varepsilon_4, \quad \varepsilon_4 \sim N(0, 0.1)

Y = X_4 + X_5 + X_4 \cdot X_5 + \varepsilon, \quad \varepsilon \sim N(0, 0.1)

Feature Properties:

• X1, X3, X5: Independent uniform(0,1) distributions

• X2: Nearly perfect copy of X1 (correlation approximately 0.99)

• X4: Noisy copy of X3 (correlation approximately 0.67)

• Y depends on X4, X5, and their interaction

Value

A regression task (mlr3::TaskRegr) with data.table backend.

References

Ewald F, Bothmann L, Wright M, Bischl B, Casalicchio G, König G (2024). “A Guide to Feature Importance Methods for Scientific Inference.” In Longo L, Lapuschkin S, Seifert C (eds.), Explainable Artificial Intelligence, 440–464. ISBN 978-3-031-63797-1, doi:10.1007/978-3-031-63797-1_22.

See Also

Other simulation: sim_dgp_scenarios

Examples

sim_dgp_ewald(100)

Simulation DGPs for Feature Importance Method Comparison

Description

These data generating processes (DGPs) are designed to illustrate specific strengths and weaknesses of different feature importance methods like PFI, CFI, and RFI. Each DGP focuses on one primary challenge to make the differences between methods clear.

Usage

sim_dgp_correlated(n = 500L, r = 0.9)

sim_dgp_mediated(n = 500L)

sim_dgp_confounded(n = 500L, hidden = TRUE)

sim_dgp_interactions(n = 500L)

sim_dgp_independent(n = 500L)

Arguments

Details

Correlated Features DGP: This DGP creates highly correlated predictors where PFI will show artificially low importance due to redundancy, while CFI will correctly identify each feature's conditional contribution.

Mathematical Model:

(X_1, X_2)^T \sim \text{MVN}(0, \Sigma)

where \Sigma is a 2 \times 2 covariance matrix with 1 on the diagonal and correlation r on the off-diagonal.

X_3 \sim N(0,1), \quad X_4 \sim N(0,1)

Y = 2 \cdot X_1 + X_3 + \varepsilon

where \varepsilon \sim N(0, 0.2^2).

Feature Properties:

• x1: Standard normal from MVN, direct causal effect on y (\beta = 2.0)

• x2: Correlated with x1 (correlation = r), NO causal effect on y (\beta = 0)

• x3: Independent standard normal, direct causal effect on y (\beta = 1.0)

• x4: Independent standard normal, no effect on y (\beta = 0)

Expected Behavior:

• Marginal methods (PFI, Marginal SAGE): Will falsely assign importance to x2 due to correlation with x1

• Conditional methods (CFI, Conditional SAGE): Should correctly assign near-zero importance to x2

• Key insight: x2 is a "spurious predictor" - correlated with causal feature but not causal itself

Mediated Effects DGP: This DGP demonstrates the difference between total and direct causal effects. Some features affect the outcome only through mediators.

Mathematical Model:

\text{exposure} \sim N(0,1), \quad \text{direct} \sim N(0,1)

\text{mediator} = 0.8 \cdot \text{exposure} + 0.6 \cdot \text{direct} + \varepsilon_m

Y = 1.5 \cdot \text{mediator} + 0.5 \cdot \text{direct} + \varepsilon

where \varepsilon_m \sim N(0, 0.3^2) and \varepsilon \sim N(0, 0.2^2).

Feature Properties:

• exposure: Has no direct effect on y, only through mediator (total effect = 1.2)

• mediator: Mediates the effect of exposure on y

• direct: Has both direct effect on y and effect on mediator

• noise: No causal relationship to y

Causal Structure: exposure -> mediator -> y <- direct -> mediator

Expected Behavior:

• PFI: Shows total effects (exposure appears important)

• CFI: Shows direct effects (exposure appears less important when conditioning on mediator)

• RFI with mediator: Should show direct effects similar to CFI

Confounding DGP: This DGP includes a confounder that affects both a feature and the outcome. Uses simple coefficients for easy interpretation.

Mathematical Model:

H \sim N(0,1)

X_1 = H + \varepsilon_1

\text{proxy} = H + \varepsilon_p, \quad \text{independent} \sim N(0,1)

Y = H + X_1 + \text{independent} + \varepsilon

where all \varepsilon \sim N(0, 0.5^2) independently.

Model Structure:

• Confounder H ~ N(0,1) (potentially unobserved)

• x1 = H + noise (affected by confounder)

• proxy = H + noise (noisy measurement of confounder)

• independent ~ N(0,1) (truly independent)

• y = H + x1 + independent + noise

Expected Behavior:

• PFI: Will show inflated importance for x1 due to confounding

• CFI: Should partially account for confounding through conditional sampling

• RFI conditioning on proxy: Should reduce confounding bias by conditioning on proxy

Interaction Effects DGP: This DGP demonstrates a pure interaction effect where features have no main effects.

Mathematical Model:

Y = 2 \cdot X_1 \cdot X_2 + X_3 + \varepsilon

where X_j \sim N(0,1) independently and \varepsilon \sim N(0, 0.5^2).

Feature Properties:

• x1, x2: Independent features with ONLY interaction effect (no main effects)

• x3: Independent feature with main effect only

• noise1, noise2: No causal effects

Expected Behavior:

• PFI: Should assign near-zero importance to x1 and x2 (no marginal effect)

• CFI: Should capture the interaction and assign high importance to x1 and x2

• Ground truth: x1 and x2 are important ONLY through their interaction

Independent Features DGP: This is a baseline scenario where all features are independent and their effects are additive. All importance methods should give similar results.

Mathematical Model:

Y = 2.0 \cdot X_1 + 1.0 \cdot X_2 + 0.5 \cdot X_3 + \varepsilon

where X_j \sim N(0,1) independently and \varepsilon \sim N(0, 0.2^2).

Feature Properties:

• important1-3: Independent features with different effect sizes

• unimportant1-2: Independent noise features with no effect

Expected Behavior:

• All methods: Should rank features consistently by their true effect sizes

• Ground truth: important1 > important2 > important3 > unimportant1,2 (approximately 0)

Value

A regression task (mlr3::TaskRegr) with data.table backend.

Functions

• sim_dgp_correlated(): Correlated features demonstrating PFI's limitations

• sim_dgp_mediated(): Mediated effects showing direct vs total importance

• sim_dgp_confounded(): Confounding scenario for conditional sampling

• sim_dgp_interactions(): Interaction effects between features

• sim_dgp_independent(): Independent features baseline scenario

References

Ewald F, Bothmann L, Wright M, Bischl B, Casalicchio G, König G (2024). “A Guide to Feature Importance Methods for Scientific Inference.” In Longo L, Lapuschkin S, Seifert C (eds.), Explainable Artificial Intelligence, 440–464. ISBN 978-3-031-63797-1, doi:10.1007/978-3-031-63797-1_22.

See Also

Other simulation: sim_dgp_ewald()

Other simulation: sim_dgp_ewald()

Other simulation: sim_dgp_ewald()

Other simulation: sim_dgp_ewald()

Other simulation: sim_dgp_ewald()

Examples

task = sim_dgp_correlated(200)
task$data()

# With different correlation
task_high_cor = sim_dgp_correlated(200, r = 0.95)
cor(task_high_cor$data()$x1, task_high_cor$data()$x2)
task = sim_dgp_mediated(200)
task$data()
# Hidden confounder scenario (traditional)
task_hidden = sim_dgp_confounded(200, hidden = TRUE)
task_hidden$feature_names  # proxy available but not confounder

# Observable confounder scenario
task_observed = sim_dgp_confounded(200, hidden = FALSE)
task_observed$feature_names  # both confounder and proxy available
task = sim_dgp_interactions(200)
task$data()
task = sim_dgp_independent(200)
task$data()

Create Feature Selection Design Matrix

Description

Creates a logical design matrix for leave-in or leave-out feature evaluation. Used internally with mlr3fselect to evaluate feature subsets.

Usage

wvim_design_matrix(
  all_features,
  feature_names = all_features,
  direction = c("leave-out", "leave-in")
)

Arguments

Value

data.table with logical columns for each feature in all_features and length(feature_names) rows, one for each entry in feature_names

Examples

task = mlr3::tsk("mtcars")

# Individual features
feature_names = task$feature_names[1:3]
wvim_design_matrix(task$feature_names, feature_names, "leave-in")
wvim_design_matrix(task$feature_names, feature_names, "leave-out")

# Feature groups
feature_groups = list(
  A = task$feature_names[1:2],
  B = task$feature_names[3:5]
)
wvim_design_matrix(task$feature_names, feature_groups, "leave-out")

xplainfi Package Options

Description

Get or set package-level options for xplainfi.

Usage

xplain_opt(...)

Arguments

Details

Options can be set in three ways (in order of precedence):

1. Using xplain_opt(option_name = value) (recommended)

2. Using options("xplain.option_name" = value)

3. Using environment variables XPLAIN_OPTION_NAME=value

Available Options

Value

• When getting a single option: the option value (logical)

• When getting multiple options: a named list of option values

• When setting options: the previous values (invisibly)

Examples

# Get current value of an option
xplain_opt("verbose")

# Get all options
xplain_opt()

# Set an option (returns previous value invisibly)
old <- xplain_opt(verbose = FALSE)
xplain_opt("verbose")  # Now FALSE

# Restore previous value
xplain_opt(verbose = old$verbose)

# Temporary option change with withr
if (requireNamespace("withr", quietly = TRUE)) {
  withr::with_options(
    list("xplain.verbose" = FALSE),
    {
      # Code here runs with verbose = FALSE
      xplain_opt("verbose")
    }
  )
}