# Common pitfalls / FAQ

This vignette collects the questions early users most often hit when running pigauto on their own data. Each section follows the same template:

- Symptom — the surprising output, in user voice.
- Why this happens — the mechanism in 2–3 sentences.
- Diagnose — 1–3 R commands that confirm the pattern.
- Fix — concrete code change with a short explanation.
- See also — links to the relevant `?function`, design memo, or Methodology bench under the Methodology navbar dropdown.

If you hit something that isn’t here and feels surprising, please open an issue — most of the items below were added because real users tripped on them.

## 1. “I called `impute()` and `result$prediction$imputed` looks like my input”

Symptom. You run `result <- impute(df, tree)` on a fully-observed dataset (e.g. the bundled `avonet300`) and read `result$prediction$imputed$Mass` expecting “the imputed values” — but the values look exactly like your input data, including legitimately huge ones (a 25 kg rhea, a 12 kg vulture).

Why this happens. `impute()` only imputes cells that are `NA` in the input. Your input was fully observed, so nothing was imputed: `result$completed` equals the input, `sum(result$imputed_mask)` is zero, and `result$prediction$imputed` contains the model’s prediction for every cell — observed and missing alike. For observed cells, the well-calibrated gate keeps the prediction close to the input value, so what comes back is essentially the original data passed through. The slot is intended for diagnostics (checking calibration on training cells), not as the imputed-values output.

Diagnose.

    library(pigauto)
    data(avonet300, tree300)
    df <- avonet300
    rownames(df) <- df$Species_Key
    df$Species_Key <- NULL
    sum(is.na(df))                  # if 0, there's nothing for impute() to do

Fix. Mask some cells before calling `impute()`, then evaluate predictions only on the held-out cells:

    set.seed(1L)
    hide   <- sample(which(!is.na(df$Mass)), 30L)
    df_obs <- df
    df_obs$Mass[hide] <- NA          # hide 30 mass values

    result <- impute(df_obs, tree300)

    result$completed$Mass[hide]      # pigauto's imputations
    df$Mass[hide]                    # held-out truth, for comparison
    sum(result$imputed_mask[, "Mass"])  # 30

For your own data with real `NA`s, the imputed values you actually care about are `result$completed[result$imputed_mask]`, not `result$prediction$imputed`.

See also. `?impute` (“What gets imputed (read this first)”), issue #67.

## 2. “My ordinal trait predicted 100 % majority class”

Symptom. You impute an ordinal trait and the prediction is the majority class for every species. For example, on `avonet300$Migration` (K = 3 ordinal: Resident / Partial / Full), 300/0/0.

Why this happens. Two things compound:

1. If your input has no `NA`s in that column, there’s nothing to impute externally (see Pitfall 1) — `result$prediction$imputed$Migration` reflects the model’s calibrated-gate output, not new imputations.
2. At default settings (`n_imputations = 1L`, `pool_method = "median"`), a small ordinal trait whose marginal distribution is heavily skewed (AVONET `Migration` is ~78 % Resident / 14 % Partial / 8 % Full at n = 300) can have its calibrated gate snap to a corner that returns the majority class for every species.

Diagnose.

    library(pigauto)
    data(avonet300, tree300)
    df <- avonet300; rownames(df) <- df$Species_Key; df$Species_Key <- NULL

    table(df$Migration)              # check the marginal distribution
    result <- impute(df, tree300, verbose = FALSE)
    table(result$prediction$imputed$Migration)

Fix. For imbalanced K-class ordinal traits, increase `n_imputations` and switch to `pool_method = "mode"` (Phase H). On the AVONET multi-seed bench this gave +6.6 percentage-point accuracy on `Migration` (K = 3) versus the default median pool.

    set.seed(1L)
    hide   <- sample(which(!is.na(df$Migration)), 30L)
    df_obs <- df
    df_obs$Migration[hide] <- NA

    # Default settings: prone to majority-class collapse on imbalanced K = 3
    result <- impute(df_obs, tree300, verbose = FALSE)
    table(result$completed$Migration[hide], df$Migration[hide])

    # Recommended for K = 3 ordinal: more draws + mode pooling
    result_mode <- impute(df_obs, tree300, n_imputations = 20L,
                          pool_method = "mode", verbose = FALSE)
    table(result_mode$completed$Migration[hide], df$Migration[hide])

See also. `?impute` (“Imbalanced K-class traits”), Phase H memo, issue #68.

## 3. “The gate stays closed and the GNN seems to do nothing”

Symptom. You expected the GNN to dominate, but inspecting the fitted model shows the calibrated gate is fully or near-fully closed (`r_cal_gnn` ≈ 0) — predictions equal the BM baseline.

Why this happens. This is the safety-floor design behaviour, not a bug. After training, pigauto picks the per-latent-column gate that minimises validation loss across the simplex \(r_\text{BM} \cdot \text{BM} + r_\text{GNN} \cdot \text{GNN} + r_\text{MEAN} \cdot \text{MEAN}\). When the GNN cannot beat BM on the held-out validation set, the optimum can be `r_cal_gnn = 0`. In that case the calibrated prediction stays on the validation-supported baseline or mean corner instead of forcing a GNN contribution. This is what the package was designed to do on high-phylogenetic-signal traits where BM is already hard to beat.

Diagnose.

    library(pigauto)
    data(avonet300, tree300)
    df <- avonet300; rownames(df) <- df$Species_Key; df$Species_Key <- NULL
    fit <- impute(df, tree300, verbose = FALSE)$fit

    # Per-latent-column calibrated gates (since v0.9.1.9002):
    fit$r_cal_bm        # r assigned to the BM baseline
    fit$r_cal_gnn       # r assigned to the GNN delta
    fit$r_cal_mean      # r assigned to the grand mean

A row where `r_cal_gnn` is small (< 0.1) means the gate has effectively closed for that latent column.

Fix. Often there is nothing to fix — the closed gate is evidence of high phylogenetic signal, not a problem. If you suspect the GNN should be helping (e.g. you’ve added covariates, or the trait has known cross-trait structure) but the gate is closed:

- Check the validation set is not pathologically small (the “small validation set” warning during fitting is a red flag — see `?fit_pigauto` “Calibration at small n”).
- Verify covariates are not all-NA or constant after preprocessing.
- For ordinal / categorical traits, see Pitfall 2 — the gate may be closing onto a majority-class corner that mode pooling resolves.

See also. `?fit_pigauto` (`phylo_signal_gate`, “Safety floor”), design spec.

## 4. “How do I know if my dataset has enough phylogenetic signal?”

Symptom. You aren’t sure whether pigauto’s BM kriging baseline will outperform a simple mean impute on your dataset.

Why this matters. pigauto’s BM baseline buys you accuracy in proportion to phylogenetic signal in the trait. At Pagel’s λ ≈ 0 (no signal), BM kriging reduces to the species mean and pigauto won’t beat a simple mean baseline; at λ ≈ 1 (strong signal), BM kriging materially outperforms the mean. The Phase 8 signal-strength sweep (re-running locally produces the evidence; the deployed Methodology dropdown surfaces it once the bench HTML is regenerated) shows the crossover empirically.

Diagnose. The fitted object stores the per-trait λ values used by `phylo_signal_gate`:

    library(pigauto)
    data(avonet300, tree300)
    df <- avonet300; rownames(df) <- df$Species_Key; df$Species_Key <- NULL

    fit <- impute(df, tree300)$fit
    fit$phylo_signal_per_trait

The output reports λ on the observed cells where it can be estimated. Discrete traits use the package’s internal continuous proxy on the latent/liability scale.

Fix. Use the lambda estimate to set expectations:

- λ > 0.7: pigauto’s BM baseline alone usually beats grand-mean. GNN may add little if the trait is well-conserved.
- 0.3 < λ < 0.7: pigauto’s GNN typically helps on top of BM, especially if you have covariates.
- λ < 0.3: phylogenetic information is weak. Consider whether a phylogenetic imputation method is the right tool at all. A simple mean impute or a covariate-only regression may do as well.

See also. `?fit_pigauto` (`phylo_signal_gate`), Phase 8 signal sweep memo.

## 5. “Predictions are way bigger than anything I observed”

Symptom. A masked log-transformed continuous trait (body mass, seed mass, fish weight) predicts a value 50–100× larger than anything observed. On AVONET, the canonical case is the cassowary: truth ≈ 35 kg, predicted up to ~540 kg.

Why this happens. For log-transformed traits, the GNN’s MC-dropout draws are on the log scale. A latent ~+3-4 σ above the training distribution survives as a ~50-100× value error after `expm1()` back-transformation. With `n_imputations = 1`, a single unlucky dropout pattern can produce this; with `pool_method = "median"` (default) the median of M draws is robust to one bad draw, but a small M (≤ 5) on the long tail of the latent distribution can still mis-pool.

Diagnose.

    # After running impute(), check whether any imputation exceeds the
    # observed maximum by an unrealistic factor:
    predicted_mass <- result$completed$Mass[result$imputed_mask[, "Mass"]]
    obs_max <- max(df$Mass, na.rm = TRUE)
    sum(predicted_mass > 5 * obs_max)

A non-zero count is a signal of tail extrapolation.

Fix. Phase G `clamp_outliers = TRUE` caps post-back-transform predictions for log-transformed continuous, count, and `zi_count` magnitude traits at `obs_max * clamp_factor` (default 5). This is opt-in because for legitimate growth-curve datasets where 5× the observed maximum is plausible, you don’t want it.

    result <- impute(df_obs, tree300,
                     clamp_outliers = TRUE,
                     clamp_factor   = 5,    # Tukey-style outlier cap
                     verbose = FALSE)

See also. `?impute` (`clamp_outliers`, `clamp_factor` arguments), AVONET Mass diagnosis memo, Phase G results.