Introduction to AIBias: Longitudinal Bias Auditing

Overview

Standard fairness audits treat bias as a snapshot — a single-point disparity measurement at one moment in time. But in sequential decision systems (loan approvals, parole reviews, hiring pipelines, credit scoring), decisions at time \(t\) feed back into the features available at time \(t+1\).

AIBias treats algorithmic bias as a longitudinal process. It tracks:

How group disparities evolve over repeated decisions
Where disparities compound via transition dynamics
Whether earlier decisions amplify later inequality

The Synthetic Lending Dataset

The package ships with lending_panel, a 600-applicant × 6-year panel of synthetic loan decisions across three racial groups.

data(lending_panel)
head(lending_panel)
#>   applicant_id year     race income credit_score approved
#> 1     APP_0001 2015 Hispanic     58          719        1
#> 2     APP_0001 2016 Hispanic     69          759        1
#> 3     APP_0001 2017 Hispanic     59          580        1
#> 4     APP_0001 2018 Hispanic     69          677        1
#> 5     APP_0001 2019 Hispanic     56          674        0
#> 6     APP_0001 2020 Hispanic     57          619        0

# Pooled approval rates by group
tapply(lending_panel$approved, lending_panel$race, mean) |> round(3)
#>    White    Black Hispanic 
#>    0.775    0.514    0.579

Even from this simple tabulation we see a gap. But it misses the dynamic story: are disadvantaged groups less able to recover from a denial? Are gaps widening over time?

Step 1 — Build the Audit Object

obj <- aib_build(
  data     = lending_panel,
  id       = "applicant_id",
  time     = "year",
  group    = "race",
  decision = "approved"
)
print(obj)
#> 
#> == AIBias Audit Object ==
#>   Units       : 600 
#>   Time points : 6 ( 2015, 2016, 2017, 2018, 2019, 2020 )
#>   Groups      : White, Black, Hispanic 
#>   Decision    : approved 
#> 
#>   Components populated:
#>     ✕ bias 
#>     ✕ transitions 
#>     ✕ amplification 
#>     ✕ adjusted 
#>     ✕ bootstrap

Step 2 — Describe Bias Trajectories

aib_describe() computes:

\(\hat{\pi}_g(t)\): group-specific decision rate at each wave
\(\hat{B}_{g,r}(t) = \hat{\pi}_g(t) - \hat{\pi}_r(t)\): raw bias trajectory
\(\hat{B}^*_{g,r}(t)\): standardized (SMD) trajectory
\(CB_{g,r}(T)\): cumulative bias burden

obj <- aib_describe(obj, ref_group = "White")
obj$bias$cumulative
#> # A tibble: 2 × 7
#>   group    ref_group CB_raw CB_normalized CB_smd peak_disparity time_to_peak
#>   <chr>    <chr>      <dbl>         <dbl>  <dbl>          <dbl>        <int>
#> 1 Black    White      -1.56        -0.261  -3.31          0.292            3
#> 2 Hispanic White      -1.17        -0.196  -2.49          0.304            3

The cumulative burden (CB_normalized) summarizes the average disparity experienced across all waves — a single policy-facing number.

Trajectory Plot

plot(obj, type = "trajectory")

Bias trajectory plot

Both groups show a persistent negative disparity from wave 1. The gap is relatively stable, suggesting a persistent rather than purely growing pattern — but the dynamic analysis below reveals compounding beneath the surface.

Heatmap — Disparity Surface

plot(obj, type = "heatmap")

Group-time disparity heatmap

The heatmap displays the full group × time disparity surface. Red cells indicate disadvantaged periods.

Step 3 — Transition Analysis

The key question for compounding bias: are disadvantaged groups less likely to recover after a denial, and less likely to retain approval?

obj <- aib_transition(obj, ref_group = "White")

# Recovery and retention gaps
obj$transitions$recovery_gap
#> # A tibble: 2 × 4
#>   group    p_recovery_g p_recovery_ref recovery_gap
#>   <fct>           <dbl>          <dbl>        <dbl>
#> 1 Black           0.41           0.656       -0.246
#> 2 Hispanic        0.464          0.656       -0.193
obj$transitions$retention_gap
#> # A tibble: 2 × 4
#>   group    p_retention_g p_retention_ref retention_gap
#>   <fct>            <dbl>           <dbl>         <dbl>
#> 1 Black            0.629           0.821        -0.192
#> 2 Hispanic         0.670           0.821        -0.151

plot(obj, type = "transition")

Transition probabilities plot

The transition plot reveals the mechanism of compounding. Despite some overall disparity in approval rates, the recovery gap is the most consequential finding: after a denial, Black applicants recover at a much lower rate than White applicants, locking them into unfavorable states.

Markov State Evolution

The Markov amplification operator \(A^{state}_{g,r}(T) = \sum_t \|v_g(t) - v_r(t)\|\) quantifies cumulative divergence in state distributions:

obj$transitions$amp_state
#> # A tibble: 2 × 2
#>   group    A_state
#>   <chr>      <dbl>
#> 1 Black       1.56
#> 2 Hispanic    1.15

Step 4 — Amplification Analysis

The amplification index measures:

\[A_{g,r}(t) = B_{g,r}(t \mid 1) - B_{g,r}(t \mid 0)\]

If \(A_{g,r}(t) \neq 0\), prior decision state is modifying the group disparity — the hallmark of dynamic rather than static bias.

obj <- aib_amplify(obj, ref_group = "White")
obj$amplification$cumulative
#> # A tibble: 2 × 5
#>   group    A_cumulative A_mean A_max T_amplifying
#>   <chr>           <dbl>  <dbl> <dbl>        <int>
#> 1 Black           0.266 0.0533 0.288            5
#> 2 Hispanic        0.212 0.0423 0.187            5

plot(obj, type = "amplification")

Amplification index plot

Narrative Interpretation

obj$amplification$narratives
#> # A tibble: 2 × 2
#>   group    narrative                                                            
#>   <chr>    <chr>                                                                
#> 1 Black    Recovery disparity = -0.246: group Black is less likely to regain fa…
#> 2 Hispanic Recovery disparity = -0.193: group Hispanic is less likely to regain…

Step 5 — Covariate Adjustment

To separate “case mix” differences from residual disparity, fit a covariate-adjusted model:

obj <- aib_adjust(
  obj,
  formula   = ~ income + credit_score,
  method    = "glm",
  ref_group = "White"
)

# Adjusted trajectory
head(obj$adjusted$trajectory)

Step 6 — Bootstrap Confidence Intervals

obj <- aib_bootstrap(obj, B = 500, seed = 2024, conf = 0.95)
plot(obj, type = "trajectory")  # Now includes ribbon CIs

One-Shot: `aib_audit()`

Run the full pipeline in one call:

result <- aib_audit(
  lending_panel,
  id        = "applicant_id",
  time      = "year",
  group     = "race",
  decision  = "approved",
  ref_group = "White",
  bootstrap = TRUE,
  B         = 200,
  seed      = 42
)

summary(result)

Formal Definition of Bias Amplification

A decision system exhibits bias amplification for group \(g\) relative to reference group \(r\) over times \(1, \ldots, T\) if:

\(|B_{g,r}(t)| > |B_{g,r}(s)|\) for some \(t > s\) (disparity grows), and
Either \(A_{g,r}(t) = B_{g,r}(t \mid 1) - B_{g,r}(t \mid 0) \neq 0\) (prior decisions modulate current disparity), or
\(P_g(t) \neq P_r(t)\) (group transition matrices are unequal).

Proposition: If \(p_g^{11}(t) < p_r^{11}(t)\) and \(p_g^{01}(t) < p_r^{01}(t)\) for all \(t\), then under common initial conditions the favorable-decision probability for group \(g\) weakly decreases relative to group \(r\) over time, implying nonnegative cumulative disparity against group \(g\).

This distinguishes static persistent bias (constant gap) from dynamic compounding bias (self-reinforcing gap driven by the decision process itself).

Summary of Core Estimands

Estimand	Function	Formula
Bias trajectory	`aib_describe()`	\(B_{g,r}(t)\)
Standardized trajectory	`aib_describe()`	\(B^*_{g,r}(t)\)
Cumulative burden	`aib_describe()`	\(CB_{g,r}(T)\)
Recovery gap	`aib_transition()`	\(\Delta^{01}_{g,r}\)
Retention gap	`aib_transition()`	\(\Delta^{11}_{g,r}\)
Amplification index	`aib_amplify()`	\(A_{g,r}(t)\)
Bias persistence	`aib_persistence()`	\(PB_{g,r}(c)\)