---
title: "Comparability of translated exam forms"
output: markdown::html_format
vignette: >
  %\VignetteIndexEntry{Comparability of translated exam forms}
  %\VignetteEngine{knitr::knitr}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(collapse = TRUE, comment = "#>")
```

When an exam is offered in a second language, the question is whether a score
means the same thing in both. Standard DIF tools struggle when the translated
group is small, differs in ability, and many items shift in the same
direction. transDIF is built for that situation.

## Data

A source-language group of 2,000 and a translated-language group of 150 who
are 0.5 logits lower on average. Items with idioms, cultural referents,
measurement units or heavy vocabulary tend to shift in translation.

```{r}
library(transDIF)
sim <- td_simulate(n_ref = 2000, n_focal = 150, n_items = 40, seed = 5)
colSums(sim$features[-1])
```

## Calibrate each group, then link and detect DIF together

```{r}
cal <- td_calibrate(sim$responses, sim$group)
dif <- td_dif(cal)
dif
```

The linking shift (the ability difference) is the center of the densest
cluster of items, not the mean of all items, so it does not assume that DIF
cancels out. Compare with linking on
all items:

```{r}
c(true = -sim$truth$focal_mean, robust = dif$link[["c"]], all_items = dif$c_mean)
```

## Does DIF change pass rates?

```{r}
td_impact(dif, cut = 24, n_draws = 100, seed = 1)
```

## How much do the conclusions depend on the linking?

Linking assumes that DIF-free items form the largest cluster. When many
items show DIF, the data may not settle which items are DIF-free.
`td_sensitivity()` shows the results under each linking assumption, how
stable the mode is, and which items' DIF verdicts depend on the choice:

```{r}
sens <- td_sensitivity(dif, cut = 24, B = 100, n_draws = 50, seed = 1)
sens
```

The report has three parts:

- **Linkings.** The ability difference, the number of flagged items and the
  pass-rate impact under three assumptions: the densest cluster of items is
  DIF-free (mode), flagged items are removed until the rest agree
  (purification), and DIF cancels out over all items (mean).
- **Stability of the mode.** A parametric bootstrap redraws each item's
  between-language difference and re-estimates the mode. The share of
  bootstrap modes within the tolerance (0.15 logits by default) of the
  estimate shows whether the densest cluster is clear or whether a slightly
  different sample would have picked another cluster.
- **Linking-sensitive items.** Items whose DIF flag changes from one linking
  to another.

The verdict is "robust" only when the linkings agree within the tolerance
*and* the mode is stable (at least 80% of bootstrap modes within the
tolerance). Here the linkings agree closely, but with 150 candidates the mode
is not yet stable enough, so the verdict is "sensitive". Each part is
available for reporting:

```{r}
sens$linkings
sens$bootstrap
sens$items[sens$items$linking_sensitive, ]
```

### Two contrasting cases

With a larger translated-language group, the same 40-item design gives a
clear answer: the linkings agree and the mode barely moves under resampling.

```{r}
big <- td_simulate(n_ref = 1000, n_focal = 400, n_items = 40, seed = 1)
dif_big <- td_dif(td_calibrate(big$responses, big$group))
td_sensitivity(dif_big, cut = 24, B = 100, n_draws = 50, seed = 1)
```

Even a robust verdict can list a few linking-sensitive items: they sit near
the flagging boundary, so a small shift in the linking moves them across it.
They deserve review, but the overall conclusion does not depend on them.

A short test with pervasive DIF and a small group is the opposite case. No
cluster of DIF-free items dominates, the linkings disagree, and the mode
jumps between clusters under resampling:

```{r}
heavy <- c(idiom = 0.18, cultural = 0.18, units = 0.10, vocabulary = 0.2)
short <- td_simulate(n_ref = 1000, n_focal = 80, n_items = 16,
                     feature_prev = heavy, seed = 5)
dif_short <- suppressWarnings(td_dif(td_calibrate(short$responses, short$group)))
td_sensitivity(dif_short, cut = 10, B = 100, n_draws = 50, seed = 1)
```

Here the conclusion itself changes with the assumption: the estimated effect
of DIF on the pass rate is +3.6 points under the mode, +2.4 under
purification and essentially zero under mean linking, whose interval
includes zero.

### Reading the verdict

A "sensitive" verdict does not say which linking is right, and it does not
mean the mode is wrong. It says that a conclusion depends on an assumption
the data cannot check. In the package's simulations, "sensitive" verdicts
were concentrated in the hardest designs (short tests, small groups,
pervasive DIF), where linking errors are larger for every method; within a
given design, the verdict did not tell more accurate estimates from less
accurate ones. In practice:

1. When the verdict is robust, report the mode-based results.
2. When it is sensitive, report the results under all three linkings, say
   which conclusions change, and do not present a single pass-rate impact as
   settled.
3. In either case, send the linking-sensitive items to content and
   translation experts. Only their review can settle which items should
   anchor the scale.

The report can be included in the comparability report with
`td_report(dif, ..., sensitivity = sens)`.

## What should translators look at?

```{r}
td_features(dif, sim$features)
```

## A report to start from

```{r}
cat(td_report(dif, languages = c("English", "French")))
```

With very small translated-language groups (around 50 candidates), every
method has little power to detect DIF, so treat flags as candidates for
expert review rather than conclusions.