# Simulation based calibration for OncoBayes2

####

#### Tue Aug 4 19:33:43 2026

This report documents the results of a simulation based calibration (SBC) run for `OncoBayes2`. TODO

The calibration data presented here has been generated at and with the `OncoBayes` git version as:

    ## Created:  2026-08-04 09:41:25 UTC
    ## git hash: ab2f203cb91716f8b84c798cfd3f22aec3a75ad6
    ## MD5:      0ab6c545ab3bf1d299bbcf08facd3248

The MD5 hash of the calibration data file presented here must match the above listed MD5:

    ## /home/runner/work/OncoBayes2/OncoBayes2/inst/sbc/calibration.rds
    ##                               "0ab6c545ab3bf1d299bbcf08facd3248"

# Introduction

Simulation based calibration (SBC) is a necessary condition which must be met for any Bayesian analysis with proper priors. The details are presented in Talts, et. al (see https://arxiv.org/abs/1804.06788).

Self-consistency of any Bayesian analysis with a proper prior:

\[ p(\theta) = \iint \mbox{d}\tilde{y} \, \mbox{d}\tilde{\theta} \, p(\theta|\tilde{y}) \, p(\tilde{y}|\tilde{\theta}) \, p(\tilde{\theta}) \] \[ \Leftrightarrow p(\theta) = \iint \mbox{d}\tilde{y} \, \mbox{d}\tilde{\theta} \, p(\theta,\tilde{y},\tilde{\theta}) \]

SBC procedure:

Repeat \(s=1, ..., S\) times:

1. Sample from the prior \[\tilde{\theta} \sim p(\theta)\]

2. Sample fake data \[\tilde{y} \sim p(y|\tilde{\theta})\]

3. Obtain \(L\) posterior samples \[\{\theta_1, ..., \theta_L\} \sim p(\tilde{\theta}|\tilde{y})\]

4. Calculate the rank \(r_s\) of the prior draw \(\tilde{\theta}\) wrt to the posterior sample \(\{\theta_1, ..., \theta_L\} \sim p(\tilde{\theta}|\tilde{y})\) which falls into the range \([0,L]\) out of the possible \(L+1\) ranks. The rank is calculated as \[r_s = \sum_{l=1}^L \mathbb{I}[ \theta_l < \tilde{\theta}]\]

The \(S\) ranks then form a uniform \(0-1\) density and the count in each bin has a binomial distribution with probability of \[p(r \in \mbox{Any Bin}) =\frac{(L+1)}{S}.\]

## Model description TODO

The fake data simulation function returns … TODO. Please refer to the `sbc_tools.R` and `make_reference_rankhist.R` R programs for the implementation details.

The reference runs are created with \(L=1023\) posterior draws for each replication and a total of \(S=10^4\) replications are run per case. For the evaluation here the results are reduced to \(B=L'+1=64\) bins to ensure a sufficiently large sample size per bin.

# SBC results

## Sampler Diagnostics Overview

data_scenario | N | total_divergent | min_ess_bulk | min_ess_tail | max_Rhat | total_large_Rhat | min_lp_ess_bulk | min_lp_ess_tail
--- | --- | --- | --- | --- | --- | --- | --- | ---
combo2_EX | 10000 | 1 | 268.345 | 80.350 | 1.019 | 0 | 393.622 | 535.898
combo2_EXNEX | 10000 | 0 | 44.161 | 40.562 | 1.061 | 0 | 364.527 | 482.689
combo3_EXNEX | 10000 | 0 | 63.058 | 31.794 | 1.037 | 0 | 388.331 | 452.627
log2bayes_EX | 10000 | 0 | 950.458 | 747.960 | 1.023 | 0 | 308.346 | 538.114
log2bayes_EXNEX | 10000 | 0 | 299.184 | 342.943 | 1.015 | 0 | 380.328 | 426.596

Large Rhat is defined as exceeding \(1.1\).

## Sampler Adaptation & Performance Overview

ESS speed is in units of ESS per second.

## \(\chi^2\) Statistic: Single-agent logistic regression, stratified

param | statistic | df | p.value
--- | --- | --- | ---
beta_group[A,I(log(drug_A/1)),intercept] | 33.018 | 31 | 0.369
beta_group[A,I(log(drug_A/1)),log_slope] | 44.422 | 31 | 0.056
beta_group[B,I(log(drug_A/1)),intercept] | 31.251 | 31 | 0.454
beta_group[B,I(log(drug_A/1)),log_slope] | 25.626 | 31 | 0.739
beta_group[C,I(log(drug_A/1)),intercept] | 20.557 | 31 | 0.923
beta_group[C,I(log(drug_A/1)),log_slope] | 42.534 | 31 | 0.081
beta_group[D,I(log(drug_A/1)),intercept] | 27.635 | 31 | 0.640
beta_group[D,I(log(drug_A/1)),log_slope] | 32.384 | 31 | 0.398
beta_group[E,I(log(drug_A/1)),intercept] | 43.846 | 31 | 0.063
beta_group[E,I(log(drug_A/1)),log_slope] | 29.952 | 31 | 0.520
beta_group[F,I(log(drug_A/1)),intercept] | 25.235 | 31 | 0.757
beta_group[F,I(log(drug_A/1)),log_slope] | 29.120 | 31 | 0.563
mu_log_beta[I(log(drug_A/1)),intercept] | 29.222 | 31 | 0.558
mu_log_beta[I(log(drug_A/1)),log_slope] | 39.622 | 31 | 0.138
tau_log_beta[stratum_1,I(log(drug_A/1)),tau_intercept] | 40.909 | 31 | 0.110
tau_log_beta[stratum_1,I(log(drug_A/1)),tau_log_slope] | 29.530 | 31 | 0.542
tau_log_beta[stratum_2,I(log(drug_A/1)),tau_intercept] | 36.083 | 31 | 0.243
tau_log_beta[stratum_2,I(log(drug_A/1)),tau_log_slope] | 40.736 | 31 | 0.113

## \(\chi^2\) Statistic: Single-agent logistic regression, EXchangeable/NonEXchangeable

param | statistic | df | p.value
--- | --- | --- | ---
beta_group[A,I(log(drug_A/1)),intercept] | 35.078 | 31 | 0.281
beta_group[A,I(log(drug_A/1)),log_slope] | 26.803 | 31 | 0.682
beta_group[B,I(log(drug_A/1)),intercept] | 33.299 | 31 | 0.356
beta_group[B,I(log(drug_A/1)),log_slope] | 31.046 | 31 | 0.464
beta_group[C,I(log(drug_A/1)),intercept] | 29.485 | 31 | 0.544
beta_group[C,I(log(drug_A/1)),log_slope] | 20.429 | 31 | 0.926
mu_log_beta[I(log(drug_A/1)),intercept] | 27.219 | 31 | 0.661
mu_log_beta[I(log(drug_A/1)),log_slope] | 27.706 | 31 | 0.636
tau_log_beta[stratum_1,I(log(drug_A/1)),tau_intercept] | 39.296 | 31 | 0.146
tau_log_beta[stratum_1,I(log(drug_A/1)),tau_log_slope] | 43.622 | 31 | 0.066

## \(\chi^2\) Statistic: Double combination, fully exchangeable, stratified

param | statistic | df | p.value
--- | --- | --- | ---
beta_group[A,I(log(drug_A/1)),intercept] | 32.890 | 31 | 0.375
beta_group[A,I(log(drug_A/1)),log_slope] | 27.872 | 31 | 0.628
beta_group[A,I(log(drug_B/2)),intercept] | 30.810 | 31 | 0.476
beta_group[A,I(log(drug_B/2)),log_slope] | 34.816 | 31 | 0.291
beta_group[B,I(log(drug_A/1)),intercept] | 17.837 | 31 | 0.972
beta_group[B,I(log(drug_A/1)),log_slope] | 31.616 | 31 | 0.436
beta_group[B,I(log(drug_B/2)),intercept] | 44.909 | 31 | 0.051
beta_group[B,I(log(drug_B/2)),log_slope] | 29.555 | 31 | 0.540
beta_group[C,I(log(drug_A/1)),intercept] | 31.130 | 31 | 0.460
beta_group[C,I(log(drug_A/1)),log_slope] | 26.586 | 31 | 0.693
beta_group[C,I(log(drug_B/2)),intercept] | 44.755 | 31 | 0.052
beta_group[C,I(log(drug_B/2)),log_slope] | 31.776 | 31 | 0.428
beta_group[D,I(log(drug_A/1)),intercept] | 23.821 | 31 | 0.818
beta_group[D,I(log(drug_A/1)),log_slope] | 30.893 | 31 | 0.472
beta_group[D,I(log(drug_B/2)),intercept] | 38.554 | 31 | 0.165
beta_group[D,I(log(drug_B/2)),log_slope] | 29.837 | 31 | 0.526
eta_group[A,I(drug_A/1 * drug_B/2)] | 46.579 | 31 | 0.036
eta_group[B,I(drug_A/1 * drug_B/2)] | 24.544 | 31 | 0.788
eta_group[C,I(drug_A/1 * drug_B/2)] | 24.237 | 31 | 0.801
eta_group[D,I(drug_A/1 * drug_B/2)] | 26.893 | 31 | 0.678
mu_eta[m[1]] | 32.838 | 31 | 0.377
mu_log_beta[I(log(drug_A/1)),intercept] | 23.277 | 31 | 0.839
mu_log_beta[I(log(drug_A/1)),log_slope] | 16.723 | 31 | 0.983
mu_log_beta[I(log(drug_B/2)),intercept] | 44.723 | 31 | 0.053
mu_log_beta[I(log(drug_B/2)),log_slope] | 33.670 | 31 | 0.339
tau_eta[stratum_1,m[1]] | 37.293 | 31 | 0.202
tau_eta[stratum_2,m[1]] | 24.384 | 31 | 0.795
tau_log_beta[stratum_1,I(log(drug_A/1)),tau_intercept] | 32.480 | 31 | 0.394
tau_log_beta[stratum_1,I(log(drug_A/1)),tau_log_slope] | 25.754 | 31 | 0.733
tau_log_beta[stratum_1,I(log(drug_B/2)),tau_intercept] | 21.933 | 31 | 0.885
tau_log_beta[stratum_1,I(log(drug_B/2)),tau_log_slope] | 34.157 | 31 | 0.318
tau_log_beta[stratum_2,I(log(drug_A/1)),tau_intercept] | 40.947 | 31 | 0.109
tau_log_beta[stratum_2,I(log(drug_A/1)),tau_log_slope] | 46.253 | 31 | 0.038
tau_log_beta[stratum_2,I(log(drug_B/2)),tau_intercept] | 28.442 | 31 | 0.598
tau_log_beta[stratum_2,I(log(drug_B/2)),tau_log_slope] | 46.573 | 31 | 0.036

## \(\chi^2\) Statistic: Double combination, EXchangeable/NonEXchangeable

param | statistic | df | p.value
--- | --- | --- | ---
beta_group[A,I(log(drug_A/1)),intercept] | 27.443 | 31 | 0.650
beta_group[A,I(log(drug_A/1)),log_slope] | 20.768 | 31 | 0.918
beta_group[A,I(log(drug_B/2)),intercept] | 44.250 | 31 | 0.058
beta_group[A,I(log(drug_B/2)),log_slope] | 35.834 | 31 | 0.252
beta_group[B,I(log(drug_A/1)),intercept] | 28.288 | 31 | 0.606
beta_group[B,I(log(drug_A/1)),log_slope] | 24.058 | 31 | 0.808
beta_group[B,I(log(drug_B/2)),intercept] | 37.997 | 31 | 0.181
beta_group[B,I(log(drug_B/2)),log_slope] | 23.930 | 31 | 0.813
beta_group[C,I(log(drug_A/1)),intercept] | 38.010 | 31 | 0.180
beta_group[C,I(log(drug_A/1)),log_slope] | 32.358 | 31 | 0.400
beta_group[C,I(log(drug_B/2)),intercept] | 41.901 | 31 | 0.092
beta_group[C,I(log(drug_B/2)),log_slope] | 21.728 | 31 | 0.891
eta_group[A,I(drug_A/1 * drug_B/2)] | 34.234 | 31 | 0.315
eta_group[B,I(drug_A/1 * drug_B/2)] | 24.582 | 31 | 0.786
eta_group[C,I(drug_A/1 * drug_B/2)] | 42.061 | 31 | 0.089
mu_eta[m[1]] | 22.253 | 31 | 0.875
mu_log_beta[I(log(drug_A/1)),intercept] | 37.248 | 31 | 0.204
mu_log_beta[I(log(drug_A/1)),log_slope] | 37.158 | 31 | 0.206
mu_log_beta[I(log(drug_B/2)),intercept] | 33.466 | 31 | 0.348
mu_log_beta[I(log(drug_B/2)),log_slope] | 23.059 | 31 | 0.847
tau_eta[stratum_1,m[1]] | 24.973 | 31 | 0.769
tau_log_beta[stratum_1,I(log(drug_A/1)),tau_intercept] | 33.581 | 31 | 0.343
tau_log_beta[stratum_1,I(log(drug_A/1)),tau_log_slope] | 36.826 | 31 | 0.217
tau_log_beta[stratum_1,I(log(drug_B/2)),tau_intercept] | 22.912 | 31 | 0.852
tau_log_beta[stratum_1,I(log(drug_B/2)),tau_log_slope] | 22.214 | 31 | 0.876

## \(\chi^2\) Statistic: Triple combination, EXchangeable/NonEXchangeable

param | statistic | df | p.value
--- | --- | --- | ---
beta_group[A,I(log(drug_A/1)),intercept] | 40.410 | 31 | 0.120
beta_group[A,I(log(drug_A/1)),log_slope] | 33.523 | 31 | 0.346
beta_group[A,I(log(drug_B/2)),intercept] | 29.894 | 31 | 0.523
beta_group[A,I(log(drug_B/2)),log_slope] | 24.954 | 31 | 0.770
beta_group[A,I(log(drug_C/4)),intercept] | 48.832 | 31 | 0.022
beta_group[A,I(log(drug_C/4)),log_slope] | 31.334 | 31 | 0.449
beta_group[B,I(log(drug_A/1)),intercept] | 38.394 | 31 | 0.169
beta_group[B,I(log(drug_A/1)),log_slope] | 36.205 | 31 | 0.239
beta_group[B,I(log(drug_B/2)),intercept] | 38.438 | 31 | 0.168
beta_group[B,I(log(drug_B/2)),log_slope] | 24.096 | 31 | 0.807
beta_group[B,I(log(drug_C/4)),intercept] | 17.882 | 31 | 0.971
beta_group[B,I(log(drug_C/4)),log_slope] | 24.602 | 31 | 0.785
beta_group[C,I(log(drug_A/1)),intercept] | 33.050 | 31 | 0.367
beta_group[C,I(log(drug_A/1)),log_slope] | 25.088 | 31 | 0.764
beta_group[C,I(log(drug_B/2)),intercept] | 31.910 | 31 | 0.421
beta_group[C,I(log(drug_B/2)),log_slope] | 30.970 | 31 | 0.468
beta_group[C,I(log(drug_C/4)),intercept] | 49.069 | 31 | 0.021
beta_group[C,I(log(drug_C/4)),log_slope] | 26.778 | 31 | 0.683
eta_group[A,I(drug_A/1 * drug_B/2 * drug_C/4)] | 24.230 | 31 | 0.801
eta_group[A,I(drug_A/1 * drug_B/2)] | 23.584 | 31 | 0.827
eta_group[A,I(drug_A/1 * drug_C/4)] | 32.090 | 31 | 0.412
eta_group[A,I(drug_B/2 * drug_C/4)] | 23.123 | 31 | 0.845
eta_group[B,I(drug_A/1 * drug_B/2 * drug_C/4)] | 39.002 | 31 | 0.153
eta_group[B,I(drug_A/1 * drug_B/2)] | 38.208 | 31 | 0.175
eta_group[B,I(drug_A/1 * drug_C/4)] | 45.875 | 31 | 0.042
eta_group[B,I(drug_B/2 * drug_C/4)] | 50.413 | 31 | 0.015
eta_group[C,I(drug_A/1 * drug_B/2 * drug_C/4)] | 25.146 | 31 | 0.761
eta_group[C,I(drug_A/1 * drug_B/2)] | 27.046 | 31 | 0.670
eta_group[C,I(drug_A/1 * drug_C/4)] | 21.939 | 31 | 0.885
eta_group[C,I(drug_B/2 * drug_C/4)] | 25.030 | 31 | 0.766
mu_eta[m[1]] | 40.768 | 31 | 0.113
mu_eta[m[2]] | 32.262 | 31 | 0.404
mu_eta[m[3]] | 29.632 | 31 | 0.536
mu_eta[m[4]] | 35.251 | 31 | 0.274
mu_log_beta[I(log(drug_A/1)),intercept] | 26.086 | 31 | 0.717
mu_log_beta[I(log(drug_A/1)),log_slope] | 39.686 | 31 | 0.136
mu_log_beta[I(log(drug_B/2)),intercept] | 26.368 | 31 | 0.704
mu_log_beta[I(log(drug_B/2)),log_slope] | 25.587 | 31 | 0.741
mu_log_beta[I(log(drug_C/4)),intercept] | 23.795 | 31 | 0.819
mu_log_beta[I(log(drug_C/4)),log_slope] | 32.160 | 31 | 0.409
tau_eta[stratum_1,m[1]] | 34.886 | 31 | 0.288
tau_eta[stratum_1,m[2]] | 29.830 | 31 | 0.526
tau_eta[stratum_1,m[3]] | 38.227 | 31 | 0.174
tau_eta[stratum_1,m[4]] | 18.739 | 31 | 0.959
tau_log_beta[stratum_1,I(log(drug_A/1)),tau_intercept] | 22.592 | 31 | 0.864
tau_log_beta[stratum_1,I(log(drug_A/1)),tau_log_slope] | 37.050 | 31 | 0.210
tau_log_beta[stratum_1,I(log(drug_B/2)),tau_intercept] | 39.315 | 31 | 0.145
tau_log_beta[stratum_1,I(log(drug_B/2)),tau_log_slope] | 39.533 | 31 | 0.140
tau_log_beta[stratum_1,I(log(drug_C/4)),tau_intercept] | 26.605 | 31 | 0.692
tau_log_beta[stratum_1,I(log(drug_C/4)),tau_log_slope] | 29.888 | 31 | 0.523

## Session Info

    ## R version 4.6.1 (2026-06-24)
    ## Platform: x86_64-pc-linux-gnu
    ## Running under: Ubuntu 24.04.4 LTS
    ##
    ## Matrix products: default
    ## BLAS:   /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3
    ## LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.26.so;  LAPACK version 3.12.0
    ##
    ## locale:
    ##  [1] LC_CTYPE=C.UTF-8       LC_NUMERIC=C           LC_TIME=C.UTF-8
    ##  [4] LC_COLLATE=C.UTF-8     LC_MONETARY=C.UTF-8    LC_MESSAGES=C.UTF-8
    ##  [7] LC_PAPER=C.UTF-8       LC_NAME=C              LC_ADDRESS=C
    ## [10] LC_TELEPHONE=C         LC_MEASUREMENT=C.UTF-8 LC_IDENTIFICATION=C
    ##
    ## time zone: UTC
    ## tzcode source: system (glibc)
    ##
    ## attached base packages:
    ## [1] tools     stats     graphics  grDevices utils     datasets  methods
    ## [8] base
    ##
    ## other attached packages:
    ## [1] ggplot2_4.0.3    broom_1.0.13     tidyr_1.3.2      dplyr_1.2.1
    ## [5] assertthat_0.2.1 knitr_1.51       here_1.0.2
    ##
    ## loaded via a namespace (and not attached):
    ##  [1] gtable_0.3.6       jsonlite_2.0.0     compiler_4.6.1     tidyselect_1.2.1
    ##  [5] jquerylib_0.1.4    scales_1.4.0       yaml_2.3.12        fastmap_1.2.0
    ##  [9] R6_2.6.1           generics_0.1.4     backports_1.5.1    tibble_3.3.1
    ## [13] rprojroot_2.1.1    RColorBrewer_1.1-3 bslib_0.12.0       pillar_1.11.1
    ## [17] rlang_1.3.0        cachem_1.1.0       xfun_0.60          S7_0.2.2
    ## [21] sass_0.4.10        otel_0.2.0         cli_3.6.6          withr_3.0.3
    ## [25] magrittr_2.0.5     digest_0.6.39      grid_4.6.1         lifecycle_1.0.5
    ## [29] vctrs_0.7.3        evaluate_1.0.5     glue_1.8.1         farver_2.1.2
    ## [33] rmarkdown_2.31     purrr_1.2.2        pkgconfig_2.0.3    htmltools_0.5.9