Package {safestats}

Add a savi reference

Description

Keys are of the form firstAuthorYearFirstWord

Usage

addCite(..., breakLine = TRUE)

Arguments

Value

a character string

Examples

addCite(grunwald2024safe)

A ceiling function with a tolerance at the 13th digit

Description

A ceiling function with a tolerance at the 13th digit

Usage

ceil(x, digits = 13)

Arguments

Value

integer

Examples

ceiling(27/21*21)
ceil(27/21*21)

Checks consistency between the sided of the hypothesis and the minimal clinically relevant effect size or savi test defining parameter. Throws an error if the one-sided hypothesis is incongruent with the

Description

Checks consistency between the sided of the hypothesis and the minimal clinically relevant effect size or savi test defining parameter. Throws an error if the one-sided hypothesis is incongruent with the

Usage

checkAndReturnEsMinParameterSide(
  paramToCheck,
  alternative = c("twoSided", "greater", "less"),
  esMinName = c("noName", "meanDiffMin", "meanDiffTrue", "phiS", "deltaMin", "deltaS",
    "hrMin", "thetaS", "deltaTrue", "g", "kappaG"),
  paramDomain = NULL
)

Arguments

Value

paramToCheck after checking, perhaps with a change in sign

Check consistency between nPlan and the testType for one and two-sample z and t-tests

Description

Check consistency between nPlan and the testType for one and two-sample z and t-tests

Usage

checkAndReturnNPlan(
  nPlan,
  ratio = 1,
  testType = c("oneSample", "paired", "twoSample")
)

Arguments

Value

nPlan a vector of sample sizes of length 1 or 2

Checks consistency between the sided of the hypothesis and the minimal clinically relevant effect size or safe test defining parameter. Throws an error if the one-sided hypothesis is incongruent with the

Description

Checks consistency between the sided of the hypothesis and the minimal clinically relevant effect size or safe test defining parameter. Throws an error if the one-sided hypothesis is incongruent with the

Usage

checkAndReturnsEsMinParameterSide(
  paramToCheck,
  alternative = c("twoSided", "greater", "less"),
  esMinName = c("noName", "meanDiffMin", "phiS", "deltaMin", "deltaS", "hrMin", "thetaS",
    "deltaTrue"),
  paramDomain = NULL
)

Arguments

Value

paramToCheck after checking, perhaps with a change in sign

Helper function to check whether arguments are specified in a function at a higher level and already provided in the design object.

Description

Helper function to check whether arguments are specified in a function at a higher level and already provided in the design object.

Usage

checkDoubleArgumentsDesignObject(designObj, ...)

Arguments

Value

Returns nothing only used for its side-effects to produces warnings if needed.

Examples

designObj <- designSaviZ(0.4)

checkDoubleArgumentsDesignObject(designObj, "alpha"=NULL, alternative=NULL)
# Throws a warning
checkDoubleArgumentsDesignObject(designObj, "alpha"=0.4, alternative="d")

Computes 6 equivalent forms of the confluent hypergeometric function 1f1

Description

Repeated calls of genhypergeo, which provides further details.

Usage

compute1F1AllVersions(U, L, z, tol = 0, maxiter = 2000)

Arguments

Value

a vector

Examples

compute1F1AllVersions(U=-359, L=1/2, z=-0.1891234)

Helper function: Computes 1-power based on meanDiffTrue and nPlan

Description

Helper function: Computes 1-power based on meanDiffTrue and nPlan

Usage

computeBetaBatchSaviZ(
  meanDiffTrue,
  nPlan,
  alpha = 0.05,
  sigma = 1,
  kappa = sigma,
  alternative = c("twoSided", "greater", "less"),
  testType = c("oneSample", "paired", "twoSample"),
  parameter = NULL,
  eType = c("mom", "eGauss", "imom", "eCauchy", "grow")
)

Arguments

Value

numeric that represents the type II error

Functions

• computeBetaBatchSafeZ(): Deprecated version of computeBetaBatchSaviZ

References

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.

Helper function to compute uncertainty regarding nPlan estimates

Description

Helper function to compute uncertainty regarding nPlan estimates

Usage

computeBetaBootstrapper(samplingResult, parameter, nPlan, nBoot)

Arguments

Value

list with bootstrap objects

Examples

samplingResult <- sampleStoppingTimesSaviT(0.7, nSim=10, nMax=20)
result <- computeNPlanBootstrapper(samplingResult, 0.7, 0.2, 20, nBoot=1e2)

Helper function: Computes the power of the saviTTest based on deltaMin and nPlan

Description

Helper function: Computes the power of the saviTTest based on deltaMin and nPlan

Usage

computePowerSaviT(
  deltaTrue,
  nPlan,
  alpha = 0.05,
  alternative = c("twoSided", "greater", "less"),
  testType = c("oneSample", "paired", "twoSample"),
  parameter = NULL,
  deltaMin = deltaTrue,
  eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "lai"),
  wantSamplePaths = TRUE,
  nuMin = 2,
  wantSimData = TRUE,
  pb = TRUE,
  seed = NULL,
  nSim = 1000L,
  nBoot = nSim,
  relevanceTest = FALSE,
  relevanceSize = NULL,
  alphaRelevance = NULL,
  ...
)

Arguments

Value

a list which contains at least power and an adapted bootObject of class boot().

Functions

• computeBetaSafeT(): Deprecated version of computeBetaSaviT

References

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.

Examples

computePowerSaviT(deltaTrue=0.7, 27, nSim=10)

Helper function: Computes the power based on meanDiffTrue and nPlan

Description

Helper function: Computes the power based on meanDiffTrue and nPlan

Usage

computePowerSaviZ(
  meanDiffTrue,
  nPlan,
  alpha = 0.05,
  alternative = c("twoSided", "greater", "less"),
  sigma = 1,
  kappa = sigma,
  meanDiffMin = meanDiffTrue,
  testType = c("oneSample", "paired", "twoSample"),
  parameter = NULL,
  eType = c("mom", "eGauss", "imom", "eCauchy", "grow"),
  wantSamplePaths = TRUE,
  wantSimData = TRUE,
  relevanceTest = FALSE,
  relevanceSize = NULL,
  alphaRelevance = NULL,
  pb = TRUE,
  seed = NULL,
  nSim = 1000L,
  nBoot = nSim,
  ...
)

Arguments

Value

a list which contains at least power and an adapted bootObject of class boot.

Functions

• computeBetaSafeZ(): Deprecated version of computeBetaSaviZ

References

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.

Examples

computePowerSaviZ(meanDiffTrue=0.7, 20, nSim=10)

Computes the bootObj for sequential sampling procedures regarding nPlan, beta, the implied target

Description

Computes the bootObj for sequential sampling procedures regarding nPlan, beta, the implied target

Usage

computeBootObj(
  values,
  power = NULL,
  nPlan = NULL,
  nBoot = 1000L,
  alpha = NULL,
  beta = NULL,
  objType = c("nPlan", "nMean", "power", "beta", "betaFromEValues", "logImpliedTarget",
    "expectedStopTime")
)

Arguments

Value

bootObj

Examples

computeBootObj(1:100, objType="nPlan", beta=0.3)

Estimate an upper or lower bound for a savi confidence sequence on the logarithm of the odds ratio for two proportions.

Description

Estimate an upper or lower bound for a savi confidence sequence on the logarithm of the odds ratio for two proportions.

Usage

computeConfidenceBoundForLogOddsTwoProportions(
  ya,
  yb,
  saviDesign,
  bound = c("lower", "upper"),
  deltaStart,
  deltaStop,
  precision
)

Arguments

Value

numeric: the established lower- or upper bound on the logarithm of the odds ratio between the groups

Examples

balancedSaviDesign <- designSaviTwoProportions(na = 1,
                                               nb = 1,
                                               nBlocksPlan = 10,
                                               alpha = 0.05)
#hypothesize OR < 1 (i.e., log OR < 0)
ya <- c(1,1,1,1,1,1,1,1,0,1)
yb <- c(0,0,0,0,1,0,0,0,0,0)
#one-sided CI for OR-, establish upper bound on log odds ratio
computeConfidenceBoundForLogOddsTwoProportions(ya = ya,
                                           yb = yb,
                                           saviDesign = balancedSaviDesign,
                                           bound = "upper",
                                           deltaStart = -0.01,
                                           deltaStop = -4,
                                           precision = 20)

Estimate Lower and Upper Bounds on the Confidence Sequence (Interval) for the Difference Divergence Measure for Two Proportions

Description

Estimate Lower and Upper Bounds on the Confidence Sequence (Interval) for the Difference Divergence Measure for Two Proportions

Usage

computeConfidenceBoundsForDifferenceTwoProportions(
  ya,
  yb,
  precision,
  saviDesign
)

Arguments

Value

list with found lower and upper bound.

Examples

balancedSaviDesign <- designSaviTwoProportions(na = 1,
                                               nb = 1,
                                               nBlocksPlan = 10,
                                               alpha = 0.05)
ya <- c(1,1,1,1,1,1,1,1,0,1)
yb <- c(0,0,0,0,1,0,0,0,0,0)
computeConfidenceBoundsForDifferenceTwoProportions(ya = ya,
                                               yb = yb,
                                               precision = 20,
                                               saviDesign = balancedSaviDesign)

Helper function: Computes the savi confidence sequence for the mean in a T-test

Description

Helper function: Computes the savi confidence sequence for the mean in a T-test

Usage

computeConfidenceIntervalT(
  meanObs,
  sdObs,
  nEff,
  nu,
  parameter,
  eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "lai"),
  alternative = c("twoSided", "greater", "less"),
  ciValue = 0.95,
  maxRoot = 11,
  nuMin = 2
)

Arguments

Value

numeric vector that contains the upper and lower bound of the savi confidence sequence

Examples

computeConfidenceIntervalT(meanObs=0.3, sdObs=2, nEff=12, nu=11, parameter=0.4)

Helper function: Computes the savi confidence sequence for a Z-test

Description

Helper function: Computes the savi confidence sequence for a Z-test

Usage

computeConfidenceIntervalZ(
  nEff,
  meanObs,
  parameter,
  sigma = 1,
  ciValue = 0.95,
  alternative = "twoSided",
  a = NULL,
  g = NULL,
  eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "freq", "credibleInterval"),
  maxRoot = 20
)

Arguments

Value

numeric vector that contains the upper and lower bound of the savi confidence sequence

References

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.

Examples

computeConfidenceIntervalZ(nEff=15, meanObs=0.3, parameter=0.2)

Computes the credible interval of a two-sample t-test based on conjugate priors

Description

Computes the credible interval of a two-sample t-test based on conjugate priors

Usage

computeConjugateCredibleIntervalTwoSampleT(
  x1,
  sdObs1,
  n1,
  x2,
  sdObs2,
  n2,
  a1 = 3.98,
  g1 = 0.03,
  a2 = 4.02,
  g2 = 0.05,
  aGamma = 2,
  bGamma = 1/2,
  ciValue = 0.95
)

Arguments

Value

a vector of length two representing the credible interval

References

Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.

Examples

computeConjugateCredibleIntervalTwoSampleT(1, 1, 3, 1, 1, 3)

Helper function: Computes the type II error under optional stopping based on the minimal clinically relevant hazard ratio and the maximum number of nEvents.

Description

Helper function: Computes the type II error under optional stopping based on the minimal clinically relevant hazard ratio and the maximum number of nEvents.

Usage

computeLogrankBetaFrom(
  hrMin,
  nEvents,
  alpha = 0.05,
  alternative = c("twoSided", "greater", "less"),
  m0 = 50000L,
  m1 = 50000L,
  testType = c("oneSample", "paired", "twoSample"),
  ratio = 1,
  parameter = NULL,
  eType = c("mom", "eGauss", "imom", "eCauchy", "grow"),
  wantSamplePaths = TRUE,
  groupSizePerTimeFunction = returnOne,
  pb = TRUE,
  seed = NULL,
  nSim = 1000L,
  nBoot = nSim,
  relevanceTest = FALSE,
  ...
)

Arguments

Value

a list which contains at least beta and an adapted bootObject of class boot.

Author(s)

Muriel Felipe Perez-Ortiz and Alexander Ly

References

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

ter Schure, J., Pérez-Ortiz, M. F., Ly, A., & Grünwald, P. D. (2024). The Safe Logrank Test: Error control under continuous monitoring with unlimited horizon. The New England Journal of Statistics in Data Science, 2(2), 190-214, https://doi.org/10.51387/24-NEJSDS65.

Examples

computeLogrankBetaFrom(hrMin=0.7, 300, nSim=10)

Helper function: Computes the planned sample size based on the minimal clinical relevant hazard ratio, alpha and beta under optional stopping.

Description

Helper function: Computes the planned sample size based on the minimal clinical relevant hazard ratio, alpha and beta under optional stopping.

Usage

computeLogrankNEvents(
  hrMin,
  beta,
  m0 = 50000,
  m1 = 50000,
  alpha = 0.05,
  alternative = c("twoSided", "greater", "less"),
  nSim = 1000L,
  nBoot = nSim,
  groupSizePerTimeFunction = returnOne,
  nMax = 1000L,
  parameter = NULL,
  digits = getOption("digits"),
  pb = TRUE
)

Arguments

Value

a list which contains at least nEvents and an adapted bootObject of class boot.

Author(s)

Muriel Felipe Perez-Ortiz and Alexander Ly

References

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

ter Schure, J., Pérez-Ortiz, M. F., Ly, A., & Grünwald, P. D. (2024). The Safe Logrank Test: Error control under continuous monitoring with unlimited horizon. The New England Journal of Statistics in Data Science, 2(2), 190-214, https://doi.org/10.51387/24-NEJSDS65.

Examples

computeLogrankNEvents(0.7, 0.2, nSim=10)

Helper function to computes the logrank statistic for 'Surv' objects of type "right" and "counting" with the hypergeometric variance.

Description

This function was created to complement survdiff from the 'survival' package, which is restricted to 'Surv' objects of type "right". Most likely survdiff is much faster

Usage

computeLogrankZ(
  survObj,
  group,
  computeZ = TRUE,
  computeExactE = FALSE,
  theta0 = 1,
  thetaS = NULL,
  ...
)

Arguments

Value

Returns a list containing at least the following components:

nEvents

the number of events.

z

the observed logrank statistic.

oMinEVector

vector of observed minus expected.

varVector

vector of hypergeometric variances.

stopTimeVector

vector at which the events occurred.

References

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

ter Schure, J., Pérez-Ortiz, M. F., Ly, A., & Grünwald, P. D. (2024). The Safe Logrank Test: Error control under continuous monitoring with unlimited horizon. The New England Journal of Statistics in Data Science, 2(2), 190-214, https://doi.org/10.51387/24-NEJSDS65.

Schoenfeld, D. (1981). The asymptotic properties of nonparametric tests for comparing survival distributions. Biometrika, 68(1), 316-319, https://doi.org/10.2307/2335833.

Examples

data <- generateSurvData(nP = 5,
                         nT = 5,
                         lambdaP = 0.03943723,
                         lambdaT = 0.5*0.03943723,
                         endTime = 40,
                         seed = 2006)

survObj <- survival::Surv(data$time, data$status)

survObj <- survival::Surv(data$time, data$status)

result <- computeLogrankZ(survObj, data$group)
result$z
sqrt(survival::survdiff(survObj~data$group)$chisq)

Computes the meanDiffMin that is detectable with probability "power" for given nPlan

Description

Computes the meanDiffMin that is detectable with probability "power" for given nPlan

Usage

computeMinEsBatchSaviZ(
  nPlan,
  alpha = 0.05,
  power = 0.8,
  sigma = 1,
  kappa = sigma,
  alternative = c("twoSided", "greater", "less"),
  testType = c("oneSample", "paired", "twoSample"),
  parameter = NULL,
  beta = NULL,
  eType = c("mom", "eGauss", "imom", "eCauchy", "grow"),
  lowEsTrue = 0.01,
  highEsTrue = 0.002,
  ...
)

Arguments

Value

numeric > 0 that represents the minimal detectable mean difference

Functions

• computeMinEsBatchSafeZ(): Deprecated version of computeMinEsBatchSaviZ

References

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.

Examples

computeMinEsBatchSaviZ(27)

Computes the smallest detectable deltaMin with power probability, for the provided sample size

Description

Computes the smallest detectable deltaMin with power probability, for the provided sample size

Usage

computeMinEsBatchSaviT(
  nPlan,
  alpha = 0.05,
  power = 0.8,
  alternative = c("twoSided", "greater", "less"),
  testType = c("oneSample", "paired", "twoSample"),
  parameter = NULL,
  beta = NULL,
  eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "lai"),
  lowEsTrue = 0.01,
  highEsTrue = 3,
  ...
)

Arguments

Value

numeric > 0 that represents the minimal detectable effect size

References

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.

Examples

computeMinEsBatchSaviT(27)

Help function to compute the effective sample size based on a length 2 vector of samples

Description

Help function to compute the effective sample size based on a length 2 vector of samples

Usage

computeNEff(n, testType = c("oneSample", "paired", "twoSample"), silent = TRUE)

Arguments

Value

a numeric that represents the effective sample size.

Helper function: Computes nPlan based on meanDiffTrue, alpha and power.

Description

Helper function: Computes nPlan based on meanDiffTrue, alpha and power.

Usage

computeNPlanBatchSaviZ(
  meanDiffTrue,
  alpha = 0.05,
  power = 0.8,
  sigma = 1,
  kappa = sigma,
  alternative = c("twoSided", "greater", "less"),
  testType = c("oneSample", "paired", "twoSample"),
  eType = c("mom", "eGauss", "imom", "eCauchy", "grow"),
  parameter = NULL,
  meanDiffMin = NULL,
  beta = NULL,
  highN = 10000L,
  ratio = 1,
  ...
)

Arguments

Value

a list which contains at least nPlan and the savi test defining parameter.

Functions

• computeNPlanBatchSafeZ(): Deprecated version of computeNPlanBatchSaviZ

References

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.

Helper function: Computes the planned sample size for the savi T-test based deltaMin, alpha and power

Description

Helper function: Computes the planned sample size for the savi T-test based deltaMin, alpha and power

Usage

computeNPlanBatchSaviT(
  deltaTrue,
  alpha = 0.05,
  power = 0.8,
  alternative = c("twoSided", "greater", "less"),
  testType = c("oneSample", "paired", "twoSample"),
  eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "lai"),
  sigma = 1,
  sigma2 = 1,
  parameter = NULL,
  beta = NULL,
  ratio = 1,
  deltaMin = NULL,
  ...
)

Arguments

Value

a list which contains at least nPlan and the savi test defining parameter

References

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.

Helper function to compute uncertainty regarding nPlan estimates

Description

Helper function to compute uncertainty regarding nPlan estimates

Usage

computeNPlanBootstrapper(
  samplingResult,
  parameter,
  power,
  nPlanBatch,
  nBoot,
  beta = NULL
)

Arguments

Value

list with bootstrap objects

Examples

samplingResult <- sampleStoppingTimesSaviT(0.7, nSim=10, nMax=20)
result <- computeNPlanBootstrapper(samplingResult, 0.7, 0.2, 20, nBoot=1e2)

Helper function: Computes nPlan based on deltaMin and power

Description

Helper function: Computes nPlan based on deltaMin and power

Usage

computeNPlanSaviT(
  deltaTrue,
  power = 0.8,
  alpha = 0.05,
  alternative = c("twoSided", "less", "greater"),
  testType = c("oneSample", "paired", "twoSample"),
  deltaMin = NULL,
  beta = NULL,
  ratio = 1,
  parameter = NULL,
  nMax = 1e+08,
  eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "lai"),
  wantSamplePaths = TRUE,
  wantSimData = TRUE,
  nuMin = 2,
  pb = TRUE,
  seed = NULL,
  nSim = 1000L,
  nBoot = nSim,
  sigma = 1,
  sigma2 = 1,
  relevanceTest = FALSE,
  relevanceSize = NULL,
  alphaRelevance = NULL,
  ...
)

Arguments

Value

a list which contains at least nPlan and an adapted bootObject of class boot().

Functions

• computeNPlanSafeT(): Deprecated version of computeNPlanSaviT

References

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.

Examples

computeNPlanSaviT(0.7, 0.2, nSim=10)

Helper function: Computes nPlan based on meanDiffTrue, alpha and power

Description

Helper function: Computes nPlan based on meanDiffTrue, alpha and power

Usage

computeNPlanSaviZ(
  meanDiffTrue,
  power = 0.8,
  alpha = 0.05,
  alternative = c("twoSided", "less", "greater"),
  testType = c("oneSample", "paired", "twoSample"),
  sigma = 1,
  kappa = sigma,
  meanDiffMin = NULL,
  beta = NULL,
  ratio = 1,
  parameter = NULL,
  nMax = 1e+08,
  eType = c("mom", "eGauss", "imom", "eCauchy", "grow"),
  wantSamplePaths = TRUE,
  wantSimData = TRUE,
  pb = TRUE,
  seed = NULL,
  nSim = 1000L,
  nBoot = nSim,
  relevanceTest = FALSE,
  relevanceSize = NULL,
  alphaRelevance = NULL,
  highN = 10000L,
  ...
)

Arguments

Value

a list which contains at least nPlan and an adapted bootObject of class boot.

Functions

• computeNPlanSafeZ(): Deprecated version of computeNPlanSaviZ

References

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.

Examples

computeNPlanSaviZ(0.7, 0.2, nSim=10)

Helper function to compute uncertainty regarding nPlan estimates

Description

Helper function to compute uncertainty regarding nPlan estimates

Usage

computePowerBootstrapper(samplingResult, parameter, nPlan, nBoot)

Arguments

Value

list with bootstrap objects

Examples

samplingResult <- sampleStoppingTimesSaviT(0.7, nSim=10, nMax=20)
result <- computeNPlanBootstrapper(samplingResult, 0.7, 0.2, 20, nBoot=1e2)

Computes the sufficient statistics needed to compute 'logrankSingleZ'

Description

Computes the sufficient statistics needed to compute 'logrankSingleZ'

Usage

computeStatsForLogrank(
  survDataFrame,
  y0Index,
  y1Index,
  timeNow,
  timeBefore,
  survType = "right",
  ...
)

Arguments

Value

Returns a list containing at least the following components:

obs0

number of observations in the control group.

obs1

number of observations in the treatment group.

y0

total number of participants in the control group.

y1

total number of participants in the treatment group.

#'

Examples

data <- generateSurvData(nP = 5,
                         nT = 5,
                         lambdaP = 0.03943723,
                         lambdaT = 0.5*0.03943723,
                         endTime = 40,
                         seed = 2006)

survObj <- survival::Surv(data$time, data$status)

survDataFrame <- as.data.frame(as.matrix(survObj))
y0Index <- which(data$group=="P")
y1Index <- which(data$group=="T")

timeNow <- 4
timeBefore <- 0

computeStatsForLogrank(survDataFrame, y0Index, y1Index, timeNow, timeBefore)

timeNow <- 13
timeBefore <- 4

computeStatsForLogrank(survDataFrame, y0Index, y1Index, timeNow, timeBefore)

Helper function to compute the relevant summary statistics in z and t-tests

Description

Helper function to compute the relevant summary statistics in z and t-tests

Usage

computeZTSumStats(
  x,
  y = NULL,
  sequential = NULL,
  paired = FALSE,
  varEqual = TRUE,
  testType = NULL
)

Arguments

Value

a list of summary statistics

Examples

n1 <- 23
n2 <- 39

x <- rnorm(n1, sd=7)
y <- rnorm(n2, sd=3)

computeZTSumStats(x=x, y=y)

A "subjective" Bayes factor for the two-sample T-test

Description

Based on conjugate priors with a total of 8 hyperparameters.

Usage

conjugateBfTStat(
  x1,
  sdObs1,
  n1,
  x2,
  sdObs2,
  n2,
  a1 = 3.98,
  g1 = 0.03,
  a2 = 4.02,
  g2 = 0.05,
  a0 = 4,
  g0 = 2,
  aGamma = 2,
  bGamma = 1/2,
  log = FALSE
)

Arguments

Value

numeric > 0 representing the subjective Bayes factor outcome in favour of the alternative over the null

References

Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.

Examples

conjugateBfTStat(5.2, 2, 3, 3.4, 2, 12)

A "subjective" Bayes factor for the two-sample T-test

Description

Based on conjugate priors with a total of 8 hyperparameters.

Usage

conjugateBfTStatOld(
  x1,
  sdObs1,
  n1,
  x2,
  sdObs2,
  n2,
  a1 = 3.98,
  g1 = 0.5,
  a2 = 4.02,
  g2 = 0.1,
  a0 = 4,
  g0 = 2,
  aGamma = 2,
  bGamma = 1/2,
  log = FALSE
)

Arguments

Value

numeric > 0 representing the subjective Bayes factor outcome in favour of the alternative over the null

References

Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.

Examples

conjugateBfTStat(5.2, 2, 3, 3.4, 2, 12)

Construct a savi design object to be set in the design function

Description

Construct a savi design object to be set in the design function

Usage

constructSaviDesignObj(testName)

Arguments

Value

a savi design object

Functions

• constructSafeDesignObj(): Deprecated version of constructSaviDesignObj

Examples

obj <- constructSaviDesignObj("Z-Test")

Construct a savi test object to be set in the savi testing function

Description

Construct a savi test object to be set in the savi testing function

Usage

constructSaviTestObj(testName)

Arguments

Value

a savi test object

Functions

• constructSafeTestObj(): Deprecated version of constructSaviTestObj

Examples

obj <- constructSaviTestObj("Z-Test")

Construct a list to be set in the sampleStoppingTimes... function

Description

Construct a list to be set in the sampleStoppingTimes... function

Usage

constructSampleStoppingTimesList(
  nSim = 1000L,
  nMax = 1000L,
  wantEValuesAtNMax = FALSE,
  wantSamplePaths = TRUE
)

Arguments

Value

a list with names

Examples

obj <- constructSampleStoppingTimesList()

Computes a Sequence of (Effective) Sample Sizes of Z- and T-Tests

Description

Helper function that outputs a sequence of sample sizes, effective sample sizes, and the degrees of freedom depending on the type of T-test. Also used for Z-tests.

Usage

defineTTestN(
  lowN = 3,
  highN = 100,
  ratio = 1,
  testType = c("oneSample", "paired", "twoSample")
)

Arguments

Value

Returns the sample sizes and degrees of freedom.

Design a Frequentist T-Test

Description

Computes the number of samples necessary to reach a tolerable type I and desired power for the frequentist T-test.

Usage

designFreqT(
  deltaMin,
  alpha = 0.05,
  power = 0.8,
  alternative = c("twoSided", "greater", "less"),
  h0 = 0,
  testType = c("oneSample", "paired", "twoSample"),
  ...
)

Arguments

Value

Returns an object of class 'freqTDesign'. An object of class 'freqTDesign' is a list containing at least the following components:

nPlan

the planned sample size(s).

esMin

the minimal clinically relevant standardised effect size provided by the user.

alpha

the tolerable type I error provided by the user.

power

the desired power provided by the user.

lowN

the smallest n of the search space for n provided by the user.

highN

the largest n of the search space for n provided by the user.

testType

any of "oneSample", "paired", "twoSample" provided by the user.

alternative

any of "twoSided", "greater", "less" provided by the user.

Examples

designFreqT(0.5)

Design a Frequentist Z-Test

Description

Computes the number of samples necessary to reach a tolerable type I and desired power for the frequentist Z-test.

Usage

designFreqZ(
  meanDiffMin,
  alternative = c("twoSided", "greater", "less"),
  alpha = 0.05,
  power = 0.8,
  testType = c("oneSample", "paired", "twoSample"),
  ratio = 1,
  sigma = 1,
  h0 = 0,
  kappa = sigma,
  lowN = 3L,
  highN = 100L,
  ...
)

Arguments

Value

returns a 'freqZDesign' object.

Examples

freqDesign <- designFreqZ(meanDiffMin = 0.5, highN = 100)
freqDesign$nPlan
freqDesign2 <- designFreqZ(meanDiffMin = 0.2, lowN = 32, highN = 200)
freqDesign2$nPlan

Helper function that extract the results for the design scenario 1a: Target nPlan

Description

Helper function that extract the results for the design scenario 1a: Target nPlan

Usage

designSavi1aHelper(
  samplingResult,
  esMin,
  power,
  ratio,
  beta = NULL,
  testType = c("oneSample", "paired", "twoSample")
)

Arguments

Value

a list of partial results for the design scenario 1a

Functions

• designSafe1aHelper(): Deprecated version of designSavi1aHelper

Examples

samplingResult <- computeNPlanSaviZ(0.7, nSim=10, nMax=20)
result <- designSavi1aHelper(samplingResult, 0.7, 0.2, 1)

Helper function that extract the results for the design scenario 1a: Target nPlan

Description

Helper function that extract the results for the design scenario 1a: Target nPlan

Usage

designSavi2Helper(
  samplingResult,
  esMin,
  nPlan,
  ratio,
  testType = c("oneSample", "paired", "twoSample")
)

Arguments

Value

a list of partial results for the design scenario 1a

Functions

• designSafe2Helper(): Deprecated version of designSavi2Helper

Examples

samplingResult <- computeNPlanSaviZ(0.7, nSim=10, nMax=20)
result <- designSavi1aHelper(samplingResult, 0.7, 0.2, 1)

Designs a Safe Anytime-Valid Logrank Test Experiment

Description

A designed experiment requires (1) an anticipated number of events nEvents, or even better nPlan, the number of participants to be recruited in the study, and (2) the parameter of the savi test, i.e., thetaS. Provided with a clinically relevant minimal hazard ratio hrMin, this function outputs thetaS = hrMin as the savi test defining parameter in accordance to the GROW criterion. If a tolerable type II error beta is provided then nEvents can be sampled. The sampled nEvents is then the smallest nEvents for which hrMin is found with power of at least 1 - beta under optional stopping. If exact equal FALSE, then the computations exploit the local asymptotic normal approximation to sampling distribution of the logrank test derived by Schoenfeld (1981).

Usage

designSaviLogrank(
  hrMin = NULL,
  power = NULL,
  nEvents = NULL,
  alpha = 0.05,
  h0 = 1,
  alternative = c("twoSided", "greater", "less"),
  m0 = 50000L,
  m1 = 50000L,
  testType = c("exactLogrank", "gaussianLogrank"),
  ratio = 1,
  exact = TRUE,
  parameter = NULL,
  eType = c("mom", "eGauss", "imom", "eCauchy", "grow"),
  wantSamplePaths = TRUE,
  groupSizePerTimeFunction = returnOne,
  pb = TRUE,
  seed = NULL,
  nSim = 1000L,
  nBoot = nSim,
  beta = NULL,
  relevanceTest = FALSE,
  relevanceSize = NULL,
  wantEValuesAtNMax = NULL,
  wantSimData = FALSE,
  ...
)

Arguments

Value

Returns a saviDesign object that includes:

nEvents

the anticipated number of events, either (1) specified by the user, or (2) computed based on beta and thetaMin.

parameter

the parameter that defines the savi test. Here log(thetaS).

esMin

the minimally clinically relevant hazard ratio specified by the user.

alpha

the tolerable type I error provided by the user.

beta

the tolerable type II error provided by the user.

alternative

any of "twoSided", "greater", "less" provided by the user.

testType

"logrank".

ratio

default is 1. It defines the ratio between the planned randomisation of condition 2 over condition 1.

pilot

FALSE to indicate that the design is not a pilot study.

call

the expression with which this function is called.

References

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

ter Schure, J., Pérez-Ortiz, M. F., Ly, A., & Grünwald, P. D. (2024). The Safe Logrank Test: Error control under continuous monitoring with unlimited horizon. The New England Journal of Statistics in Data Science, 2(2), 190-214, https://doi.org/10.51387/24-NEJSDS65.

Schoenfeld, D. (1981). The asymptotic properties of nonparametric tests for comparing survival distributions. Biometrika, 68(1), 316-319, https://doi.org/10.2307/2335833.

Examples

designSaviLogrank(hrMin=0.7)
designSaviLogrank(hrMin=0.7, exact=FALSE)
designSaviLogrank(hrMin=0.7, beta=0.3, nSim=10)
designSaviLogrank(hrMin=0.7, nEvents=190, nSim=10)

Helper function to design a savi logrank test (output beta)

Description

Finds the parameter and beta when provided with only alpha, esMin, and nPlan

Usage

designSaviLogrank2WantBeta(
  hrMin,
  nEvents,
  alpha = 0.05,
  alternative = c("twoSided", "greater", "less"),
  m0 = 50000L,
  m1 = 50000L,
  testType = c("oneSample", "paired", "twoSample"),
  ratio = 1,
  parameter = NULL,
  eType = c("mom", "eGauss", "imom", "eCauchy", "grow"),
  wantSamplePaths = TRUE,
  groupSizePerTimeFunction = returnOne,
  pb = TRUE,
  seed = NULL,
  nSim = 1000L,
  nBoot = nSim,
  ...
)

Arguments

Value

A list with the parameter and beta amongst other items

References

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

ter Schure, J., Pérez-Ortiz, M. F., Ly, A., & Grünwald, P. D. (2024). The Safe Logrank Test: Error control under continuous monitoring with unlimited horizon. The New England Journal of Statistics in Data Science, 2(2), 190-214, https://doi.org/10.51387/24-NEJSDS65.

Schoenfeld, D. (1981). The asymptotic properties of nonparametric tests for comparing survival distributions. Biometrika, 68(1), 316-319, https://doi.org/10.2307/2335833.

Examples

designSaviLogrank2WantBeta(hrMin=0.9, nEvents=7, nSim=10)

Design a Safe Anytime-Valid Experiment to Test Means with a Z Test

Description

A designed experiment requires (1) a sample size nPlan to plan for, and (2) a savi test defining parameter. The design involves alpha and the three quantities: (1) nPlan, (2) power, and (3) a minimal clinically relevant standarised mean difference deltaMin.

Scenario 1.a

Goal: "nPlan" and optimal E-variable. Given: deltaMin and power.

Scenario 1.b

Goal: an optimal E-variable. Given: deltaMin only.

Scenario 2

Goal: "power" and optimal E-variable. Given: deltaMin and nPlan.

Scenario 3.a

Goal: "deltaMin" and optimal E-variable. Given: power and nPlan.

Scenario 3.b

Goal: an optimal E-variable. Given: nPlan only.

Usage

designSaviT(
  deltaMin = NULL,
  power = NULL,
  nPlan = NULL,
  alpha = 0.05,
  h0 = 0,
  alternative = c("twoSided", "greater", "less"),
  testType = c("oneSample", "paired", "twoSample"),
  ratio = 1,
  parameter = NULL,
  beta = NULL,
  eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "lai"),
  wantSamplePaths = TRUE,
  wantSimData = TRUE,
  deltaTrue = NULL,
  sigma = 1,
  sigma2 = sigma,
  lowEsTrue = 0.01,
  highEsTrue = 3,
  varEqual = TRUE,
  pb = TRUE,
  seed = NULL,
  nSim = 1000L,
  nBoot = nSim,
  relevanceTest = FALSE,
  relevanceSize = NULL,
  alphaRelevance = NULL,
  betaDefault = 0.2,
  highN = 10000L,
  wantSampling = TRUE,
  nuMin = 2,
  ...
)

Arguments

Details

Every scenario returns an E-variable adapted to the input. Scenario 1.a, for instance, outputs the parameter of the provided eType (default mom) savi test, see matchEParameterWith for details, and nPlan. The nPlan is based on samples paths drawn under deltaTrue (if not specified, then deltaTrue=deltaMin by default). The resulting nPlan corresponds to the power (say 80%) quantile of the first-passage time distribution associated with E crossing threshold 1/alpha.

Value

Returns an object of class 'saviDesign'. An object of class 'saviDesign' is a list containing at least the following components:

parameter

the savi test defining parameter, see matchEParameterWith.

alpha

the tolerable type I error provided by the user.

pilot

logical, specifying whether it's a pilot design, which occurs when saviTTest is called without a designObj.

testName

"T-Test".

call

the expression with which this function is called.

Functions

• designSafeT(): Deprecated version of designSaviT

• designSafeT2WantBeta(): Deprecated version of designSaviT2WantBeta

• designSafeT3WantEsMin(): Deprecated version of designSafeT3WantEsMin

• designSafeT3bWantParameter(): Deprecated version of designSaviT3bWantParameter

• computeNPlanBatchSafeT(): Deprecated version of computeNPlanBatchSaviT

• computeMinEsBatchSafeT(): Deprecated version of computeMinEsBatchSaviT

References

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.

Examples

# Scenario 1.b: Goal: an E-variable
designObj <- designSaviT(deltaMin=0.8)

# Scenario 1.a: Goal: "nPlan" and optimal E-variable.
designObj <- designSaviT(deltaMin=0.8, power=0.6, alpha=0.2,
                         alternative="greater", nSim=100)

plot(designObj)

# Scenario 1a. with relevance testing, also stopping for practically null
designObj <- designSaviT(deltaMin=0.8, power=0.6, alpha=0.2,
                         alternative="greater", nSim=100,
                         relevanceTest=TRUE)

plot(designObj)

# Scenario 2: Goal: "power" and optimal E-variable
designObj <- designSaviT(deltaMin=0.8, nPlan=16, nSim=100)

# Scenario 3.a: Goal: "meanDiffMin" and optimal E-variable
designObj <- designSaviT(power=0.7, nPlan=16)

# Scenario 3.b: Goal: an optimal E-variable. Given: nPlan only.
designObj <- designSaviT(nPlan=16)

Helper function to designing a savi T-test (output nPlan)

Description

Finds the parameter and power when provided with only alpha, esMin, and nPlan

Usage

designSaviT1aWantNPlan(
  deltaMin,
  power,
  alpha = 0.05,
  alternative = c("twoSided", "greater", "less"),
  testType = c("oneSample", "paired", "twoSample"),
  ratio = 1,
  parameter = NULL,
  deltaTrue = NULL,
  beta = NULL,
  eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "lai"),
  wantSamplePaths = TRUE,
  wantSimData = TRUE,
  pb = TRUE,
  seed = NULL,
  nSim = 1000L,
  nBoot = nSim,
  relevanceTest = FALSE,
  relevanceSize = NULL,
  sigma = 1,
  sigma2 = 1,
  alphaRelevance = NULL,
  nuMin = 2,
  ...
)

Arguments

Value

A list with the parameter and the targeted nPlan amongst other items

Functions

• designSafeT1aWantNPlan(): Deprecated version of designSaviT1aWantNPlan

References

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.

Examples

designSaviT1aWantNPlan(deltaMin=0.9, power=0.7, nSim=10)

Designs a Savi Experiment to Test Two Proportions in Stream Data

Description

The design requires the number of observations one expects to collect in each group in each data block. I.e., when one expects balanced data, one could choose na = nb = 1 and would be allowed to analyse the data stream each time a new observation in both groups has come in. The best results in terms of power are achieved when the data blocks are chosen as small as possible, as this allows for analysing and updating the savi test as often as possible, to fit the data best. Further, the design requires two out of the following three parameters to be known:

• the power one aims to achieve (1 - beta),

• the minimal relevant difference between the groups (delta)

• the number of blocks planned (nBlocksPlan),

where the unknown out of the three will be estimated. In the case of an exploratory "pilot" analysis, one can also only provide the number of blocks planned.

Usage

designSaviTwoProportions(
  na,
  nb,
  nBlocksPlan = NULL,
  beta = NULL,
  delta = NULL,
  alternativeRestriction = c("none", "difference", "logOddsRatio"),
  alpha = 0.05,
  pilot = "FALSE",
  hyperParameterValues = NULL,
  previousSaviTestResult = NULL,
  M = 1000,
  simThetaAMin = NULL,
  simThetaAMax = NULL
)

Arguments

Value

Returns a 'saviDesign' object that includes:

nPlan

the sample size(s) to plan for. Computed based on beta and meanDiffMin, or provided by the user if known.

parameter

the savi test defining parameter: here the hyperparameters.

esMin

the minimally clinically relevant effect size provided by the user.

alpha

the tolerable type I error provided by the user.

beta

the tolerable type II error specified by the user.

alternative

any of "twoSided", "greater", "less" based on the alternativeRestriction provided by the user.

testType

here 2x2

pilot

logical, specifying whether it's a pilot design.

call

the expression with which this function is called.

Examples

#plan for an experiment to detect minimal difference of 0.6 with a balanced design
set.seed(3152021)
designSaviTwoProportions(na = 1,
                         nb = 1,
                         alpha = 0.1,
                         beta = 0.20,
                         delta = 0.6,
                         alternativeRestriction = "none",
                         M = 75)

#savi analysis of a pilot: number of samples already known
designSaviTwoProportions(na = 1,
                          nb = 1,
                          nBlocksPlan = 20,
                          pilot = TRUE)

#specify own hyperparameters
hyperParameterValues <- list(betaA1 = 10, betaA2 = 1, betaB1 = 1, betaB2 = 10)
designSaviTwoProportions(na = 1,
                         nb = 1,
                         alpha = 0.1,
                         beta = 0.20,
                         delta = 0.6,
                         hyperParameterValues = hyperParameterValues,
                         alternativeRestriction = "none",
                         M = 75)

#restrict range of proportions for estimating nPlan in the control group
designSaviTwoProportions(na = 1,
                         nb = 1,
                         beta = 0.20,
                         delta = 0.3,
                         alternativeRestriction = "none",
                         M = 75,
                         simThetaAMin = 0.1, simThetaAMax = 0.2)

Design a Safe Anytime-Valid Experiment to Test Means with a Z Test

Description

A designed experiment requires (1) a sample size nPlan to plan for, and (2) a savi test defining parameter. The design involves alpha and the three quantities: (1) nPlan, (2) power, and (3) a minimal clinically relevant mean difference meanDiffMin.

Scenario 1.a

Goal: "nPlan" and optimal E-variable. Given: meanDiffMin and power.

Scenario 1.b

Goal: an optimal E-variable. Given: meanDiffMin only.

Scenario 2

Goal: "power" and optimal E-variable. Given: meanDiffMin and nPlan.

Scenario 3.a

Goal: "meanDiffMin" and optimal E-variable. Given: power and nPlan.

Scenario 3.b

Goal: an optimal E-variable. Given: nPlan only.

Usage

designSaviZ(
  meanDiffMin = NULL,
  power = NULL,
  nPlan = NULL,
  alpha = 0.05,
  h0 = 0,
  alternative = c("twoSided", "greater", "less"),
  sigma = 1,
  kappa = sigma,
  meanDiffTrue = NULL,
  beta = NULL,
  testType = c("oneSample", "paired", "twoSample"),
  ratio = 1,
  parameter = NULL,
  eType = c("mom", "eGauss", "imom", "eCauchy", "grow"),
  wantSamplePaths = TRUE,
  wantSimData = TRUE,
  lowEsTrue = 0.01,
  highEsTrue = 3,
  pb = TRUE,
  seed = NULL,
  nSim = 1000L,
  nBoot = nSim,
  relevanceTest = FALSE,
  relevanceSize = NULL,
  alphaRelevance = NULL,
  betaDefault = 0.2,
  highN = 10000L,
  wantSampling = TRUE,
  ...
)

Arguments

Details

Every scenario returns an E-variable adapted to the input. Scenario 1.a, for instance, outputs the parameter of the provided eType (default mom) savi test, see matchEParameterWith for details, and nPlan. The nPlan is based on samples paths drawn under meanDiffTrue (if not specified, then meanDiffTrue=meanDiffMin by default). The resulting nPlan corresponds to the power (say 80%) quantile of the first-passage time distribution associated with E crossing threshold 1/alpha.

Value

Returns a saviDesign object that includes:

parameter

the savi test defining parameter, see matchEParameterWith.

alpha

the tolerable type I error provided by the user.

pilot

logical, specifying whether it's a pilot design, which occurs when saviZTest is called without a designObj.

testName

"Z-Test".

call

the expression with which this function is called.

Functions

• designSafeZ(): Deprecated version of designSaviZ

References

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.

Examples

# Scenario 1.b: Goal: an E-variable
designObj <- designSaviZ(meanDiffMin=0.8)

# Scenario 1.a: Goal: "nPlan" and optimal E-variable.
designObj <- designSaviZ(meanDiffMin=0.8, power=0.6, alpha=0.2,
                         alternative="greater", nSim=100)

plot(designObj)

# Scenario 1a. with relevance testing, also stopping for practically null
designObj <- designSaviZ(meanDiffMin=0.8, power=0.6, alpha=0.2,
                         alternative="greater", nSim=100,
                         relevanceTest=TRUE)

plot(designObj)

# Scenario 2: Goal: "power" and optimal E-variable
designObj <- designSaviZ(meanDiffMin=0.8, nPlan=16, nSim=100)

# Scenario 3.a: Goal: "meanDiffMin" and optimal E-variable
designObj <- designSaviZ(power=0.7, nPlan=16)

# Scenario 3.b: Goal: an optimal E-variable. Given: nPlan only.
designObj <- designSaviZ(nPlan=16)

Helper function to designing a savi Z-test (output nPlan)

Description

Finds the parameter and power when provided with only alpha, esMin, and nPlan

Usage

designSaviZ1aWantNPlan(
  meanDiffMin,
  power,
  alpha = 0.05,
  alternative = c("twoSided", "greater", "less"),
  sigma = 1,
  kappa = sigma,
  beta = NULL,
  testType = c("oneSample", "paired", "twoSample"),
  ratio = 1,
  parameter = NULL,
  meanDiffTrue = NULL,
  eType = c("mom", "eGauss", "imom", "eCauchy", "grow"),
  wantSamplePaths = TRUE,
  wantSimData = TRUE,
  pb = TRUE,
  seed = NULL,
  nSim = 1000L,
  nBoot = nSim,
  relevanceTest = FALSE,
  relevanceSize = NULL,
  alphaRelevance = NULL,
  highN = 10000L,
  ...
)

Arguments

Value

A list with the parameter and the targeted nPlan amongst other items

Functions

• designSafeZ1aWantNPlan(): Deprecated version of designSaviZ1aWantNPlan

References

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.

Examples

designSaviZ1aWantNPlan(meanDiffMin=0.9, power=0.7, nSim=10)

Helper function to designing a Z-test (output power)

Description

Finds the parameter and power when provided with only alpha, esMin, and nPlan

Usage

designSaviZ2WantPower(
  meanDiffTrue,
  nPlan,
  alpha = 0.05,
  alternative = c("twoSided", "greater", "less"),
  sigma = 1,
  kappa = sigma,
  meanDiffMin = meanDiffTrue,
  testType = c("oneSample", "paired", "twoSample"),
  ratio = 1,
  parameter = NULL,
  eType = c("mom", "eGauss", "imom", "eCauchy", "grow"),
  wantSamplePaths = TRUE,
  wantSimData = TRUE,
  relevanceTest = FALSE,
  relevanceSize = NULL,
  alphaRelevance = NULL,
  pb = TRUE,
  seed = NULL,
  nSim = 1000L,
  nBoot = nSim,
  ...
)

Arguments

Value

A list with the parameter and power amongst other items

Functions

• designSafeZ2WantBeta(): Deprecated version of designSaviZ2WantBeta

References

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.

Examples

designSaviZ2WantPower(meanDiffTrue=0.9, nPlan=7, nSim=10)

Helper function to designing a Z-test (output esMin)

Description

Finds the parameter and esMin when provided with only alpha, power, and nPlan

Usage

designSaviZ3WantEsMin(
  power,
  nPlan,
  alpha = 0.05,
  alternative = c("twoSided", "greater", "less"),
  sigma = 1,
  kappa = sigma,
  testType = c("oneSample", "paired", "twoSample"),
  parameter = NULL,
  beta = NULL,
  eType = c("mom", "eGauss", "imom", "eCauchy", "grow"),
  lowEsTrue = 0.01,
  highEsTrue = 3,
  ...
)

Arguments

Value

A list with the parameter and the targeted esMin amongst other items

Functions

• designSafeZ3WantEsMin(): Deprecated version of designSaviZ3WantEsMin

References

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.

Examples

designSaviZ3WantEsMin(power=0.7, nPlan=10)

Helper function to designing a Z-test (output meanDiffMin based on the shortest interval at nPlan)

Description

Finds the parameter and meanDiffMin when provided with only alpha, nPlan

Usage

designSaviZ3bWantParameter(
  nPlan,
  alpha = 0.05,
  alternative = c("twoSided", "greater", "less"),
  sigma = 1,
  kappa = sigma,
  testType = c("oneSample", "paired", "twoSample"),
  parameter = NULL,
  eType = c("mom", "eGauss", "imom", "eCauchy", "grow"),
  ...
)

Arguments

Value

A list with the parameter and the parameter amongst other items

Functions

• designSafeZ3bWantParameter(): Deprecated version of designSaviZ3bWantParameter

References

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.

Examples

designSaviZ3bWantParameter(nPlan=13)

Helper function to designing a savi T-test (output power)

Description

Finds the parameter and power when provided with only alpha, esMin, and nPlan

Usage

designSaviT2WantPower(
  deltaTrue,
  nPlan,
  alpha = 0.05,
  alternative = c("twoSided", "greater", "less"),
  deltaMin = deltaTrue,
  testType = c("oneSample", "paired", "twoSample"),
  ratio = 1,
  parameter = NULL,
  eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "lai"),
  wantSamplePaths = TRUE,
  wantSimData = TRUE,
  relevanceTest = FALSE,
  relevanceSize = NULL,
  alphaRelevance = NULL,
  pb = TRUE,
  seed = NULL,
  nSim = 1000L,
  nBoot = nSim,
  nuMin = 2,
  ...
)

Arguments

Value

A list with the parameter and beta amongst other items

References

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.

Examples

designSaviT2WantPower(deltaTrue=0.9, nPlan=7, nSim=10)

Helper function to designing a Savi T-test (output esMin)

Description

Finds the parameter and esMin when provided with only alpha, power, and nPlan

Usage

designSaviT3WantEsMin(
  power,
  nPlan,
  alpha = 0.05,
  alternative = c("twoSided", "greater", "less"),
  testType = c("oneSample", "paired", "twoSample"),
  parameter = NULL,
  beta = NULL,
  eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "lai"),
  lowEsTrue = 0.01,
  highEsTrue = 3,
  ...
)

Arguments

Value

A list with the parameter and the targeted esMin amongst other items

References

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.

Examples

designSaviT3WantEsMin(power=0.7, nPlan=10)

Helper function to designing a savi T-test (output deltaMin based on the shortest interval at nPlan)

Description

Finds the parameter and deltaMin when provided with only alpha and nPlan

Usage

designSaviT3bWantParameter(
  nPlan,
  alpha = 0.05,
  alternative = c("twoSided", "greater", "less"),
  testType = c("oneSample", "paired", "twoSample"),
  parameter = NULL,
  eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "lai"),
  ...
)

Arguments

Value

A list with the parameter and the parameter amongst other items

References

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.

Examples

designSaviT3bWantParameter(nPlan=20)

Helper function: Get all names as entered by the user

Description

Helper function: Get all names as entered by the user

Usage

extractNameFromArgs(list, name)

Arguments

Value

returns a character string

Generates Normally Distributed Data Depending on the Design

Description

The designs supported are "oneSample", "paired", "twoSample".

Usage

generateNormalData(
  nPlan,
  nSim = 1000L,
  deltaTrue = NULL,
  muGlobal = 0,
  sigma = 1,
  sigma2 = 1,
  paired = FALSE,
  seed = NULL,
  meanDiffTrue = NULL
)

Arguments

Value

Returns a list of two data matrices contains at least the following components:

dataGroup1

a matrix of data dimension nSim by nPlan[1].

dataGroup2

a matrix of data dimension nSim by nPlan[2].

Examples

generateNormalData(20, 15, deltaTrue=0.3)

Generate Survival Data which Can Be Analysed With the survival Package

Description

Generate Survival Data which Can Be Analysed With the survival Package

Usage

generateSurvData(
  nP,
  nT,
  alpha = 1,
  lambdaP,
  lambdaT,
  seed = NULL,
  nDigits = 0,
  startTime = 1,
  endTime = 180,
  orderTime = TRUE,
  competeRatio = 0
)

Arguments

Value

A data set with time, status and group.

References

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

ter Schure, J., Pérez-Ortiz, M. F., Ly, A., & Grünwald, P. D. (2024). The Safe Logrank Test: Error control under continuous monitoring with unlimited horizon. The New England Journal of Statistics in Data Science, 2(2), 190-214, https://doi.org/10.51387/24-NEJSDS65.

Examples

generateSurvData(800, 800, alpha=1, lambdaP=0.008, lambdaT=0.008/2)

Helper function: Get all arguments as entered by the user

Description

Helper function: Get all arguments as entered by the user

Usage

getArgs()

Value

a list of variable names of class "call" that can be changed into names

Gets the Label of the Alternative Hypothesis

Description

Helper function that outputs the alternative hypothesis of the analysis.

Usage

getNameAlternative(
  alternative = c("twoSided", "greater", "less"),
  testType,
  h0 = 0
)

Arguments

Value

Returns a character string with the name of the analysis.

Gets the Label of the Test

Description

Helper function that outputs the name of the analysis.

Usage

getNameTestType(testType, testName, relevanceTest = FALSE)

Arguments

Value

Returns a character string with the name of the analysis.

Checks Whether a Vector of Object Inherits from the Class 'try-error'

Description

Checks whether any of the provided objects contains a try error.

Usage

isTryError(...)

Arguments

Value

Returns TRUE if there's some object that's a try-error, FALSE when all objects are not try-errors.

Examples

x <- 1
y <- "a"
z <- try(integrate(exp, -Inf, Inf))
isTryError(x, y)
isTryError(x, y, z)

Logarithmic hyperbolic cosine

Description

Logarithmic hyperbolic cosine

Usage

lcosh(x)

Arguments

Value

numeric value

Examples

lcosh(7)

Helper function computes single component of the exact logrank e-value

Description

Helper function computes single component of the exact logrank e-value

Usage

logrankSingleEExact(obs0, obs1, y0, y1, thetaS, theta0 = 1, ...)

Arguments

Value

Returns a list containing at least the following components:

logP0

Log likelihood of Fisher's hypergeometric at the null

logEValueLess

Log likelihood of Fisher's hypergeometric at the alternative

logEValueGreater

Log likelihood of Fisher's hypergeometric at 1/alternative

References

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

ter Schure, J., Pérez-Ortiz, M. F., Ly, A., & Grünwald, P. D. (2024). The Safe Logrank Test: Error control under continuous monitoring with unlimited horizon. The New England Journal of Statistics in Data Science, 2(2), 190-214, https://doi.org/10.51387/24-NEJSDS65.

Examples

#'
y0Vector <- c(5, 4, 3, 3, 2, 1)
y1Vector <- c(5, 5, 4, 2, 2, 0)
obs0Vector <- c(1, 1, 0, 1, 0, 1)
obs1Vector <- c(0, 0, 1, 0, 1, 0)

logEValueGreater <- logEValueLess <- vector("numeric", length(y0Vector))

for (i in seq_along(y0Vector)) {
  tempResult <- logrankSingleEExact(obs0=obs0Vector[i], obs1=obs1Vector[i],
                                    y0=y0Vector[i], y1=y1Vector[i],
                                    thetaS=0.7, theta0=1)
  logEValueLess[i] <- tempResult[["logEValueLess"]]
  logEValueGreater[i] <- tempResult[["logEValueGreater"]]
}

eValueLess <- exp(sum(logEValueLess))
eValueLess #1.116161
eValueGreater <- exp(sum(logEValueGreater))
eValueGreater # 0.7665818
eValue <- 1/2*eValueLess + 1/2*eValueGreater
eValue # 0.9413714

Helper function computes single component of the logrank statistic

Description

Helper function computes single component of the logrank statistic

Usage

logrankSingleZ(obs0, obs1, y0, y1, ...)

Arguments

Value

Returns a list containing at least the following components:

oMinE

observed minus expected.

v

hypergeometric variance.

References

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

ter Schure, J., Pérez-Ortiz, M. F., Ly, A., & Grünwald, P. D. (2024). The Safe Logrank Test: Error control under continuous monitoring with unlimited horizon. The New England Journal of Statistics in Data Science, 2(2), 190-214, https://doi.org/10.51387/24-NEJSDS65.

Examples

y0Vector <- c(6, 4, 4, 1, 0)
y1Vector <- c(6, 6, 5, 2, 2)
obs0Vector <- c(1, 0, 2, 1, 0)
obs1Vector <- c(0, 1, 1, 0, 1)

varVector <- oMinEVector <-y0Vector

for (i in seq_along(y0Vector)) {
  tempResult <- logrankSingleZ(obs0=obs0Vector[i], obs1=obs1Vector[i],
                              y0=y0Vector[i], y1=y1Vector[i])
  oMinEVector[i] <- tempResult[["oMinE"]]
  varVector[i] <- tempResult[["v"]]
}

sum(oMinEVector)/sqrt(sum(varVector))

Helper function to create running intersections

Description

Helper function to create running intersections

Usage

makeRunningIntersection(x, upper = TRUE)

Arguments

Value

a sequence of numerics representing the running minimum or maximum

Examples

makeRunningIntersection(c(6, -1, 3, 12))

Checks and outputs a threshold for a minimal efficacy analysis

Description

Checks and outputs a threshold for a minimal efficacy analysis

Usage

matchAlphaRelevanceWith(
  alphaRelevance,
  alpha = NULL,
  power = NULL,
  beta = NULL,
  betaDefault = 0.2
)

Arguments

Value

a alphaRelevance threshold

Examples

matchAlphaRelevanceWith(0.3)

Match the parameter of a savi z or t-test

Description

Based on the minimal clinically relevant effect size esMin, sigma (for z-tests), alternative and eType

Usage

matchEParameterWith(
  esMin,
  analysisType = c("z", "t", "logRank"),
  sigma = 1,
  alternative = c("twoSided", "greater", "less"),
  eType = c("mom", "eGauss", "imom", "eCauchy", "grow"),
  parameter = NULL
)

Arguments

Value

the parameter, a numeric value

Examples

matchEParameterWith(0.4)

Match the meanDiffMin of a savi z-test

Description

Based on the parameter, sigma, alternative and eType

Usage

matchEsMinWith(
  parameter,
  analysisType = c("z", "t"),
  sigma = 1,
  alternative = c("twoSided", "greater", "less"),
  eType = c("mom", "eGauss", "imom", "eCauchy", "grow")
)

Arguments

Value

the parameter, a numeric value

Examples

matchEsMinWith(parameter=0.4)

Helper function to check whether the power or beta (redundant now) argument is given

Description

Helper function to check whether the power or beta (redundant now) argument is given

Usage

matchPowerWith(power, beta = NULL)

Arguments

Value

numeric representing power

Examples

matchPowerWith(0.8)

Match the parameter of a minimal efficacy savi z-test

Description

Based on the relevanceSize, meanDiffMin, alternative and eType

Usage

matchRelevanceParameterWith(relevanceSize, esMin, esTrue)

Arguments

Value

the parameter, a numeric value

Examples

matchRelevanceParameterWith(0.4)

Computes the p-value for the Z-test

Description

Computes the p-value for the Z-test

Usage

pValueFromZStat(z, alternative = c("twoSided", "less", "greater"), ...)

pValueZTest(
  x,
  y = NULL,
  paired = FALSE,
  ciValue = NULL,
  alpha = 0.05,
  sigma = 1,
  h0 = 0,
  alternative = c("twoSided", "greater", "less"),
  ...
)

Arguments

Value

pValueTest object

Examples

pValueZTest(rnorm(10))

Plots Results of Simulations for Comparing Hyperparameters for Savi Tests of Two Proportions

Description

Plots Results of Simulations for Comparing Hyperparameters for Savi Tests of Two Proportions

Usage

## S3 method for class 'savi2x2Sim'
plot(x, ...)

Arguments

Value

Plot data, mainly called for side effects, the plot of simulation results.

Examples

priorList1 <- list(betaA1 = 10, betaA2 = 1, betaB1 = 1, betaB2 = 10)
priorList2 <- list(betaA1 = 0.18, betaA2 = 0.18, betaB1 = 0.18, betaB2 = 0.18)
priorList3 <- list(betaA1 = 1, betaA2 = 1, betaB1 = 1, betaB2 = 1)

simResult <- simulateTwoProportions(
  hyperparameterList = list(priorList1, priorList2, priorList3),
  alternativeRestriction = "none",
  alpha = 0.1, beta = 0.2, na = 1, nb = 1,
  deltamax = -0.4, deltamin = -0.9, deltaGridSize = 3,
  M = 10
  )

plot(simResult)

Plots the saviDesign object for designs with sample paths

Description

Plots the saviDesign object for designs with sample paths

Usage

## S3 method for class 'saviDesign'
plot(
  x,
  main = NULL,
  xlab = NULL,
  ylab = NULL,
  xlim = NULL,
  ylim = NULL,
  maxNBins = 40,
  numSamplePaths = 100,
  wantStepLines = FALSE,
  wantQuantiles = NULL,
  breaks = NULL,
  lwd = 2,
  pch = 15,
  colQuant = "#AA0000",
  overColour = "#8FC6E3",
  overColourBorder = "#1F78B4E6",
  underColour = "#E7A72199",
  underColourBorder = "#BD8E17FF",
  continueColour = "#8CA252E6",
  continueColourBorder = "#637939E6",
  histInnerColour = col,
  col = overColour,
  border = overColourBorder,
  cex = 1.3,
  yLabPAdj = -1,
  wantNotStoppedHist = FALSE,
  wantNotStoppedSamplePaths = TRUE,
  wantLegend = TRUE,
  legendAdjRelevance = c(-0.1, 0.5, 1),
  legendAdj = c(0.2, 0.8),
  legendCexFactor = 0.85,
  histFewestAtTop = FALSE,
  density = NULL,
  wantTitle = TRUE,
  ...
)

Arguments

Value

Nothing it only plots

Functions

• plot(safeDesign): Deprecated version of plot.saviDesign

Plots the saviTest object for sequential analyses

Description

Plots the saviTest object for sequential analyses

Usage

## S3 method for class 'saviTest'
plot(
  x,
  main = NULL,
  xlab = NULL,
  ylab = NULL,
  xlim = NULL,
  ylim = NULL,
  lwd = 3,
  cex = 1.3,
  fillPlot = NULL,
  switchNFill = 10000,
  logScale = NULL,
  switchNLog = 30,
  h0Colour = "darkgrey",
  overColour = "#8FC6E3",
  overColourBorder = "#1F78B4E6",
  underColour = "#E7A72199",
  underColourBorder = "#BD8E17FF",
  continueColour = "#8CA252E6",
  continueColourBorder = "#637939E6",
  col = overColour,
  border = overColourBorder,
  wantConfSeqPlot = FALSE,
  add = FALSE,
  density = NULL,
  angle = 45,
  xaxt = NULL,
  yaxt = NULL,
  fillOddEven = FALSE,
  runInt = TRUE,
  wantRelevance = NULL,
  ...
)

Arguments

Value

Returns nothing just plots

Functions

• plot(safeTest): Deprecated version of plot.saviTest

Plot bounds of a savi confidence sequence of the difference or log odds ratio for two proportions against the number of data blocks in two data streams ya and yb.

Description

Plot bounds of a savi confidence sequence of the difference or log odds ratio for two proportions against the number of data blocks in two data streams ya and yb.

Usage

plotConfidenceSequenceTwoProportions(
  ya,
  yb,
  saviDesign,
  differenceMeasure = c("difference", "odds"),
  precision = 100,
  deltaStart = 0.001,
  deltaStop = 3,
  trueDifference = NA
)

Arguments

Value

no return value; called for its side effects, a plot of the confidence sequence.

Examples

set.seed(39413)
ya <- rbinom(n = 30, size = 1, prob = 0.1)
yb <- rbinom(n = 30, size = 1, prob = 0.8)
balancedSaviDesign <- designSaviTwoProportions(na = 1,
                                               nb = 1,
                                               nBlocksPlan = 30)
plotConfidenceSequenceTwoProportions(ya = ya,
                                     yb = yb,
                                     saviDesign = balancedSaviDesign,
                                     differenceMeasure = "difference",
                                     precision = 15,
                                     trueDifference = 0.7)

#log odds ratio difference measure
plotConfidenceSequenceTwoProportions(ya = ya,
                                     yb = yb,
                                     saviDesign = balancedSaviDesign,
                                     differenceMeasure = "odds",
                                     precision = 15,
                                     deltaStop = 5,
                                     trueDifference = log(36))

#switch ya and yb: observe negative log odds ratio in the data, plot mirrored in x-axis
plotConfidenceSequenceTwoProportions(ya = yb,
                                     yb = ya,
                                     saviDesign = balancedSaviDesign,
                                     differenceMeasure = "odds",
                                     precision = 15,
                                     deltaStop = 5,
                                     trueDifference = -log(36))

Plots the Histogram of Stopping Times

Description

Helper function to display the histogram of stopping times.

Usage

plotHistogramDistributionStoppingTimes(
  saviSim,
  nPlan,
  deltaTrue,
  showOnlyNRejected = FALSE,
  nBin = 25L,
  ...
)

Arguments

Value

a histogram object, and called for its side-effect to plot the histogram.

Prints Results of Simulations for Comparing Hyperparameters for Savi Tests of Two Proportions

Description

Prints Results of Simulations for Comparing Hyperparameters for Savi Tests of Two Proportions

Usage

## S3 method for class 'savi2x2Sim'
print(x, ...)

Arguments

Value

The data frame with simulation results, called for side effects to pretty print the simulation results.

Examples

priorList1 <- list(betaA1 = 10, betaA2 = 1, betaB1 = 1, betaB2 = 10)
priorList2 <- list(betaA1 = 0.18, betaA2 = 0.18, betaB1 = 0.18, betaB2 = 0.18)
priorList3 <- list(betaA1 = 1, betaA2 = 1, betaB1 = 1, betaB2 = 1)

simResult <- simulateTwoProportions(
  hyperparameterList = list(priorList1, priorList2, priorList3),
  alternativeRestriction = "none",
  alpha = 0.1, beta = 0.2, na = 1, nb = 1,
  deltamax = -0.4, deltamin = -0.9, deltaGridSize = 3,
  M = 10
  )

Print Method for Savi Test Objects

Description

Printing objects of class 'saviTest' modelled after print.power.htest().

Usage

## S3 method for class 'saviDesign'
print(x, digits = getOption("digits"), prefix = "\t", ...)

Arguments

Value

No returned value, called for side effects.

Functions

• print(safeDesign): Deprecated version of print.saviDesign

Examples

designSaviZ(meanDiffMin=0.5)
designSaviT(deltaMin=0.5)
designSaviLogrank(hrMin=0.7)

Print Method for Savi Test Objects

Description

Printing objects of class 'saviTest' modelled after print.htest().

Usage

## S3 method for class 'saviTest'
print(x, digits = getOption("digits"), prefix = "\t", ...)

Arguments

Value

No returned value, called for side effects.

Functions

• print(safeTest): Deprecated version of print.saviTest

Examples

saviTTest(rnorm(19))

Randomly samples from a logrank distribution

Description

Draws a number of occurrences in group 1 (treatment) out of obsTotal number of occurrences.

Usage

rLogrank(n = 1, y0, y1, obsTotal, theta)

Arguments

Value

integer representing the number of occurrences in group 1 out of obsTotal number of occurrences.

Author(s)

Muriel Felipe Perez-Ortiz and Alexander Ly

References

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

ter Schure, J., Pérez-Ortiz, M. F., Ly, A., & Grünwald, P. D. (2024). The Safe Logrank Test: Error control under continuous monitoring with unlimited horizon. The New England Journal of Statistics in Data Science, 2(2), 190-214, https://doi.org/10.51387/24-NEJSDS65.

Examples

rLogrank(y0=360, y1=89, obsTotal=12, theta=3.14)

Auxiliary function for sampling of the logrank simulations to return the integer 1 event per time.

Description

Auxiliary function for sampling of the logrank simulations to return the integer 1 event per time.

Usage

returnOne()

Value

1

Examples

returnOne()

Safe Anytime-Valid Logrank Test

Description

A savi test to test whether there is a difference between two survival curves. This function builds on the Mantel-Cox version of the logrank test.

Usage

saviLogrankTest(
  formula,
  designObj = NULL,
  ciValue = NULL,
  data = NULL,
  survTime = NULL,
  group = NULL,
  pilot = FALSE,
  exact = TRUE,
  computeZ = TRUE,
  ...
)

saviLogrankTestStat(
  z,
  nEvents,
  designObj,
  ciValue = NULL,
  dataNull = 1,
  sigma = 1
)

Arguments

Value

Returns an object of class 'saviTest'. An object of class 'saviTest' is a list containing at least the following components:

statistic

the value of the summary, i.e., z-statistic or the e-value.

nEvents

The number of observed events.

eValue

the e-value of the savi test.

confSeq

An anytime-valid confidence sequence.

estimate

To be implemented: An estimate of the hazard ratio.

testType

"logrank".

dataName

a character string giving the name(s) of the data.

designObj

an object of class "saviDesign" obtained from designSaviLogrank.

sumStats

a list containing.the time of events, the progression of the risk sets and events.

call

the expression with which this function is called.

Functions

• saviLogrankTestStat(): Savi Logrank Test based on Summary Statistic Z All provided data (i.e., z-scores) are assumed to be centred on a hazard ratio = 1, thus, log(hr) = 0 , and the proper (e.g., hypergeometric) scaling is applied to the data, so sigma = 1. The null hypothesis in the design object pertains to the population and is allowed to differ from log(theta) = 0.

References

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

ter Schure, J., Pérez-Ortiz, M. F., Ly, A., & Grünwald, P. D. (2024). The Safe Logrank Test: Error control under continuous monitoring with unlimited horizon. The New England Journal of Statistics in Data Science, 2(2), 190-214, https://doi.org/10.51387/24-NEJSDS65.

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

ter Schure, J., Pérez-Ortiz, M. F., Ly, A., & Grünwald, P. D. (2024). The Safe Logrank Test: Error control under continuous monitoring with unlimited horizon. The New England Journal of Statistics in Data Science, 2(2), 190-214, https://doi.org/10.51387/24-NEJSDS65.

Examples

# Example taken from survival::survdiff

designObj <- designSaviLogrank(hrMin=1/2)

ovData <- survival::ovarian
ovData$survTime <- survival::Surv(ovData$futime, ovData$fustat)

saviLogrankTest(formula=survTime~ rx, data=ovData, designObj=designObj)

saviLogrankTest(survTime=survTime, group=rx, data=ovData, designObj=designObj)

# Examples taken from coin::logrank_test
## Example data (Callaert, 2003, Tab. 1)
#'
callaert <- data.frame(
  time = c(1, 1, 5, 6, 6, 6, 6, 2, 2, 2, 3, 4, 4, 5, 5),
  group = factor(rep(0:1, c(7, 8)))
)

designObj <- designSaviLogrank(hrMin=1/2)

saviLogrankTest(survival::Surv(callaert$time)~callaert$group,
                designObj = designObj)

saviLogrankTest(survTime=survival::Surv(callaert$time),
                group=callaert$group, designObj = designObj)

result <- saviLogrankTest(survTime=survival::Surv(callaert$time),
                group=callaert$group, designObj = designObj)

result

##  Sequentially
# Greater
eValueGreater <- exp(cumsum(result$sumStats$logEValueGreater))
# Less
eValueLess <- exp(cumsum(result$sumStats$logEValueLess))

# twoSided
eValueTwoSided <- 1/2*eValueGreater+1/2*eValueLess

eValueTwoSided
result$eValue

###### Example switching between savi exact and savi Gaussian logrank test

designObj <- designSaviLogrank(0.8, alternative="less")

dat <- safestats::generateSurvData(300, 300, 2, 0.0065, 0.0065*0.8, seed=1)
survTime <- survival::Surv(dat$time, dat$status)

resultE <- saviLogrankTest(survTime ~ dat$group,
                           designObj = designObj)

resultG <- saviLogrankTest(survTime ~ dat$group,
                           designObj = designObj, exact=FALSE)

resultE
resultG

###### Example switching between savi exact and savi Gaussian logrank test other side

designObj <- designSaviLogrank(1/0.8, alternative="greater")

resultE <- saviLogrankTest(survTime ~ dat$group,
                           designObj = designObj)

resultG <- saviLogrankTest(survTime ~ dat$group,
                           designObj = designObj, exact=FALSE)

if (log(resultE$eValue) >= 0 && log(resultG$eValue) >= 0 )
  stop("one-sided wrong")

Safe Anytime-Valid Student's T-Test.

Description

Savi one- and two-sample T-tests. Takes as input vector(s) of data and a designObj from designSaviT. The function is modelled after t.test().

Usage

saviTTest(x, ...)

## Default S3 method:
saviTTest(
  x,
  y = NULL,
  designObj = NULL,
  paired = FALSE,
  varEqual = TRUE,
  ciValue = NULL,
  maxRoot = 10,
  sequential = NULL,
  tDensity = FALSE,
  nuMin = 2,
  wantCi = TRUE,
  ...
)

## S3 method for class 'formula'
saviTTest(formula, data, subset, na.action, ...)

savi.t.test(
  x,
  y = NULL,
  paired = FALSE,
  designObj = NULL,
  varEqual = TRUE,
  ciValue = NULL,
  ...
)

Arguments

Value

Returns an object of class 'saviTest'. An object of class 'saviTest' is a list containing at least the following components:

statistic

the value of the t-statistic.

n

The realised sample size(s).

eValue

the realised e-value from the savi test.

confSeq

A savi confidence interval for the mean (difference)

estimate

the estimated means or mean (difference) depending on whether it was a one-sample test or a two-sample test.

stderr

the standard error of the mean (difference), used as denominator in the t-statistic formula.

dataName

a character string giving the name(s) of the data.

designObj

an object of class "saviDesign" obtained from designSaviT().

call

the expression with which this function is called.

Functions

• safeTTestStat(): Deprecated version of saviTTestStat

• safeTTestStatNEffNu(): Deprecated version of saviTTestStatNEffNu #nu >0 degrees of freedom

• safeTTestStatNEffNuMom(): Deprecated version of saviTTestStatNEffNuMom

• safeTTestStatTDensity(): Deprecated version of saviTTestStatTDensity

• safeTTest(): Deprecated version of saviTTest

• safeTTest.default(): Deprecated version of saviTTest.default

• safeTTest.formula(): Deprecated version of saviTTest.formula

• safe.t.test(): Deprecated version of savi.t.test

References

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.

Pérez-Ortiz, M. F., Lardy, T., de Heide, R., & Grünwald, P. D. (2024). E-statistics, group invariance and anytime valid testing. The Annals of Statistics, 52(4), 1410-1432, http://dx.doi.org/10.1214/24-AOS2394.

Wang, H., & Ramdas, A. (in press). Anytime-valid t-tests and confidence sequences for Gaussian means with unknown variance. Sequential Analysis, https://doi.org/10.48550/arXiv.2310.03722.

Examples

## Examples with simulated data ----
set.seed(1)
x <- rnorm(30, mean=0)
y <- rnorm(40, mean=0)

# Because no designObj is specified, a default
# designObj is used with deltaMin = 1/2,
# which can be thought of as a medium effect size
# according to Cohen.

res <- saviTTest(x=x, y=y)

# By default sequential=TRUE, because length(x) <= 200.
# This allows us to visualise the e-value as a function of
# the n1 and associated n2, where the ratio of sample sizes,
# ratio=n2/n1 is maintained. Here the e-value at n1=6 uses data
# x[1:6] and y[1:ceil(ratio*6)]
plot(res)

# Plots the confidence sequence
plot(res, wantConfSeqPlot=TRUE)

# See ?designSaviT for more info
# This designObj also allows for
# evidence quantification that the
# mean difference is minimal clinically
# relevant, here, larger than
# relevanceSize=meanDiffMin
designObj <- designSaviT(deltaMin=0.7, alpha=0.05,
                         alternative="twoSided",
                         testType="twoSample",
                         relevanceTest=TRUE)

res <- saviTTest(x=x, y=y, designObj=designObj)

plot(res)

# Note that the e-value against relevance falls below alphaRelevance
# We can reject the hypothesis that the effect is relevantly large,
# larger than designObj$relevanceTestSim$parameter, after a sample size of
min(which(res$eRelevanceVec <= designObj$relevanceTestSim$alpha))

set.seed(2)
x <- rnorm(30, mean=0.6)
y <- rnorm(40, mean=0)

res <- saviTTest(x, y, designObj=designObj)

plot(res)
# We could have stopped sampling after
min(which(res$eValueVec >= 1/designObj$alpha))
# the yellow curve crosses 1/alpha sooner,
# but we do **not** compare eRelevance >= 1/alpha, only eRelevance <= alphaRelevance.

## Classical example: Student's sleep data -----
plot(extra ~ group, data = sleep)

designObj <- designSaviT(deltaMin=0.6, testType="twoSample")

## Traditional interface
with(sleep, saviTTest(extra[group == 1], extra[group == 2],
                      designObj=designObj))

## Formula interface
saviTTest(extra ~ group, data = sleep, designObj=designObj)

## Formula interface to one-sample test
designObj1 <- designSaviT(deltaMin=0.6,
                          testType="oneSample",
                          sigma=2)

saviTTest(extra ~ 1, data = sleep, designObj=designObj1)

## Formula interface to paired test
## The sleep data are actually paired, so could have been in wide format:
designObjPaired <- designSaviT(deltaMin=0.6,
                               testType="paired",
                               sigma=1.4)
sleep2 <- reshape(sleep, direction = "wide",
                  idvar = "ID", timevar = "group")
saviTTest(Pair(extra.1, extra.2) ~ 1, data = sleep2,
          designObj=designObjPaired)

Perform a Savi Test for Two Proportions with Stream Data

Description

Perform a savi test for two proportions (a 2x2 contingency table test) with a result object retrieved through the design function for planning an experiment to compare two proportions in this package, designSaviTwoProportions().

Usage

saviTwoProportionsTest(
  ya,
  yb,
  designObj = NULL,
  wantConfidenceSequence = FALSE,
  ciValue = NULL,
  confidenceBoundGridPrecision = 20,
  logOddsConfidenceSearchBounds = c(0.01, 5),
  pilot = FALSE
)

savi.prop.test(
  ya,
  yb,
  designObj = NULL,
  wantConfidenceSequence = FALSE,
  ciValue = NULL,
  confidenceBoundGridPrecision = 20,
  logOddsConfidenceSearchBounds = c(0.01, 5),
  pilot = FALSE
)

Arguments

Value

Returns an object of class 'saviTest'. An object of class 'saviTest' is a list containing at least the following components:

n

The realised sample size(s).

eValue

the e-value of the savi test.

dataName

a character string giving the name(s) of the data.

designObj

an object of class "saviDesign" described in designSaviTwoProportions().

Examples

#balanced design
yb <- c(1,0,1,1,1,0,1)
ya <- c(1,0,1,0,0,0,1)
saviDesign <- designSaviTwoProportions(na = 1,
                                       nb = 1,
                                       beta = 0.20,
                                       delta = 0.6,
                                       alternativeRestriction = "none",
                                       M = 1e1)
saviTwoProportionsTest(ya = ya, yb = yb, designObj = saviDesign)

#pilot
saviTwoProportionsTest(ya = ya, yb = yb, pilot = TRUE)

#unbalanced design
yb <- c(1,0,1,1,1,0,1)
ya <- c(2,2,1,2,0,2,2)
saviDesign <- designSaviTwoProportions(na = 2,
                                       nb = 1,
                                       beta = 0.20,
                                       delta = 0.6,
                                       alternativeRestriction = "none",
                                       M = 1e1)
saviTwoProportionsTest(ya = ya, yb = yb, designObj = saviDesign)

Safe Anytime-Valid Z-Test

Description

Savi one- and two-sample Z-tests. Takes as input vector(s) of data and a designObj from designSaviZ. The function is modelled after t.test().

Usage

saviZTest(x, ...)

## Default S3 method:
saviZTest(
  x,
  y = NULL,
  paired = FALSE,
  designObj = NULL,
  ciValue = NULL,
  maxRoot = 10,
  sequential = NULL,
  ...
)

## S3 method for class 'formula'
saviZTest(formula, data, subset, na.action, ...)

savi.z.test(x, y = NULL, paired = FALSE, designObj = NULL, ...)

Arguments

Value

Returns an object of class 'saviTest'. An object of class 'saviTest' is a list containing at least the following components:

statistic

the value of the test statistic. Here the z-statistic.

n

The realised sample size(s).

eValue

the e-value of the savi test.

confSeq

a savi confidence interval for the mean (difference).

estimate

the estimated means or mean (difference) depending on whether it was a one-sample test or a two-sample test.

dataName

a character string giving the name(s) of the data.

designObj

an object of class "saviDesign" described in designSaviZ().

call

the expression with which this function is called.

Functions

• safeZTest(): Deprecated version of saviZTest

• safeZTest(default): Deprecated version of saviZTest.default

• safe.z.test(): Deprecated version of saviZTest

References

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.

Examples

## Examples with simulated data ----
set.seed(1)
x <- rnorm(30, mean=0)
y <- rnorm(40, mean=0)

# Because no designObj is specified, a default
# designObj is used with meanDiffMin = 1/2 and sigma=1,
# which can be thought of as a medium effect size
# according to Cohen.

res <- saviZTest(x=x, y=y)

# By default sequential=TRUE, because length(x) <= 200.
# This allows us to visualise the e-value as a function of
# the n1 and associated n2, where the ratio of sample sizes,
# ratio=n2/n1 is maintained. Here the e-value at n1=6 uses data
# x[1:6] and y[1:ceil(ratio*6)]
plot(res)

# Plots the confidence sequence
plot(res, wantConfSeqPlot=TRUE)

# See ?designSaviZ for more info
# This designObj also allows for
# evidence quantification that the
# mean difference is minimal clinically
# relevant, here, larger than
# relevanceSize=meanDiffMin
designObj <- designSaviZ(meanDiffMin=0.7, alpha=0.05,
                         alternative="twoSided",
                         testType="twoSample",
                         relevanceTest=TRUE)

res <- saviZTest(x=x, y=y, designObj=designObj)

plot(res)

# Note that the e-value against relevance falls below alphaRelevance
# We can reject the hypothesis that the effect is relevantly larger,
# larger than designObj$relevanceTestSim$parameter after a sample size of
min(which(res$eRelevanceVec <= designObj$relevanceTestSim$alpha))

set.seed(2)
x <- rnorm(30, mean=0.6)
y <- rnorm(40, mean=0)

res <- saviZTest(x, y, designObj=designObj)

plot(res)
# We could have stopped sampling after
min(which(res$eValueVec >= 1/designObj$alpha))
# the yellow curve crosses 1/alpha sooner,
# but we do **not** compare eRelevance >= 1/alpha, only eRelevance <= alphaRelevance

## Classical example: Student's sleep data -----
plot(extra ~ group, data = sleep)

designObj <- designSaviZ(meanDiffMin=0.6, sigma=2,
                         testType="twoSample")

## Traditional interface
with(sleep, saviZTest(extra[group == 1], extra[group == 2],
                      designObj=designObj))

## Formula interface
saviZTest(extra ~ group, data = sleep, designObj=designObj)

## Formula interface to one-sample test
designObj1 <- designSaviZ(meanDiffMin=0.6,
                          testType="oneSample",
                          sigma=2)

saviZTest(extra ~ 1, data = sleep, designObj=designObj1)

## Formula interface to paired test
## The sleep data are actually paired, so could have been in wide format:
designObjPaired <- designSaviZ(meanDiffMin=0.6,
                               testType="paired",
                               sigma=1.4)
sleep2 <- reshape(sleep, direction = "wide",
                  idvar = "ID", timevar = "group")
saviZTest(Pair(extra.1, extra.2) ~ 1, data = sleep2,
          designObj=designObjPaired)

Computes E-Values Based on the Z-Statistic

Description

Computes e-values using the z-statistic and the sample sizes based on the test defining parameter phiS.

Evidence for practical equivalence requires the e-value to be small, i.e. smaller than alphaRelevance. If the alternative holds true, i.e. meanDiffTrue >= relevanceSize, then there is no more than alphaRelevance probability of ever seeing eRelevance <= alphaRelevance.

Usage

saviZTestStat(
  z,
  n1,
  n2 = NULL,
  parameter,
  alternative = c("twoSided", "less", "greater"),
  paired = FALSE,
  sigma = 1,
  eType = c("mom", "eGauss", "imom", "eCauchy", "grow"),
  ...
)

saviRelevanceZStat(
  z,
  n1,
  n2 = NULL,
  parameter = NULL,
  alternative = c("twoSided", "less", "greater"),
  paired = FALSE,
  sigma = 1,
  eType = "grow",
  relevanceSize = NULL,
  ...
)

Arguments

Value

Returns an e-value.

References

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.

Examples

saviZTestStat(z=3, n1=100, parameter=0.4, eType="grow")
saviZTestStat(z=3, n1=100, parameter=0.4^2, eType="eGauss")
saviZTestStat(z=3, n1=100, parameter=0.4, eType="eCauchy")
# evidence for the alternative over minimal efficacy
saviRelevanceZStat(z=3, n1=100, parameter=0.4)
# evidence for minimal efficacy over the alternative
saviRelevanceZStat(z=0.35, n1=100, parameter=0.4)

Simulate stopping times for the exact savi logrank test

Description

Simulate stopping times for the exact savi logrank test

Usage

sampleLogrankStoppingTimes(
  hrTrue,
  alpha = 0.05,
  alternative = c("twoSided", "less", "greater"),
  m0 = 50000L,
  m1 = 50000L,
  nSim = 1000L,
  groupSizePerTimeFunction = returnOne,
  seed = NULL,
  power = NULL,
  beta = NULL,
  relevanceTest = FALSE,
  relevanceSize = NULL,
  wantEValuesAtNMax = FALSE,
  wantSamplePaths = TRUE,
  wantSimData = TRUE,
  parameter = NULL,
  nMax = Inf,
  pb = TRUE,
  hrMin = NULL,
  ...
)

Arguments

Value

a list with stoppingTimes and breakVector. Entries of breakVector are 0, 1. A 1 represents stopping due to exceeding nMax, and 0 due to 1/alpha threshold crossing, or running out of participants, which implies that the corresponding stopping time is Inf.

Author(s)

Muriel Felipe Perez-Ortiz and Alexander Ly

References

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

ter Schure, J., Pérez-Ortiz, M. F., Ly, A., & Grünwald, P. D. (2024). The Safe Logrank Test: Error control under continuous monitoring with unlimited horizon. The New England Journal of Statistics in Data Science, 2(2), 190-214, https://doi.org/10.51387/24-NEJSDS65.

Examples

sampleLogrankStoppingTimes(0.7, nSim=10, nMax=30)

Simulate stopping times for the savi T-test

Description

Simulate stopping times for the savi T-test

Usage

sampleStoppingTimesSaviT(
  deltaTrue,
  alpha = 0.05,
  alternative = c("twoSided", "less", "greater"),
  testType = c("oneSample", "paired", "twoSample"),
  ratio = 1,
  deltaMin = NULL,
  parameter = NULL,
  lowN = 3L,
  nMax = 100000000L,
  eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "lai"),
  wantEValuesAtNMax = FALSE,
  nuMin = 2,
  power = NULL,
  wantSamplePaths = TRUE,
  wantSimData = TRUE,
  sigma = 1,
  sigma2 = 1,
  pb = TRUE,
  seed = NULL,
  nSim = 1000L,
  relevanceTest = FALSE,
  relevanceSize = NULL,
  beta = NULL,
  alphaRelevance = NULL,
  ...
)

Arguments

Value

a list with stoppingTimes and breakVector. Entries of breakVector are 0, 1. A 1 represents stopping due to exceeding nMax, and 0 due to 1/alpha threshold crossing, which implies that in corresponding stopping time is Inf.

Functions

• sampleStoppingTimesSafeT(): Deprecated version of sampleStoppingTimesSaviT

References

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.

Examples

sampleStoppingTimesSaviT(0.7, nSim=10, nMax=20)

Simulate stopping times for the savi Z-test

Description

Simulate stopping times for the savi Z-test

Usage

sampleStoppingTimesSaviZ(
  meanDiffTrue,
  alpha = 0.05,
  alternative = c("twoSided", "less", "greater"),
  testType = c("oneSample", "paired", "twoSample"),
  sigma = 1,
  kappa = sigma,
  ratio = 1,
  meanDiffMin = NULL,
  parameter = NULL,
  nMax = 1000L,
  eType = c("mom", "eGauss", "imom", "eCauchy", "grow"),
  wantEValuesAtNMax = FALSE,
  power = NULL,
  wantSamplePaths = TRUE,
  wantSimData = TRUE,
  pb = TRUE,
  seed = NULL,
  nSim = 1000L,
  relevanceTest = FALSE,
  relevanceSize = NULL,
  beta = NULL,
  alphaRelevance = NULL,
  ...
)

Arguments

Value

a list with stoppingTimes and breakVector. Entries of breakVector are 0, 1. A 1 represents stopping due to exceeding nMax, and 0 due to 1/alpha threshold crossing, which implies that in corresponding stopping time is Inf.

Functions

• sampleStoppingTimesSafeZ(): Deprecated version of sampleStoppingTimesSaviZ

References

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.

Examples

sampleStoppingTimesSaviZ(0.7, nSim=10, nMax=20)

Computes E-Values Based on the T-Statistic

Description

A summary stats version of saviTTest() with the data replaced by t, n1 and n2, and the design object by parameter

This is saviTTestStat() based on t-densities instead of hypergeometric functions.

Evidence for practical equivalence requires the e-value to be small, i.e. smaller than alphaRelevance. If the alternative holds true, i.e. deltaTrue >= relevanceSize, then there is no more than alphaRelevance probability of ever seeing eRelevance <= alphaRelevance.

Usage

saviTTestStat(
  t,
  n1,
  n2 = NULL,
  parameter,
  alternative = c("twoSided", "less", "greater"),
  tDensity = FALSE,
  paired = FALSE,
  nuMin = 2,
  eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "lai"),
  ...
)

saviTTestStatNEffNu(
  t,
  nEff,
  nu,
  parameter,
  alternative = c("twoSided", "less", "greater"),
  tDensity = FALSE,
  paired = FALSE,
  nuMin = 2,
  eType = c("mom", "eGauss", "imom", "eCauchy", "grow", "lai"),
  ...
)

saviTTestStatNEffNuMom(
  t,
  nEff,
  nu,
  parameter,
  alternative = c("twoSided", "less", "greater"),
  tDensity = FALSE,
  paired = FALSE,
  k = 1,
  ...
)

saviTTestStatNEffNuGrow(
  t,
  nEff,
  nu,
  parameter,
  alternative = c("twoSided", "less", "greater"),
  tDensity = FALSE,
  paired = FALSE,
  ...
)

saviTTestStatTDensity(
  t,
  parameter,
  nu,
  nEff,
  alternative = c("twoSided", "less", "greater"),
  paired = FALSE,
  ...
)

saviRelevanceTStatNEffNu(
  t,
  nEff,
  nu,
  parameter = NULL,
  alternative = c("twoSided", "less", "greater"),
  eType = "grow",
  tDensity = FALSE,
  paired = FALSE,
  relevanceSize,
  nuMin = 2,
  ...
)

Arguments

Value

Returns a numeric that represent the e10, that is, the e-value in favour of the alternative over the null

References

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.

Pérez-Ortiz, M. F., Lardy, T., de Heide, R., & Grünwald, P. D. (2024). E-statistics, group invariance and anytime valid testing. The Annals of Statistics, 52(4), 1410-1432, http://dx.doi.org/10.1214/24-AOS2394.

Wang, H., & Ramdas, A. (in press). Anytime-valid t-tests and confidence sequences for Gaussian means with unknown variance. Sequential Analysis, https://doi.org/10.48550/arXiv.2310.03722.

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.

Pérez-Ortiz, M. F., Lardy, T., de Heide, R., & Grünwald, P. D. (2024). E-statistics, group invariance and anytime valid testing. The Annals of Statistics, 52(4), 1410-1432, http://dx.doi.org/10.1214/24-AOS2394.

Wang, H., & Ramdas, A. (in press). Anytime-valid t-tests and confidence sequences for Gaussian means with unknown variance. Sequential Analysis, https://doi.org/10.48550/arXiv.2310.03722.

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.

Pérez-Ortiz, M. F., Lardy, T., de Heide, R., & Grünwald, P. D. (2024). E-statistics, group invariance and anytime valid testing. The Annals of Statistics, 52(4), 1410-1432, http://dx.doi.org/10.1214/24-AOS2394.

Wang, H., & Ramdas, A. (in press). Anytime-valid t-tests and confidence sequences for Gaussian means with unknown variance. Sequential Analysis, https://doi.org/10.48550/arXiv.2310.03722.

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.

Pérez-Ortiz, M. F., Lardy, T., de Heide, R., & Grünwald, P. D. (2024). E-statistics, group invariance and anytime valid testing. The Annals of Statistics, 52(4), 1410-1432, http://dx.doi.org/10.1214/24-AOS2394.

Wang, H., & Ramdas, A. (in press). Anytime-valid t-tests and confidence sequences for Gaussian means with unknown variance. Sequential Analysis, https://doi.org/10.48550/arXiv.2310.03722.

Grünwald, P. D., de Heide, R., & Koolen, W. (2024). Safe testing. Journal of the Royal Statistical Society. Series B (Methodological), 86(5), 1091-1128. (With discussions), https://doi.org/10.1093/jrsssb/qkae011.

Ly, A, Boehm, Grünwald, P. D., Ramdas, A., & van Ravenzwaaij, D. (2024). Safe Anytime-Valid Inference: Practical maximally flexible sampling designs for experiments based on e-values. PsyArXiv Preprint, https://doi.org/10.31234/osf.io/h5vae.

Pérez-Ortiz, M. F., Lardy, T., de Heide, R., & Grünwald, P. D. (2024). E-statistics, group invariance and anytime valid testing. The Annals of Statistics, 52(4), 1410-1432, http://dx.doi.org/10.1214/24-AOS2394.

Wang, H., & Ramdas, A. (in press). Anytime-valid t-tests and confidence sequences for Gaussian means with unknown variance. Sequential Analysis, https://doi.org/10.48550/arXiv.2310.03722.

Examples

saviTTestStat(t=1, n1=100, parameter=0.4)
saviTTestStat(t=3, n1=100, parameter=0.3)
# evidence for the alternative over minimal efficacy
saviRelevanceTStatNEffNu(t=3, nEff=100, nu=60, parameter=0.4)
# evidence for minimal efficacy over the alternative
saviRelevanceTStatNEffNu(t=0.35, nEff=100, nu=60, parameter=0.4)

Helper function to demonstrate the selective continuation of paired z-tests

Description

Helper function to demonstrate the selective continuation of paired z-tests

Usage

selectivelyContinueZOrTTestData(
  designObj,
  deltaTrue = NULL,
  testName = c("Z-Test", "T-Test"),
  n1New,
  muGlobal,
  sigma,
  meanDiffTrue = NULL,
  nSim = 1000L,
  eValuesOld,
  eOverOld,
  trackCrossingOld,
  firstPassageTimeOld,
  eStoppedOld,
  seed = NULL
)

Arguments

Value

a list with the new e-values amongst other

Helper function to determine the lower and upper bound for the search space for standardised meanDiffMin

Description

Helper function to determine the lower and upper bound for the search space for standardised meanDiffMin

Usage

setLowAndHighEsTrueZ(
  nEff,
  eType = "mom",
  alternative = "twoSided",
  lowEsTrue = 1e-07,
  highEsTrue = 0.001
)

Arguments

Value

a list of low and high values for the parameter

Examples

setLowAndHighEsTrueZ(10)

Sets 'safestats' Plot Options and Returns the Current Plot Options.

Description

Sets 'safestats' Plot Options and Returns the Current Plot Options.

Usage

setSafeStatsPlotOptionsAndReturnOldOnes(...)

Arguments

Value

Returns a list with the user specified plot options.

Examples

oldPar <- setSafeStatsPlotOptionsAndReturnOldOnes()
graphics::plot(1:10, 1:10)
setPar <- graphics::par(oldPar)

Simulate the coverage of a savi confidence sequence for differences between proportions for a given distribution and savi design.

Description

Simulate the coverage of a savi confidence sequence for differences between proportions for a given distribution and savi design.

Usage

simulateCoverageDifferenceTwoProportions(
  successProbabilityA,
  trueDelta,
  saviDesign,
  precision = 100,
  M = 1000,
  numberForSeed = NA
)

Arguments

Value

the proportion of simulations where the trueDelta was included in the confidence sequence.

Examples

balancedSaviDesign <- designSaviTwoProportions(na = 1,
                                               nb = 1,
                                               nBlocksPlan = 20)
simulateCoverageDifferenceTwoProportions(successProbabilityA = 0.2,
                                         trueDelta = 0,
                                         saviDesign = balancedSaviDesign,
                                         M = 100,
                                         precision = 20,
                                         numberForSeed = 1082021)

Simulate incorrect optional stopping with fisher's exact test's p-value as the stopping rule.

Description

Simulate incorrect optional stopping with fisher's exact test's p-value as the stopping rule.

Usage

simulateIncorrectStoppingTimesFisher(
  thetaA,
  thetaB,
  alpha,
  na,
  nb,
  maxSimStoptime = 10000,
  M = 1000,
  numberForSeed = NULL
)

Arguments

Value

list with stopping times and rejection decisions.

Examples

simulateIncorrectStoppingTimesFisher(thetaA = 0.3,
                                     thetaB = 0.3,
                                     alpha = 0.05,
                                     na = 1,
                                     nb = 1,
                                     M = 10,
                                     maxSimStoptime = 100,
                                     numberForSeed = 251)

Simulate an optional stopping scenario according to a savi design for two proportions

Description

Simulate an optional stopping scenario according to a savi design for two proportions

Usage

simulateOptionalStoppingScenarioTwoProportions(saviDesign, M, thetaA, thetaB)

Arguments

Value

list with the simulation results of the savi test under optional stopping with the following components:

powerOptioStop

Proportion of sequences where H0 was rejected

nMean

Mean stopping time

probLessNDesign

Proportion of experiments stopped before nBlocksPlan was reached

lowN

Minimum stopping time

eValues

All achieved E values

allN

All stopping times

allSaviDecisions

Decisions on rejecting H0 for each M

allRejectedN

Stopping times of experiments where H0 was rejected

Examples

balancedSaviDesign <- designSaviTwoProportions(na = 1,
                                               nb = 1,
                                               nBlocksPlan = 30)
optionalStoppingSimulationResult <- simulateOptionalStoppingScenarioTwoProportions(
  saviDesign = balancedSaviDesign,
  M = 1e2,
  thetaA = 0.2,
  thetaB = 0.5
)

Compare Different Hyperparameter Settings for Savi Tests of Two Proportions.

Description

Simulates for a range of divergence parameter values (differences or log odds ratios) the worst-case stopping times (i.e., number of data blocks collected) and expected stopping times needed to achieve the desired power for each hyperparameter setting provided.

Usage

simulateTwoProportions(
  hyperparameterList,
  alternativeRestriction = c("none", "difference", "logOddsRatio"),
  deltaDesign = NULL,
  alpha,
  beta,
  na,
  nb,
  deltamax = 0.9,
  deltamin = 0.1,
  deltaGridSize = 8,
  M = 100,
  maxSimStoptime = 10000,
  thetaAgridSize = 8
)

Arguments

Value

Returns an object of class "savi2x2Sim". An object of class "savi2x2Sim" is a list containing at least the following components:

simData

A data frame containing simulation results with worst case and expected stopping times for each hyperparameter setting, for the specified or default range of effect sizes.

alpha

the significance threshold used in the simulations

beta

the type-II error control used in the simulations

deltaDesign

the value of restriction on the alternative hypothesis parameter space used for the E variables in the simulations

restriction

the type of restriction used for the E variables in the simulation

hyperparameters

list of the hyperparameters tested in the simulation

Examples

priorList1 <- list(betaA1 = 10, betaA2 = 1, betaB1 = 1, betaB2 = 10)
priorList2 <- list(betaA1 = 0.18, betaA2 = 0.18, betaB1 = 0.18, betaB2 = 0.18)
priorList3 <- list(betaA1 = 1, betaA2 = 1, betaB1 = 1, betaB2 = 1)

simResult <- simulateTwoProportions(
  hyperparameterList = list(priorList1, priorList2, priorList3),
  alternativeRestriction = "none",
  alpha = 0.1, beta = 0.2, na = 1, nb = 1,
  deltamax = -0.4, deltamin = -0.9, deltaGridSize = 3,
  M = 10
  )

print(simResult)
plot(simResult)