10  Conditional power and B-values

In some cases, rather than working with \(Z_{1}, Z_{2}, \ldots, Z_{k}\) as in Section 2.3, it is desirable to consider variables representing incremental sets of observations between analyses. We consider a discrete version of the Brownian motion formulation of Proschan, Lan and Wittes (Proschan et al. 2006) and apply the same principles to demonstrate how to compute conditional power and conditional error using the calculation tools we have been applying for group sequential designs. Those who prefer to skip the background materials can proceed directly to Section 10.4.1.

10.1 Z-values, B-values and S-values

We let \(X_{1}\), \(X_{2},\ldots\) be independent and identically distributed normal random variables with mean \(\delta\) and variance \(\sigma^2\), as before. We let \(0<n_1<n_2<\ldots<n_k\) for some \(k>1\). For \(i=1,\) \(2,\ldots, k\) define \(\mathcal{I}_i=n_i/\sigma^2\) and \(t_i=n_i/n_k=\mathcal{I}_i/\mathcal{I}_k\). We first define cumulative S-values and treatment effect estimates for \(i=1,2,\ldots,k\):

\[ \begin{array}{rclcl} S_i&=&\sum_{m=1}^{n_i}X_m&\sim&N(n_i\delta,n_i\sigma^2),\\ \\ \hat{\delta}_i&=&S_i/n_i&\sim&N(\delta,\sigma^2/n_i). \end{array} \]

Letting \(\theta=\delta/\sigma\), we define Z-values and B-values and write the cumulative quantities in terms of the standardized parameter \(\theta\):

\[ \begin{array}{rcl} \hat{\theta}_i&=&\hat{\delta}_i/\sigma\sim N(\theta,1/n_i),\\ \\ Z_{i}&=&\sqrt{n_i}\hat{\theta}_i\sim N(\sqrt{n_i}\theta, 1),\\ \\ B_{i}&=&\sqrt{t_i}Z_i\sim N(\sqrt{n_k} t_i\theta, t_i). \end{array} \]

Thus, \(B_i=\sqrt{t_i}Z_i\) for \(1\le i\le k\). These forms for \(Z_i\) and \(B_i\) apply to trials with many types of endpoints where the treatment effect estimator and information take different forms. Scharfstein, Tsiatis and Robbins (Scharfstein et al. 1997) have generalized work from many others by showing that if \(Z_i\) is an asymptotically efficient test for a single parameter in a parametric or semi-parametric model then the asymptotic theory holds. Proschan, Lan and Wittes (Proschan et al. 2006) provide extensive coverage of application of the Brownian motion version of B-values used here to group sequential trials with several different types of endpoints.

We now re-write bounds and boundary crossing probabilities in terms of B-values. For \(i=1,2,\ldots,k\) we define \(b_i=u_i\sqrt{t_i}\), \(a_i=l_i\sqrt{t_i}\) and note that

\[ \begin{array}{rcl} \alpha_i(\theta)&=&P_{\theta}\{\{Z_i \geq u_i\} \cap_{j=1}^{i-1}\{l_j\leqslant Z_j<u_j\}\}\\ \\ &=&P_{\theta}\{\{B_i \geq b_i\} \cap_{j=1}^{i-1}\{a_j\leqslant B_j<b_j\}\}.\\ \end{array} \tag{10.1}\]

Formulas for \(\beta_i(\theta)\) and \(\alpha_i^{+}(\theta)\) can be rewritten in an analogous fashion:

\[\begin{align} \alpha_i^{+}(\theta) & = & P_{\theta}\{\{B_i \geq b_i\} \cap_{j=1}^{i-1}\{B_j<b_j\}\}\\ \beta_i(\theta) & = & P_{\theta}\{\{B_i \leqslant a_i\} \cap_{j=1}^{i-1}\{a_j\leqslant B_j<b_j\}\} \end{align}\]

10.2 Incremental formulation

We now formulate a group sequential set of outcomes in terms of independent increments to simplify conditional power calculations. Define \(n_0=S_0=0\) and for \(0\leq i < j\leq k\), \(n_{i,j}=n_j-n_i\), \(t_{i,j}=n_{i,j}/n_k\), and \(\mathcal{I}_{i,j}=n_{i,j}/\sigma^2\).

\[ \begin{array}{rclclcl} S_{i,j}&=&\sum_{m=n_i+1}^{n_j}X_m&\sim& N(n_{i,j}\delta,n_{i,j}\sigma^2),\\ \\ Z_{i,j}&=&S_{i,j}/(\sqrt{n_{i,j}}\sigma)&\sim& N(\sqrt{n_{i,j}}\theta,1)&\sim&N(\sqrt{\mathcal{I}_{i,j}}\delta,1), \\ \\ B_{i,j}&=&S_{i,j}/(\sqrt{n_k}\sigma)&\sim& N(\sqrt{n_k}t_{i,j}\theta,t_{i,j})&\sim&N(\sqrt{\mathcal{I}_k}t_{i,j}\delta,t_{i,j}). \end{array} \]

Analogous to before, \(B_{i,j}=\sqrt{t_{i,j}}Z_{i,j}\) and we can apply the ultimate conditional power and conditional error calculations below broadly to group sequential designs for which the general asymptotic theory is applicable. For \(0\le i < j\leq k\) we note that \(B_i=B_{0,i}\) and that \(B_j=B_i+B_{i,j}\) where \(B_i\) and \(B_{i,j}\) are independent and normally distributed. Thus, the conditional distribution of \(B_j\) assuming \(B_i=c\) for some \(1\leq i<j\leqslant k\) and constant \(c\) is normal with mean \(c+\sqrt{n_k}t_{i,j}\theta\) and variance \(t_{i,j}\).

10.3 Simple conditional power and conditional error

For a fixed assumed value of \(\theta\), conditional power is the probability of crossing an upper boundary given the observed interim result. The corresponding probability under \(\theta=0\) is the conditional error. Proschan, Lan and Wittes (Proschan et al. 2006) and others compute a simple version of these probabilities that ignores stopping boundaries between the interim analysis being conditioned on and a future analysis of interest. That is, assuming \(1\leq i < j\leq k\) and \(B_i=c\), simple conditional power is

\[ \begin{split} P\{B_j\geq b_j|B_i=c\}&=P\{B_{i,j}\geq b_j-c\}\\ &=1-\Phi\left(\frac{b_j-c-\theta t_{i,j}\sqrt{n_k}}{\sqrt{t_{i,j}}}\right)\\ &=1-\Phi\left(\frac{b_j-c-\delta t_{i,j}\sqrt{\mathcal{I}_k}}{\sqrt{t_{i,j}}} \right) \end{split} \tag{10.2}\]

10.3.1 Simple conditional power for the CAPTURE trial

We consider the interim data displayed in Chapter 9, but initially restrict ourselves to the design with two interim analyses. We reconstruct the test statistics in z and the design in CAPTURE, then apply Equation 10.2 to compute the simple conditional power and conditional error of crossing each future bound at the time of the first interim analysis. For each analysis, we place the information fractions (fractions of final sample size in this case) in ti and the B-value bounds for the design in b. We use the standardized parameter \(\theta\) formulation from equation Equation 10.2. The observed B-value at the first analysis is placed in bobs. We then store the effect size powered for in theta1 and the observed effect size at the first analysis, \(\hat{\theta}_1=Z_1/\sqrt{n_1}\), in thetahat. Finally, for \(j=2,3\), we store the variance \(t_j-t_1\) of \(B_{1,j}\) in t1j and its standard deviation in sd1j.

As explained in Chapter 9, the repeated calls to gsDesign() update the maximum sample size until the analysis times converge to approximately 350 and 700 patients.

library(gsDesign)

n1 <- c(175, 353, 532, 635)
n2 <- c(175, 347, 518, 630)
x1 <- c(30, 55, 84, 101)
x2 <- c(14, 37, 55, 71)
z <- testBinomial(x1 = x1, x2 = x2, n1 = n1, n2 = n2)

n.fix <- nBinomial(p1 = 0.15, p2 = 0.1, beta = 0.2)
x <- gsDesign(
  k = 3,
  n.fix = n.fix,
  beta = 0.2,
  sfupar = -3
)
x <- gsDesign(
  k = 3,
  n.fix = n.fix,
  timing = c(350, 700) / x$n.I[3],
  beta = 0.2,
  sfupar = -3
)
x <- gsDesign(
  k = 3,
  n.fix = n.fix,
  timing = c(350, 700) / x$n.I[3],
  beta = 0.2,
  sfupar = -3
)
CAPTURE <- gsDesign(
  k = 3,
  n.fix = n.fix,
  timing = c(350, 700) / x$n.I[3],
  beta = 0.2,
  sfupar = -3,
  endpoint = "binomial",
  delta1 = 0.05
)

n <- (n1 + n2)[1:2]
ti <- c(n / CAPTURE$n.I[3], 1)
b <- CAPTURE$upper$bound * sqrt(ti)
bobs <- z[1] * sqrt(ti[1])
theta1 <- CAPTURE$delta
thetahat <- z[1] / sqrt(n[1])
t1j <- ti[2:3] - ti[1]
sd1j <- sqrt(t1j)
b1j <- b[2:3] - bobs

We calculate the simple conditional error and conditional power after the first interim analysis under the originally powered and observed effect sizes.

simpCError <- pnorm(
  b1j,
  sd = sd1j,
  mean = 0,
  lower = FALSE
)
simpCPwr1 <- pnorm(
  b1j,
  sd = sd1j,
  mean = theta1 * sqrt(CAPTURE$n.I[3]) * t1j,
  lower = FALSE
)
simpCPwrHat <- pnorm(
  b1j,
  sd = sd1j,
  mean = thetahat * sqrt(CAPTURE$n.I[3]) * t1j,
  lower = FALSE
)

tibble(
  `Future analysis` = c("Second interim", "Final"),
  `Conditional error: theta = 0` = simpCError,
  `Conditional power: theta = delta` = simpCPwr1,
  `Conditional power: theta = theta-hat[1]` = simpCPwrHat
) |>
  gt() |>
  fmt_number(columns = 2:4, decimals = 3) |>
  tab_header(title = "Simple conditional boundary-crossing probabilities") |>
  tab_options(data_row.padding = px(1))

Simple conditional boundary-crossing probabilities
Future analysis	Conditional error: theta = 0	Conditional power: theta = delta	Conditional power: theta = theta-hat[1]
Second interim	0.103	0.560	0.906
Final	0.200	0.952	1.000

10.4 Conditional power and conditional error

As an alternative to futility bounds based on \(\beta\)-spending, stopping rules for futility are sometimes based on the conditional power of a positive trial given the value of a test statistic at an interim analysis. Thus, we consider the conditional probabilities of boundary crossing for a group sequential design given an interim result. Unlike the simple calculations above, these probabilities account for first crossing at every intervening analysis. Assume \(1\le i < j \leqslant k\) and \(B_i=c\). The conditional probability of first crossing an upper boundary at analysis \(j\) given \(B_i=c\) is

\[\begin{align} \alpha _{i,j}(\theta |B_i=c) & = & P_{\theta }\{\{B_{j}\geqslant b_j\}\cap_{m=i+1}^{j-1}\{a_m\leqslant B_{0,m}<b_m\}|B_{0,i}=c\}\nonumber\\ & = & P_{\theta }\{\{B_{i,j}\geqslant b_j-c\}\cap_{m=i+1}^{j-1}\{a_m-c\leqslant B_{i,m}<b_m-c\}\} \end{align}\]

Formulas for \(\alpha^{+}_{i,j}(\theta |B_i=c)\) and \(\beta_{i,j}(\theta |c)\) can be written analogously:

\[\begin{align} \alpha^{+}_{i,j}(\theta |B_i=c) & = & P_{\theta }\{\{B_{i,j}\geqslant b_j-c\}\cap_{m=i+1}^{j-1}\{ B_{i,m}<b_m-c\}\}\\ \beta _{i,j}(\theta |B_i=c) & = & P_{\theta }\{\{B_{i,j}\leqslant a_j-c\}\cap_{m=i+1}^{j-1}\{a_m-c\leqslant B_{i,m}<b_m-c\}\} \end{align}\]

Given the above and Equation 10.1, conditional boundary crossing probabilities can be computed using the same numerical tools as unconditional boundary crossing probabilities. The conditional design generated is that of Muller and Schafer (Müller and Schäfer 2001). We rewrite the incremental \(B\)-value bounds as incremental \(Z\)-value bounds as these are values produced in the function gsCP() used below. Recall that the incremental \(Z\)-value is denoted \(Z_{i,j}=B_{i,j}/\sqrt{t_{i,j}}\) for \(0\leq i<j\leq k\). If the interim B-value at analysis \(i\) is \(c\) as above, we define \(z_i=c/\sqrt{t_i}\). Then, crossing an upper bound with \(B_{i,j}\geq b_j-c\) later at analysis \(j>i\) is equivalent to

\[ Z_{i,j}=\frac{B_{i,j}}{\sqrt{t_{i,j}}} \geq \frac{b_j-c}{\sqrt{t_{i,j}}}= \frac{u_j\sqrt{t_j}-z_i\sqrt{t_i}}{\sqrt{t_{i,j}}}. \]

The incremental \(Z\)-value upper and lower bounds are defined respectively by

\[ u_{i,j}(z_{i})=\frac{u_{j}\sqrt{t_{j}}-z_{i}\sqrt{t_{i}}}{\sqrt{t_{i,j}}} \tag{10.3}\]

and

\[ l_{i,j}(z_{i})=\frac{l_{j}\sqrt{t_{j}}-z_{i}\sqrt{t_{i}}}{\sqrt{t_{i,j}}}. \tag{10.4}\]

10.4.1 Application to the CAPTURE trial

We now examine the observations after the initial interim that would be required to cross the bounds for the original design. The function gsCP() computes a new set of boundaries \(l_{i,j}\) and \(u_{i,j}\) from Equation 10.3 and Equation 10.4 based on results at interim \(i\). Values of \(\theta\) for which conditional power is computed are, by default,

• \(\theta = 0\), for conditional error.
• \(\theta = \hat{\theta}\) to compute conditional power at the interim trend as in, for example, Bauer and Köhne (Bauer and Kohne 1994), Proschan and Hunsberger (Proschan and Hunsberger 1995) and Cui, Hung and Wang (Cui et al. 1999).
• \(\theta = \delta\), the treatment effect at which the trial was originally powered, as in Liu and Chi (Liu and Chi 2001).

xCP1 <- gsCP(x = CAPTURE, i = 1, zi = z[1])

We summarize the probability of crossing an upper boundary after the first interim analysis under the observed interim effect, the null effect, and the effect for which the trial was originally powered.

conditional_summary <- tibble(
  Assumption = c(
    "Observed interim effect",
    "Null effect (conditional error)",
    "Originally powered effect"
  ),
  `Probability of upper-bound crossing` = colSums(xCP1$upper$prob)
)

conditional_summary |>
  gt() |>
  fmt_number(columns = 2, decimals = 3) |>
  tab_header(title = "Conditional probability of trial success") |>
  tab_options(data_row.padding = px(1))

Conditional probability of trial success
Assumption	Probability of upper-bound crossing
Observed interim effect	1.000
Null effect (conditional error)	0.245
Originally powered effect	0.958

The probability of crossing at the second interim analysis given the first agrees with the simple calculation in Section 10.3 under, respectively, the observed interim estimate \(\theta=\hat{\theta}_1\), the effect \(\theta=\delta\) for which the trial was originally powered, and the null value \(\theta=0\).

tibble(
  Assumption = c(
    "Observed interim effect",
    "Originally powered effect",
    "Null effect"
  ),
  `gsCP()` = xCP1$upper$prob[1, c(1, 3, 2)],
  `Simple calculation` = c(
    simpCPwrHat[1],
    simpCPwr1[1],
    simpCError[1]
  )
) |>
  gt() |>
  fmt_number(columns = 2:3, decimals = 3) |>
  tab_header(title = "Probability of crossing at the second analysis") |>
  tab_options(data_row.padding = px(1))

Probability of crossing at the second analysis
Assumption	gsCP()	Simple calculation
Observed interim effect	0.906	0.906
Originally powered effect	0.560	0.560
Null effect	0.103	0.103

As you can see, the conditional probability of success varies considerably depending on the value of \(\theta\). We will see later that there is substantial uncertainty concerning \(\theta\) at the first interim, thus calling into question use of any particular \(\theta\) value to compute conditional power.

Bauer, Peter, and F. Kohne. 1994. “Evaluation of Experiments with Adaptive Interim Analyses.” Biometrics 50 (4): 1029–41.

Cui, Lu, HM James Hung, and Sue-Jane Wang. 1999. “Modifications of Sample Size in Group Sequential Clinical Trials.” Biometrics 55 (3): 853–57.

Liu, Qing, and George YH Chi. 2001. “On Sample Size and Inference for Two-Stage Adaptive Designs.” Biometrics 57 (1): 172–77.

Müller, Hans-Helge, and Helmut Schäfer. 2001. “Adaptive Group Sequential Designs for Clinical Trials: Combining the Advantages of Adaptive and of Classical Group Sequential Approaches.” Biometrics 57 (3): 886–91.

Proschan, Michael A, and Sally A Hunsberger. 1995. “Designed Extension of Studies Based on Conditional Power.” Biometrics 51 (4): 1315–24.

Proschan, Michael A., K K Gordon Lan, and Janet Turk Wittes. 2006. Statistical Monitoring of Clinical Trials. A Unified Approach. Springer.

Scharfstein, Daniel O, Anastasios A Tsiatis, and James M Robins. 1997. “Semiparametric Efficiency and Its Implication on the Design and Analysis of Group-Sequential Studies.” Journal of the American Statistical Association 92 (440): 1342–50.