Chapter 3 Targets of Inference: Estimands
3.1 Estimands for Continuous and Binary Outcomes
3.1.1 Difference in Means
When an outcome is continuous or binary, one meaningful summary may be the mean of the outcome, and a meaningful comparison may be the difference in means estimand:
\[\theta_{DIM} = E[Y \vert A = 1] - E[Y \vert A = 0]\]
This estimand compares the mean outcome in the population of interest if all individuals received the treatment of interest to the the mean outcome in the population of interest if all individuals received the control or comparator intervention. Note that in binary outcomes, this is a difference in proportions (or risk difference) between the population where all individuals received the treatment of interest compared to receiving the control or comparator intervention.
3.2 Estimands for Ordinal Outcomes
Let \(Y\) be an ordinal outcome with \(k\) ordered categories. For each outcome category \(j \in \{1, \ldots, K\}\), the cumulative distribution function of \(Y\) given treatment \(A\) is denoted as:
\[Pr \left\{Y \le j\right\} = F(j \vert a)\]
The probability mass function of \(Y\) given treatment \(A\) is denoted as:
\[Pr \left\{Y = j\right\} = f(j \vert a) = F(j \vert a) - F(j-1 \vert a)\]
Although this notation involves numeric labels for levels, this is merely to simplify notation. Clarifications will be made as needed when distinguishing between outcomes with and without a numeric levels.
3.2.1 Difference in Mean Utility
If the levels of \(Y\) have numeric labels, and the mean value of this ordinal variable is meaningful, the difference in means estimand may still be meaningful and useful. Alternatively, if either the labels do not have a numeric interpretation, or the mean of these values is not particularly meaningful, it may be possible to create a meaningful numeric value by assigning ‘utilities’ or ‘weights’ to each level of the outcome. The quantitative and clinical meanings of the difference in means estimator will depend on the utilities assigned to the outcome scale. This allows the difference in means to be used, even if the levels of the outcome are not numeric (e.g. the Glasgow Outcome Scale, ranging from ‘Dead’, ‘Vegetative state’, ‘Severely disabled’, ‘Moderately disabled’, and ‘Good recovery’).
Let \(u(\cdot)\) denote a pre-specified mapping from the outcome labels to utility values:
\[ u(Y)= \begin{cases} u_{1} := \text{utility of } Y = 1\\ u_{2} := \text{utility of } Y = 2\\ \vdots \\ u_{k} := \text{utility of } Y = k \end{cases} \]
The utilities will usually be monotone increasing, such that each succesive level of the outcome is associated with equal or better utility. Alternatively, if lower values of the outcome are preferable, utilities will usually be monotone decreasing.
Once the utilities have been defined, the estimand is defined as:
\[\theta_{DIM} = E[u(Y) \vert A=1] - E[u(Y) \vert A=0] = \sum_{i=1}^{k}u(j)\left(f( j \vert 1) - f(j \vert 0)\right)\]
When all outcomes at or above a threshold \(t \in \{2, \ldots, k\}\) are given a utility of 1, and all others are given a utility of 0, this collapses the ordinal outcome into a binary one. The resulting estimand is the risk difference estimator of the outcome being at or above \(t\):
\[ u(Y)= \begin{cases} 1: & Y \geq t \\ 0: & Y < t \end{cases} \]
While a risk difference may be more familiar to implement and conceptually easier to interpret, it treats all outcome states either below or above the threshold identically, ignoring potential information in such outcome states.
3.2.2 Mann-Whitney (M-W) Estimand
The Mann-Whitney estimand gives the probability that a randomly-selected person assigned to treatment of interest will have an outcome on the same level or a higher level than a randomly-selected person assigned to the comparator group, with ties broken at random:
\[\theta_{MW} = P(\tilde{Y} > Y \vert \tilde{A} = 1, A = 0) + \frac{1}{2}P(\tilde{Y} = Y \vert \tilde{A} = 1, A = 0) = \\ \sum_{j=1}^{K} \left\{ F(j-1 \vert 0) + \frac{1}{2} f(j \vert 0) \right\} f(j \vert 1)\]
If there is no difference in treatments, we would expect a randomly selected individual from one group to have a higher outcome than a randomly selected individual from the other group about half the time: the null value for this estimand is \(1/2\).
Note that if higher numerical values indicate worse outcomes, the outcome scale can be reversed prior to analysis, so that the estimand can be interpreted as the probability that a randomly-selected person assigned to treatment of interest will have an outcome as good or better than a randomly-selected person assigned to the comparator group.
This estimand addresses a common concern of those choosing between treatment options, and may be easier to communicate to a lay audience.
3.2.3 Log Odds Ratio (LOR)
In the case of a binary outcome, the odds ratio of a “good” outcome (\(Y=1\)) is \(OR = odds(Y = 1 \vert A = 1)/odds(Y = 1 \vert A = 0)\): a value greater than 1 indicates a greater likelihood of a “good” outcome in the treatment of interest relative to the comparator group, and the log of the odds ratio will be positive.
In the case of an ordinal outcome with categories \(1, \ldots, K\), these categories can be collapsed into \((K-1)\) binary outcomes: \(Y \le j\) for \(j \in \{1, \ldots, (K-1) \}\). The odds ratio at threshold \(j\) compares the odds of falling at or below level \(j\) between the treatment of interest and the comparator group:
\[OR_{j} = \frac{odds(Y \le j \vert A = 1)}{odds(Y \le j \vert A = 0)}\]
When this odds ratio is greater than 1, individuals assigned to the treatment of interest are more likely to have outcomes at or below level \(j\) than those in the comparator group: the log of this odds ratio will be positive. The log odds ratio estimand combines information across the levels of an ordinal outcome by averaging the log odds of an outcome at or below each threshold across all thresholds of the outcome:
\[\theta_{LOR} = \frac{1}{K-1} \sum_{j=1}^{K-1} log \left( \frac{odds(Y \le j \vert A = 1)}{odds(Y \le j \vert A = 0)} \right) = \frac{1}{K-1} \sum_{j=1}^{K-1} log \left( \frac{F(j \vert 1)/ \left( 1 - F(j \vert 1) \right) } {F(j \vert 0)/ \left( 1 - F(j \vert 0) \right) } \right)\]
This estimand is related to the proportional odds logistic regression model, a common parametric model for analyzing ordinal outcomes. In the proportional odds model, a regression coefficient for treatment group gives the increase in the odds of being at or below a given level of the outcome associated with a unit increase in that variable holding all else constant:
\[log(odds(Y \le j \vert A)) = logit \left(P(Y \le j \vert A) \right) = \alpha_{j} + \beta A: \quad j \in \{1, \ldots, (K-1)\}\]
A positive slope indicates greater likelihood of lower scores in those assigned to receive the treatment of interest relative to the comparator group. The proportional odds assumption involves assuming that the treatment has the same effect across each binary threshold (i.e. that \(\beta\) does not vary across the \(K-1\) thresholds). When this assumption holds, the log odds ratio estimand is the same as the coefficient in the proportional odds model, but importantly, the validity of the LOR estimand does not depend on this assumption. As in binary and ordinal logistic regression, the null value for this estimand is 0.
Since \(-log(a/b) = log(b/a)\) and \(odds(Y > j \vert A = 1) = 1/odds(Y \le j \vert A = 1)\), changing the sign of the log odds ratio estimator tells us about the average log odds of having scores higher than level \(j\) in the treatment of interest relative to the comparator group:
\[-\theta_{LOR} = \frac{1}{K-1} \sum_{j=1}^{K-1} log \left( \frac{odds(Y > j \vert A = 1)}{odds(Y > j \vert A = 0)} \right)\]
3.3 Survival/Time-To-Event Outcomes
When the outcome is the time from randomization until an event of interest occurs, let \(Y\) denote the time at which the event occurs, and \(C\) denote the time at which individuals would be censored. For each individual, we observe \(\delta_{i} = I_{\{Y_{i} \le C_{i}\}}\), whether an individual is censored or the event is observed, and \(T_{i} = min\{Y_{i}, C_{i}\}\), the time at which the event or censoring occurs. Often we are interested in a pre-specified time window from randomization to a scientifically relevant point in time, known as the time horizon. Let \(\tau\) denote the time horizon of interest for inference.
3.3.1 Survival Function
When the outcome is a time-to-event, the usual target of inference is the survival function, which is the marginal probability of being event-free through time \(t\) if the entire population were assigned to study arm \(A = a\):
\[S_{0}^{(a)}(t) = Pr\{Y > t \vert A = a\}\]
Estimands of interest may include the difference in survival probability up to the time horizon \(\tau\):
\[\theta_{DSP} = Pr(Y \ge \tau \vert A = 1) - Pr(Y \ge \tau \vert A = 0)\] Instead of an additive estimand, a relative estimand can be obtained by taking the ratio of survival probabilities at the time horizon \(\tau\):
\[\theta_{RSP} = \frac{Pr(Y \ge \tau \vert A = 1)}{Pr(Y \ge \tau \vert A = 0)}\]
3.3.2 Hazard Ratio
The hazard rate for individuals receiving treatment \(A = a\), denoted \(h^{(a)}(t)\), is the instantaneous rate of the event of interest at time \(t\) among individuals who have not yet experienced the event:
\[ h^{(a)}(t) = \lim_{ \Delta t \to 0} \frac{ P(t \le Y < t + \Delta t \; \vert \; Y \ge t, A = a) }{\Delta t} = \frac{d}{dt}ln\left(S_{0}^{(a)}(t)\right)\]
The hazard ratio compares the ratio of two hazard functions:
\[\theta_{HR}(t) = \frac{h^{(1)}(t)}{h^{(0)}(t)}\]
Since both hazard functions can vary over time, the true hazard ratio can vary over time. Commonly used approaches in time-to-event analysis often require the hazard rate being approximately constant over the time interval of interest, either unconditionally or conditional on covariates. It may not be known a priori in practice whether such an assumption is reasonable, but this assumption can be empirically assessed.
When the hazard ratio varies appreciably in time, methods that make a conditional or unconditional assumption of proportional hazards are less efficient and give effect estimates whose interpretation is unclear. Even when the assumption of proportional hazards is approximately true, the hazard ratio cannot easily be translated into an easily communicated metric, such as the number of years an individual can expect to be free of the event if they were assigned to one treatment or another.
3.3.3 Restricted Mean Survival Time
Another estimand that may be of interest is the restricted mean survival time (RMST). This estimand is the average time-to-event from randomization to the time horizon \(\tau\) (e.g. life expectancy up to \(\tau\) when mortality is the event of interest). This is given by taking the area under the survival function from randomization to the time horizon:
\[RMST = E[min\{ Y, \tau \} \vert a] = \int_{0}^{\tau} S_{0}^{(a)}(t) dt\]
Treatments can be compared using a contrast of the RMST in the population where everyone receives treatment and that same population where everyone receives the control/comparator intervention. The difference in RMST contrast assesses the area between the survival curves under each treatment scenario.
\[\theta_{DRMST} = E[min\{ Y, \tau \} \vert A = 1] - E[min\{ Y, \tau \} \vert A = 0]\] A relative estimand is given by the ratio of RMST:
\[\theta_{RRMST} = \frac{E[min\{ Y, \tau \} \vert A = 1]}{E[min\{ Y, \tau \} \vert A = 0]}\]