Randomization why

In double-blind studies with properly implemented allocation concealment the risk of selection bias is low. By contrast, in open-label studies the risk of selection bias may be high, and the randomization design should provide strong encryption of the randomization sequence to minimize prediction of future allocations. Number of study centers. Many modern RCTs are implemented globally at multiple research institutions, whereas some studies are conducted at a single institution.

In the latter case, especially in single-institution open-label studies, the randomization design should be chosen very carefully, to mitigate the risk of selection bias.

An important point to consider is calibration of the design parameters. By fine-tuning these parameters, one can obtain designs with desirable statistical properties. For instance, references [ 80 , 81 ] provide guidance on how to justify the block size in the PBD to mitigate the risk of selection bias or chronological bias. The calibration of design parameters can be done using Monte Carlo simulations for the given trial setting.

Another important consideration is the scope of randomization procedures to be evaluated. This should be done judiciously, on a case-by-case basis, focusing only on the most reasonable procedures. References [ 50 , 58 , 60 ] provide good examples of simulation studies to facilitate comparisons among various restricted randomization procedures for a RCT. In parallel with the decision on the scope of randomization procedures to be assessed, one should decide upon the performance criteria against which these designs will be compared.

Among others, one might think about the two competing considerations: treatment balance and allocation randomness.

These measures can be either calculated analytically when formulae are available or through Monte Carlo simulations. It is also helpful to visualize the selected criteria. Visualizations can be done in a number of ways; e. Such visualizations can help evaluate design characteristics, both overall and at intermediate allocation steps. Another way to compare the merits of different randomization procedures is to study their inferential characteristics such as type I error rate and power under different experimental conditions.

Sometimes this can be done analytically, but a more practical approach is to use Monte Carlo simulation. The choice of the modeling and analysis strategy will be context-specific. Here we outline some considerations that may be useful for this purpose:. Data generating mechanism. To simulate individual outcome data, some plausible statistical model must be posited. The form of the model will depend on the type of outcomes e.

True treatment effects. Randomization designs to be compared. The choice of candidate randomization designs and their parameters must be made judiciously. Data analytic strategy. For any study design, one should pre-specify the data analysis strategy to address the primary research question.

Statistical tests of significance to compare treatment effects may be parametric or nonparametric, with or without adjustment for covariates. The approach to statistical inference: population model-based or randomization-based. These two approaches are expected to yield similar results when the population model assumptions are met, but they may be different if some assumptions are violated. Randomization-based tests following restricted randomization procedures will control the type I error at the chosen level if the distribution of the test statistic under the null hypothesis is fully specified by the randomization procedure that was used for patient allocation.

This is always the case unless there is a major flaw in the design such as selection bias whereby the outcome of any individual participant is dependent on treatment assignments of the previous participants. Overall, there should be a well-thought plan capturing the key questions to be answered, the strategy to address them, the choice of statistical software for simulation and visualization of the results, and other relevant details.

In this section we present four examples that illustrate how one may approach evaluation of different randomization design options at the study planning stage. These 12 procedures can be grouped into five major types. I Procedures 1, 2, 3, and 4 achieve exact final balance for a chosen sample size provided the total sample size is a multiple of the block size. III Procedures 7 and 8 are biased coin designs that sequentially adjust randomization according to imbalance measured as the difference in treatment numbers.

V Procedure 12 CRD is the most random procedure that achieves balance for large samples. We first compare the procedures with respect to treatment balance and allocation randomness. At the other extreme, we have PBD 2 for which every odd allocation is made with probability 0.

Different randomization procedures can be compared graphically. Figure 1 is a plot of expected absolute imbalance vs. Simulated expected absolute imbalance vs. Figure 2 is a plot of expected proportion of correct guesses vs.

One can observe that for CRD it is a flat pattern at 0. Rand exhibits an increasing pattern with overall fewer correct guesses compared to other randomization procedures. For the three GBCD procedures, there is a rapid initial increase followed by gradual decrease in the pattern; this makes good sense, because GBCD procedures force greater balance when the trial is small and become more random and less prone to correct guessing as the sample size increases.

Simulated expected proportion of correct guesses vs. The other ten designs are closer to 0,0. Simulated forcing index x-axis vs.

BSD 3 seems to provide overall best tradeoff between randomness and balance throughout the study. The procedures are ordered by value of d 50 , with smaller values more red indicating more optimal performance.

Our next goal is to compare the chosen randomization procedures in terms of validity control of the type I error rate and efficiency power. We shall explore the following four models:. This corresponds to a standard setup for a two-sample t-test under a population model. In this model, the outcomes are affected by a linear trend over time [ 67 ].

In this setup, we have a misspecification of the distribution of measurement errors. In this setup, at each allocation step the investigator attempts to intelligently guess the upcoming treatment assignment and selectively enroll a patient who, in their view, would be most suitable for the upcoming treatment.

T3: Randomization-based test based on ranks : This test procedure follows the same logic as T2, except that the test statistic is calculated based on ranks.

Figure 5 summarizes the results of a simulation study comparing 12 randomization designs, under 4 models for the outcome M1, M2, M3, and M4 , 4 scenarios for the mean treatment difference Null, and Alternatives 1, 2, and 3 , using 3 statistical tests T1, T2, and T3.

The operating characteristics of interest are the type I error rate under the Null scenario and the power under the Alternative scenarios. Simulated type I error rate and power of 12 restricted randomization procedures. Four scenarios for the treatment mean difference Null; Alternatives 1, 2, and 3.

Three statistical tests T1: two-sample t-test; T2: randomization-based test using mean difference; T3: randomization-based test using ranks. From Fig. In other words, when population model assumptions are satisfied, any combination of design and analysis should work well and yield reliable and consistent results. These results are consistent with some previous findings in the literature [ 67 , 68 ].

As regards power, it is reduced significantly compared to the normal random sampling scenario. The t-test seems to be most affected and the randomization-based test using ranks is most robust for a majority of the designs.

Remarkably, for CRD the power is similar with all three tests. This signifies the usefulness of randomization-based inference in situations when outcome data are subject to a linear time trend, and the importance of applying randomization-based tests at least as supplemental analyses to likelihood-based test procedures.

As regards power, all designs also have similar, consistently degraded performance: the t-test is least powerful, and the randomization-based test using ranks has highest power. Overall, under misspecification of the error distribution a randomization-based test using ranks is most appropriate; yet one should acknowledge that its power is still lower than expected.

For eleven other procedures, inflations of the type I error were observed. In general, the more random the design, the less it was affected by selection bias. These results are consistent with the theory of Blackwell and Hodges [ 28 ] which posits that TBD is least susceptible to selection bias within a class of restricted randomization designs that force exact balance.

Finally, under M4, statistical power is inflated by several percentage points compared to the normal random sampling scenario without selection bias.

The magnitude of the type I error inflation is different across the restricted randomization designs; e. For the chosen experimental scenarios, we evaluated CRD and several restricted randomization procedures, some of which belonged to the same class but with different values of the parameter e. Based on these criteria, we found that BSD 3 provides overall best performance. We also evaluated type I error and power of selected randomization procedures under several treatment response models.

We have observed important links between balance, randomness, type I error rate and power. It is beneficial to consider all these criteria simultaneously as they may complement each other in characterizing statistical properties of randomization designs. In particular, we found that a design that lacks randomness, such as PBD with blocks of 2 or 4, may be vulnerable to selection bias and lead to inflations of the type I error.

Therefore, these designs should be avoided, especially in open-label studies. As regards statistical power, since all designs in this example targeted allocation ratio which is optimal if the outcomes are normally distributed and have between-group constant variance , they had very similar power of statistical tests in most scenarios except for the one with chronological bias.

In the latter case, randomization-based tests were more robust and more powerful than the standard two-sample t-test under the population model assumption. Overall, while Example 1 is based on a hypothetical RCT, its true purpose is to showcase the thinking process in the application of our general roadmap.

The following three examples are considered in the context of real RCTs. Selection bias can arise if the investigator can intelligently guess at least part of the randomization sequence yet to be allocated and, on that basis, preferentially and strategically assigns study subjects to treatments.

Although it is generally not possible to prove that a particular study has been infected with selection bias, there are examples of published RCTs that do show some evidence to have been affected by it. Suspect trials are, for example, those with strong observed baseline covariate imbalances that consistently favor the active treatment group [ 16 ]. In what follows we describe an example of an RCT where the stratified block randomization procedure used was vulnerable to potential selection biases, and discuss potential alternatives that may reduce this vulnerability.

Etanercept was studied in patients aged 4 to 17 years with polyarticular juvenile rheumatoid arthritis [ 85 ]. The trial consisted of two parts. During the first, open-label part of the trial, patients received etanercept twice weekly for up to three months. Responders from this initial part of the trial were then randomized, at a ratio, in the second, double-blind, placebo-controlled part of the trial to receive etanercept or placebo for four months or until a flare of the disease occurred.

The primary efficacy outcome, the proportion of patients with disease flare, was evaluated in the double-blind part. Regulatory review by the Food and Drug Administrative FDA identified vulnerability to selection biases in the study design of the double-blind part and potential issues in study conduct. These findings were succinctly summarized in [ 16 ] pp. While this appears to be an attempt to improve treatment balance in this small trial, unblinding of one treatment assignment may lead to deterministic predictability of three upcoming assignments.

While the double-blind nature of the trial alleviated this concern to some extent, it should be noted that all patients did receive etanercept previously in the initial open-label part of the trial. Chances of unblinding may not be ignorable if etanercept and placebo have immediately evident different effects or side effects. The randomized withdrawal design was appropriate in this context to improve statistical power in identifying efficacious treatments, but the specific randomization procedure used in the trial increased vulnerability to selection biases if blinding cannot be completely maintained.

There were also some patients randomized out of order. Imbalance in baseline characteristics were observed in age mean ages of 8. To illustrate the latter point, let us compare predictability of two randomization procedures — permuted block design PBD and big stick design BSD for several values of the maximum tolerated imbalance MTI.

Table 3 reports two metrics for PBD and BSD: proportion of deterministic assignments within a randomization sequence, and excess correct guess probability. However, by increasing MTI, one can substantially decrease predictability. In addition to simplicity and lower predictability for the same level of MTI control, BSD has another important advantage: investigators are not accustomed to it as they are to the PBD , and therefore it has potential for complete elimination of prediction through thwarting enough early prediction attempts.

MTI randomization procedures can be also used as building elements for more complex stratified randomization schemes [ 86 ].

Chronological bias may occur if a trial recruitment period is long, and there is a drift in some covariate over time that is subsequently not accounted for in the analysis [ 29 ]. To mitigate risk of chronological bias, treatment assignments should be balanced over time. In this regard, the ICH E9 guideline has the following statement [ 31 ]:. Although unrestricted randomisation is an acceptable approach, some advantages can generally be gained by randomising subjects in blocks.

This helps to increase the comparability of the treatment groups, particularly when subject characteristics may change over time, as a result, for example, of changes in recruitment policy. It also provides a better guarantee that the treatment groups will be of nearly equal size While randomization in blocks of two ensures best balance, it is highly predictable.

In practice, a sensible tradeoff between balance and randomness is desirable. In the following example, we illustrate the issue of chronological bias in the context of a real RCT. Altman and Royston [ 87 ] gave several examples of clinical studies with hidden time trends.

For instance, an RCT to compare azathioprine versus placebo in patients with primary biliary cirrhosis PBC with respect to overall survival was an international, double-blind, randomized trial including patients of whom received azathioprine and placebo [ 88 ]. The study had a recruitment period of 7 years.

A major prognostic factor for survival was the serum bilirubin level on entry to the trial. Altman and Royston [ 87 ] provided a cusum plot of log bilirubin which showed a strong decreasing trend over time — patients who entered the trial later had, on average, lower bilirubin levels, and therefore better prognosis.

Despite that the trial was randomized, there was some evidence of baseline imbalance with respect to serum bilirubin between azathioprine and placebo groups. The azathioprine trial [ 88 ] provides a very good example for illustrating importance of both the choice of a randomization design and a subsequent statistical analysis.

We evaluated several randomization designs and analysis strategies under the given time trend through simulation. Since we did not have access to the patient level data from the azathioprine trial, we simulated a dataset of serum bilirubin values from patients that resembled that in the original paper Fig. Our main goal is to evaluate the impact of the time trend in bilirubin on the type I error rate and power. The latter two designs were found to be the top two performing procedures based on our simulation results in Example 1 cf.

Table 2. PBD 4 is the most commonly used procedure in clinical trial practice. Rand and TBD are two designs that ensure exact balance in the final treatment numbers. For data analysis, we use the Cox regression model, either with or without adjustment for serum bilirubin. Furthermore, we assess two approaches to statistical inference: population model-based and randomization-based.

For each combination of the design, experimental scenario, and data analysis strategy, a trial with patients was simulated 10, times. In each simulation, we used the same time trend in serum bilirubin as described.

Through simulation, we estimated the probability of a statistically significant baseline imbalance in serum bilirubin between azathioprine and placebo groups, type I error rate, and power. First, we observed that the designs differ with respect to their potential to achieve baseline covariate balance under the time trend.

Second, a failure to adjust for serum bilirubin in the analysis can negatively impact statistical inference. Table 4 shows the type I error and power of statistical analyses unadjusted and adjusted for serum bilirubin, using population model-based and randomization-based approaches.

These findings are consistent with the ones for the two-sample t-test described earlier in the current paper, and they agree well with other findings in the literature [ 67 ].

By contrast, population model-based covariate-adjusted analysis is valid for all seven randomization designs. As regards statistical power, unadjusted analyses are substantially less powerful then the corresponding covariate-adjusted analysis, for all designs with either population model-based or randomization-based approaches.

Thus, PBD 2 is the most powerful approach if a time trend is present, statistical analysis strategy is randomization-based, and no adjustment for time trend is made. Remarkably, the power of covariate-adjusted analysis is identical for population model-based and randomization-based approaches. Overall, this example highlights the importance of covariate-adjusted analysis, which should be straightforward if a covariate affected by a time trend is known e.

If a covariate is unknown or hidden, then unadjusted analysis following a conventional test may have reduced power and distorted type I error although the designs such as CRD and Rand do ensure valid statistical inference. Alternatively, randomization-based tests can be applied. The resulting analysis will be valid but may be potentially less powerful. The degree of loss in power following randomization-based test depends on the randomization design: designs that force greater treatment balance over time will be more powerful.

In fact, PBD 2 is shown to be most powerful under such circumstances; however, as we have seen in Example 1 and Example 2, a major deficiency of PBD 2 is its vulnerability to selection bias. From Table 4 , and taking into account the earlier findings in this paper, BSD 3 seems to provide a very good risk mitigation strategy against unknown time trends.

In our last example, we illustrate the importance of the careful choice of randomization design and subsequent statistical analysis in a nonstandard RCT with small sample size. Due to confidentiality and because this study is still in conduct, we do not disclose all details here except for that the study is an ongoing phase II RCT in a very rare and devastating autoimmune disease in children.

The study includes three periods: an open-label single-arm active treatment for 28 weeks to identify treatment responders Period 1 , a week randomized treatment withdrawal period to primarily assess the efficacy of the active treatment vs.

Because of a challenging indication and the rarity of the disease, the study plans to enroll up to 10 male or female pediatric patients in order to randomize 8 patients 4 per treatment arm in Period 2 of the study. The primary endpoint for assessing the efficacy of active treatment versus placebo is the proportion of patients with disease flare during the week randomized withdrawal phase.

In case of a successful outcome, evidence of clinical efficacy from this study will be also used as part of a package to support the claim for drug effectiveness. Very small sample sizes are not uncommon in clinical trials of rare diseases [ 90 , 91 ].

Naturally, there are several methodological challenges for this type of study. A major challenge is generalizability of the results from the RCT to a population. In this particular indication, no approved treatment exists, and there is uncertainty on disease epidemiology and the exact number of patients with the disease who would benefit from treatment patient horizon.

What does random assignment mean? The key to randomized experimental research design is in the random assignment of study subjects — for example, individual voters, precincts, media markets or some other group — into treatment or control groups. Randomization has a very specific meaning in this context. It does not refer to haphazard or casual choosing of some and not others. Randomization in this context means that care is taken to ensure that no pattern exists between the assignment of subjects into groups and any characteristics of those subjects.

Every subject is as likely as any other to be assigned to the treatment or control group. Randomization is generally achieved by employing a computer program containing a random number generator.

Randomization procedures differ based upon the research design of the experiment. Individuals or groups may be randomly assigned to treatment or control groups. Not only does this mean that the researcher can then translate the results to a larger population, but the math and the science behind the study are stronger.

Meaning you get better results. And Meier was the person who insisted on this process in the U. Today, the Food and Drug Administration requires randomized trials be conducted before they'll approve a drug. And young researchers like me learn about randomization as part of the scientific process. And while this might seem inconsequential or it causes delays in getting treatments to patients, it's better for patients in the long run because it means safer and more effective treatments that are right for them.

About CURE. About Advertise Contact TargetedOnc. Strategic Planning. Principal Deputy Director's Page. Previous NCI Directors. NCI Frederick. Advisory Boards and Review Groups. NCI Congressional Justification. Current Congress. Committees of Interest. Legislative Resources. Recent Public Laws. Search Search. Cancer Treatment. Biomarker Testing. Cancer Treatment Vaccines. Checkpoint Inhibitors. Immune System Modulators. Side Effects. Monoclonal Antibodies. T-cell Transfer Therapy.

Photodynamic Therapy. Radiation Therapy. External Beam Radiation. Stem Cell Transplant. Targeted Therapy. Steps to Find a Clinical Trial. Help With Clinical Trials Search. What Are Clinical Trials? Where Trials Take Place. Types of Clinical Trials. Phases of Clinical Trials. Use of Placebos. Research Team Members. Paying for Clinical Trials.

Insurance Coverage and Clinical Trials. Federal Government Programs. Patient Safety. Informed Consent. Children's Assent.