Aug 25, 2020
Statistical Considerations (Part 9 of Phase 3 study of Vaccine Candidate for COVID-19)
- Chris Ockenhouse1,
- Chris Gast1,
- Renee Holt1,
- Jorge Flores1
- 1Center for Vaccine Innovation and Access, PATH (Washington D.C. and Seattle, Washington)
- Coronavirus Method Development Community
- PATH
Protocol Citation: Chris Ockenhouse, Chris Gast, Renee Holt, Jorge Flores 2020. Statistical Considerations (Part 9 of Phase 3 study of Vaccine Candidate for COVID-19). protocols.io https://dx.doi.org/10.17504/protocols.io.bj55kq86
License: This is an open access protocol distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited
Protocol status: Working
We use this protocol and it's working
Created: August 22, 2020
Last Modified: August 25, 2020
Protocol Integer ID: 40861
Disclaimer
DISCLAIMER – FOR INFORMATIONAL PURPOSES ONLY; USE AT YOUR OWN RISK
The protocol content here is for informational purposes only and does not constitute legal, medical, clinical, or safety advice, or otherwise; content added to protocols.io is not peer reviewed and may not have undergone a formal approval of any kind. Information presented in this protocol should not substitute for independent professional judgment, advice, diagnosis, or treatment. Any action you take or refrain from taking using or relying upon the information presented here is strictly at your own risk. You agree that neither the Company nor any of the authors, contributors, administrators, or anyone else associated with protocols.io, can be held responsible for your use of the information contained in or linked to this protocol or any of our Sites/Apps and Services.
Abstract
This is Part 9 of "Phase 3 randomized, double-blinded, placebo-controlled trial to evaluate the safety, immunogenicity, and efficacy of Vaccine Candidate against COVID-19 in adults > 18 years of age"
This generic Phase 3 protocol was developed by the PATH team with support of the Bill and Melinda Gates Foundation. The aim of the collection is to share recommended best practices in designing and implementing a Phase 3 study of a COVID-19 vaccine candidate. As Phase 3 trials of different Vaccine Candidates proceed around the world, following the same protocols will ensure consistency and comparability of the Phase 3 trial results.
Please note that this is an evolving document, to be versioned and updated, based on community feedback and new data.
Attachments
Guidelines
This section summarizes the primary features of the statistical analysis for the study. A statistical analysis plan (SAP) will be prepared and finalized prior to observation of the first 25 percent of events (at which time the blinded attack rate will be reviewed for potential sample size adjustments) which will specify all analyses to be performed. A CRO will author the SAP and produce the statistical analysis, under the guidance and oversight of Sponsor.
The primary and secondary efficacy analyses will be assessed using Cox proportional hazards regression (time to positive confirmed SARS-CoV-2 PCR infection meeting clinical criteria following adjudication), stratified by site, age group, and sex, with the efficacy estimate obtained as 1 minus the estimated hazard ratio, with CIs similarly transformed.These hypothesis tests will be supplemented with two-sided CIs for the hazard ratios as described below.
The null (H0) and alternative (H1) hypotheses for the primary analysis of time to PCR-confirmed COVID-19 in Groups 1 and 2 are as follows:
- H0: Efficacy of Vaccine Candidate against PCR-confirmed COVID-19 is less than or equal to 30 percent.
- H1: Efficacy of Vaccine Candidate against PCR-confirmed COVID-19 is greater than 30 percent.
9.1. Overview and general design
This is a case-driven, randomized, double-blind, placebo-controlled adaptive, group-sequential Phase 3 trial of Vaccine Candidate or placebo. The trial will be conducted in X countries. The trial will include adults > 18 years of age.
9.1.1. Randomization procedures
Participants will be randomized XX:XX to receive either Vaccine Candidate (Group 1) or a control vaccine (placebo, or other vaccine) (Group 2). Randomization will be stratified by site, age group, and sex. The randomization scheme will be generated and maintained by the SDMC. Participants will be enrolled into the study online and randomized using the SDMC interactive web response system module. After demographic and eligibility data have been entered into the system, each participant enrolled into the study will be assigned a treatment code based on their cohort.
9.2. Sample size
Sample size is driven by the accrual of sufficient cases for analysis. This study will be group-sequential, providing an opportunity for an unblinded efficacy and futility analysis reviewed by the DSMB, when 40 percent of the total number of cases have been accrued. Upon the earlier of the completion of enrollment or accrual of 25 percent of the total event count, the DSMB will review the blinded incidence rate of per-protocol cases and compare this to the value used for planning. The DSMB may recommend a sample size increase, to a cap of XX,XXX, under an assumption that the low accrual is due to lower-than-expected attack rate. The DSMB will have access to blinded (to group) endpoint accrual by site to enable decisions about opening or closing specific sites.
The following assumptions and thresholds are used to obtain the required sample size:
| 6-month attack rate | 1.0% | |
| Vaccine efficacy (VE) | 60% | |
| Minimum VE to be demonstrated | 30% | |
| One-sided alpha | 2.5% | |
| Power | 90% | |
| Annual dropout rate | 5% | |
| Number of interim analyses | 1 | |
| Spending function, upper (efficacy) bound | Hwang-Shih-DeCani, parameter = -4 | |
| Spending function, lower (futility) bound, non-binding | Hwang-Shih-DeCani, parameter = -4 | |
| Enrollment of participants | Uniform over enrollment duration | |
| Randomization ratio | 1:1 |
This design yields the following characteristics:
| Number of events to trigger interim analysis | 61 | |
| Number of events to trigger primary analysis | 151 | |
| Alpha spent at interim analysis | 0.0018 | |
| Alpha spent at primary analysis | 0.0241 | |
| Probability, under alternative hypothesis, of crossing efficacy boundary at interim analysis | 0.2007 | |
| Cumulative probability, under alternative hypothesis, of crossing efficacy boundary by primary analysis | 0.9 | |
| Probability, under alternative hypothesis, of crossing futility boundary at interim analysis | 0.0074 | |
| Cumulative probability, under alternative hypothesis, of crossing futility boundary by primary analysis | 0.1 | |
| Probability, under null hypothesis, of crossing efficacy boundary at interim analysis | 0.0018 | |
| Cumulative probability, under null hypothesis, of crossing efficacy boundary by primary analysis | 0.0248 | |
| Probability, under null hypothesis, of crossing futility boundary at interim analysis | 0.3543 | |
| Cumulative probability, under null hypothesis, of crossing futility boundary by primary analysis | 0.9752 |
It is possible to accrue the required number of events quickly by enrolling many participants; or, if enrollment rates are low, by increasing the expected time to reach analysis milestones. The table below describes the balance between enrollment rate (total), enrollment duration, and expected time to reach analysis milestones (interim analysis at 40 percent of events, accrual of 100 percent of events for primary analysis).
| Number of participants enrolled per month | Duration (months) of enrollment | Expected time (months) to interim analysis | Expected time to primary analysis | Number of evaluable participants required1 | |
| 2,000 | 9 | 8 | 12 | 18,000 | |
| 4,400 | 6 | 5 | 8 | 26,000 | |
| 8,300 | 4 | 4 | 6 | 33,000 |
1Numbers are rounded up to the nearest 1,000
It is desired to achieve the primary analysis by 12 months from the time of entry of the first participant enrolled into the post-vaccination follow-up period. It is anticipated that recruitment can occur over a span of 9 months, thereby providing 3 months of follow-up on the last participant enrolled prior to conduct of the primary analysis. This design requires approximately 18,000 participants to be enrolled. It is anticipated that the interim analysis for efficacy would then be performed at approximately the same time as the end of enrollment, providing the DSMB a final opportunity to suggest a blinded sample size adjustment, just prior to unblinding for the interimefficacy analysis.
Note
Note: These numbers do not take into account any inflation if the primary analysis were to remove those who are antibody-positive at baseline from the primary analysis. That is, if 2,000 per month are enrolled but X percent are antibody-positive and therefore not evaluable in the primary analysis, then the study is actually only recruiting (1-X percent)*2000 evaluable participants per month, which will extend the stated study durations.
The primary outcome will be laboratory-confirmed COVID-19 with onset of symptoms at least 14 days after the second dose. If at final analysis the success criteria are just met (p <0.0241), the estimated hazard ratio is expected to be approximately 50%, providing a vaccine efficacy estimate of approximately 50%, with the appropriately adjust lower confidence bound above 30%.
9.3. Definitions of populations to be analyzed
9.3.1. Enrolled population
The enrolled population is defined as all screen participants who provide informed consent and are eligible for study participation, regardless of the participant’s randomization and treatment status in the study.
9.3.2. Safety population
The safety population is defined as all participants in the enrolled population who received a study vaccine and have any safety data available. Summaries among the safety population will be conducted according to the vaccine received.
9.3.3. Intention-to-Treat (ITT) population
The ITT population is defined as all participants in the enrolled population who are randomized to a study arm. Analysis among the ITT population will be conducted with participants contributing to the group to which they are randomized. All efficacy analyses will be secondarily evaluated among the ITT population.
9.3.4. Reactogenicity population
The reactogenicity population is defined as all participants in the safety population in the subset of 1,000 participants per country returning a diary card for evaluation of solicited AEs. The population is defined on a per-timepoint basis (each vaccination) and requires receipt of the vaccination for population membership.
9.3.5. Per-protocol population (PP)
The per-protocol (PP) population is defined as all participants who correctly received study vaccinations within the allowable window for the second/third vaccination per randomization, with no major protocol deviations.
Prior to database lock, the database will be searched for potentially disqualifying deviations. Additionally, protocol deviations will be collected from monitoring and medication listings will be reviewed. The following scenarios constitute potential criteria for elimination from the PP population. The list is not exhaustive, as unexpected deviations may arise requiring unique consideration.
- Significant non-compliance with vaccination visit windows.
- Any inclusion/exclusion criteria not met.
- Receipt of vaccine not stored per manufacturers approved storage condition.
- Baseline RT-PCR confirmatory test results unavailable.
- Incorrect randomization.
- Incorrect vaccination received.
Participants will remain in the PP population until such time as a major protocol deviation is encountered (e.g., receipt of an alternative investigational COVID-19 vaccine), and participants will contribute follow-up time to the PP analyses following exclusionary events. A Data Review Meeting report will provide the criteria used for determination, as well as list the participants excluded and the corresponding time point, and accompanying rationale.
Efficacy will primarily be evaluated among the PP population.
9.3.6. Immunogenicity pipulations
The primary immunogenicity population for serum IgG concentrations as measured by ELISA against SARS-CoV-2 spike protein is defined as all participants in the PP population who received a study vaccine and who contributed evaluable baseline and post-baseline samples for analysis.
The primary immunogenicity population for neutralizing antibody against SARS-CoV-2 spike protein is defined as all participants in the PP population who received a study vaccine and who contributed evaluable baseline and post-baseline samples for analysis. Among such participants, a randomly chosen subset will be analyzed.
Additional populations will be specified in the SAP.
9.4. Interim analysis, monitoring for harm
The interim analysis will be initiated upon collection of sufficient events in the PP efficacy population, and analysis will be conducted primarily on the PP efficacy population, and also among the ITT population. The analysis of the PP population will be the basis for determination of VE. The interim analysis will be reviewed by the DSMB, which will issue a recommendation to halt the study for overwhelming evidence of efficacy, for sufficient evidence of futility, or for safety concerns, or to proceed as planned. The SDMC will monitor for harm (excess of cases in the investigational arm) continuously against predefined criteria, and notify the DSMB if a threshold is crossed indicating an abundance of cases occurring in group receiving the investigational vaccine. Details will be included in the interim analysis plan.
Should efficacy criteria be met at interim analysis, enhanced passive surveillance will be continued for at least one year to assess duration of effect.After one year it will be recommended to participants in the control arm (placebo group) to receive the two doses of Vaccine Candidate. Should neither efficacy nor futility criteria be met at interim analysis, enrollment, if not complete, will continue. In both cases, safety follow-up would continue for two years, even after the initial results are released.
9.5. Analytical methodology
9.5.1. Descriptive methodology
All data collected will be summarized and/or listed. Analyses will be performed using XXX software.
Unless otherwise specified, descriptive statistics include the mean, standard deviation, median, minimum, maximum for continuous variables, and the number and proportion in each group for categorical variables. Unless otherwise specified here or in the SAP, statistical tests and CIs will be computed using a two-sided 5 percent significance level. Exact CIs will be used for univariate summaries of dichotomous variables, and score-based CIs will be used for rate differences. All proportions will use as denominator the number of participants contributing data at the specified time point within the specified group and study population.
Summaries will be presented by group and by time point, where relevant.
For the enrolled population, medical history will be listed and summarized by category. Using the WHO Drug Dictionary, concomitant medications will be tabulated by anatomical therapeutic chemical classification, preferred drug name and treatment group. Medical history will be tabulated by MedDRA System Organ Class, preferred term, and vaccine group.
Summaries of subject disposition will be prepared for all participants, including the number and percent enrolled, screened, randomized, and administered vaccine, as well as a Consolidated Standards of Reporting Trials (CONSORT) diagram describing study participation and discontinuation. The reasons for screening failures and discontinuations will be summarized and listed.
A summary and listing of visit attendance will be prepared, in addition to a summary and listing of vaccine administration and sample collection/availability for each sample.
9.5.2. Changes in analysis plan
Any deviations or changes from the statistical analyses specified in the protocol will be described and justified in the SAP and the clinical study report.
9.5.3. Baseline and demographic characteristics, and participant disposition
Descriptive statistics will be computed for demographic characteristics (e.g., height, weight, race, gender, presence/absence of SARS-CoV-2 antibody at baseline) in both the safety and PP populations, and other initial participant characteristics (e.g. medical and surgical history, concomitant diseases) in the safety population. Analyses will be conducted according to age cohort (<60 years; > 60 years) and group, using the Fisher exact test for binary variables, and analysis of variance for continuous variables.
Prior and concomitant medications will be coded using the WHO Drug Dictionary. Medical history will be coded using the most recent version of the MedDRA.
Participant disposition including dropout and reasons for dropout, as well as study population membership, will be summarized descriptively and supported with a CONSORT diagram.
9.5.4. Analysis of the primary efficacy endpoint
A RT-PCR positive for the presence of SARS-CoV-2 and symptoms consistent with COVID-19, following adjudication, meets the definition of an event if the result is obtained from a nasopharyngeal sample collected at any time during the active detection of infection phase of the trial.
Vaccine efficacy among adults with RT-PCR positive for the presence of SARS-CoV-2 and symptoms consistent with COVID-19, will be assessed using Cox proportional hazards regression, stratified by site, age group, and sex, with a covariate for group assignment to compare Groups 1 and 2. The vaccine efficacy estimates (1-hazard ratio), 95 percent CI, and p-values will be calculated from this model. Cumulative incidence graphs will also be provided. This analysis will be conducted in the PP for efficacy cohort and repeated for the safety population.
The hypothesis test of the final primary efficacy endpoint among the PP population will be based on the alpha levels to be spent at each analysis, described in the table above, and accompanied with corresponding CIs. If additional events accrue following the final event necessary for primary analysis but prior to closure of the database, a final analysis will be produced incorporating any such events. Efficacy analysis will also be conducted with the ITT population.
9.5.5. Analysis of the secondary endpoint(s)
Secondary efficacy endpoints will be evaluated with the same methodology as the primary efficacy endpoint, foregoing adjustment for sequential evaluation. Evaluation of the primary endpoint within randomization strata (age group, sex, site) will be performed similarly, stratified by randomization strata not used for subsetting. Secondary efficacy analyses will be conducted primarily in the PP population, and repeated in the ITT population.
For each efficacy endpoint, cumulative VE over time, defined as 100 percent × (1 minus the ratio of cumulative incidence by time t) will be plotted, accompanied by pointwise and simultaneous 95 percent CIs computed with the method of Parzen, Wei, and Ying [18].
9.6. Safety analysis
All safety analyses will be conducted in the safety population, according to the vaccine received. Analyses will be conducted overall, as well as according to age cohort and group.
Solicited adverse events
Solicited events will be summarized by computing the proportion of participants observed to experience any event, and any event according to grade and event type within both the vaccine and placebo groups. All analyses will be computed for immediate events only (events recorded by the clinic during the 30-minute post-vaccination observation period), and for immediate events plus those recorded by the participant on the memory aid, combined. Additional analyses will summarize the frequency and duration of events extending beyond seven days in duration. Summaries will be prepared corresponding to maximum severity and duration per participant, where relevant. Solicited AE rates will be accompanied by two-sided exact 95 percent CIs. Within cohorts, the rate of solicited AEs will be compared using a two-sided Fisher’s exact test for pairwise comparisons, both overall and by type. This will be repeated for severe events.
Unsolicited and medically attended adverse events
All unsolicited and medically attended AEs will be coded, listed, and summarized. Unless an AE is classified as an SAE, summaries of unsolicited AEs will be made using only those events recorded with onset within 28 days of vaccination, and with Grade ≥2. Additional summaries will present unsolicited AEs regardless of grade and onset, which may be recorded due to a suspected or known case of COVID-19. Unsolicited AEs will primarily be summarized on the participant level, where a participant contributes once to a given event type under the maximum severity and/or causality, as appropriate. Tables will display the number of events of a given type observed within a group, regardless of the number of participants from which they originate.
Unsolicited AEs will be summarized by severity and relationship to vaccination; SAEs will be summarized by type, relationship to vaccination, and reason for designation as SAE. In addition, all AEs, coded with MedDRA, will be summarized by System Organ Class and preferred term, and separate tables for both will be prepared for unsolicited AEs and SAEs. A table will be prepared summarizing all preferred terms occurring in ≥2 percent of participants. Additional summaries will include the rate of participants experiencing an AE that withdraw from the study. Listings will also be prepared for SAEs and related AEs. Within age-group, the rates of participants experiencing AEs Grade ≥3, related AEs, and SAEs will be compared across groups using a two-sided Fisher’s exact test.
Vaccine-enhanced disease
The potential for VED will be evaluated by comparing the severity of respiratory events (and specifically, COVID-19 cases) between groups, as well as the frequency of severe respiratory events (and specifically, COVID-19 cases) as a function of the total number of cases within each group. Severity score will be compared via the Wilcoxon Rank-sum test. The frequency of severe events among total events will be compared with the difference in proportion of such events which are severe vs. non-severe. A separate CRF will collect symptoms for a qualifying diagnosis of respiratory disease and severity of such symptoms. The overall severity category will be compared between groups overall and within age and gender strata.
Other safety measures
- Vital signs will be summarized descriptively, including change from baseline for continuous measures. Vital sign abnormalities will be summarized descriptively.
- Abnormal physical examination findings will be summarized descriptively.
- Screening clinical lab values will be summarized descriptively, and separate summaries will describe abnormalities according to grade.
- Pregnancy test results will be listed.
9.7. Serological analysis
The primary analysis will be based on the PP population for immunogenicity analysis. The antibody titer of IgG against SARS-CoV-2 will be measured in all the participants. Neutralizing antibody titers will be measured in a randomly chosen subset of XX PP participants. A secondary analysis based on the full analysis population will be performed to complement the PP analysis.
At each serum sampling time point, the percentage of participants seropositive for the vaccine antigen and binding antibody (bAb) titers will be summarized with corresponding CI. At these time points, the GMT of bAb titers will be computed with 95 percent CI. Antibody levels will be plotted using reverse cumulative distribution curves.
The percentage of participants with neutralizing antibodies to the virus strain as used in the vaccine >1:4 will be summarized as for bAb titers, among the subset of participants providing samples.
For SARS-CoV-2 neutralizing titers (PRNT50), as well as anti-vaccine IgG titers, the following descriptive statistics will be computed for each time point for each group:
- Seroconversion rates with 95 percent exact CIs will be tabulated for the post-baseline sample.
- Median of titers/concentrations will be computed along with 95 percent Cis.
- GMT/GMC with accompanying 95 percent CIs will be computed.
- Plots of the reverse cumulative distribution of antibody titers/concentrations will be generated.
- Boxplots will display the distribution of titers/concentrations separately for each group and antigen, for both baseline and post-baseline time points. Additional subgroup computations will be specified and conducted, per the SAP.
9.8. Handling of dropouts and missing data
All missing data will be assumed to be missing completely at random, and no imputation will be performed. Analysis will, therefore, exclude participants with missing or non-evaluable measurements. If an excess of data is missing or if patterns in the missing data are detected, the SAP may specify methods to accommodate this.
LITERATURE CITED
18. Parzen MI, Wei LJ, Ying Z. Simultaneous confidence intervals for the difference of two survival functions. Scand J Statist 1997;24:309-14.