This study evaluates the effectiveness of the new BNT162b2 mRNA vaccine1 against Covid-19 in a comprehensive nationwide vaccination environment. The estimated vaccine efficacy during the follow-up period beginning 7 days after the second dose was 92% for documented infection, 94% for symptoms of Covid-19, 87% for hospitalization, and 92% for severe Covid-19. The estimated efficacy over days 14 to 20 (after one dose) and days 21 to 27 (gradual shift between first and second vaccine doses) was 46% and 60% for documented infections, 57% and 66% for symptoms of Covid-19, 74% and 78% for hospitalization, 62% and 80% for severe Covid-19, and 72% and 84% for Covid-19-related death, respectively.
The first primary endpoint evaluated in the BNT162b2 randomized trial was symptomatic Covid-19. In both the randomized trial and our study, the cumulative incidence of symptomatic Covid-19 in the vaccinated and unvaccinated groups began to diverge around day 12 after the first dose.1 The estimated vaccine efficacy for Covid-19 symptoms beginning on day 7 after the second dose was 95% in the randomized trial, compared to 94% in our study. The estimated efficacy between the first dose and the second dose was 52% in the trial, compared to 29% in our study. This difference may reflect the high level of transmission in Israel during the study period,14 that affected both vaccinated subjects and controls equally during the first 12 days after administration of the first dose. To eliminate this distortion, we estimated the effectiveness of the first dose of vaccine against Covid-19 for the period from 14 to 20 days; The estimated efficacy was 57%.
The estimated efficacy for documented infection during days 14-20 was 46% in our study. A relatively similar efficacy of 51% was reported by Chodick et al. ,15th who evaluated a cohort from another healthcare organization in Israel and used a different study design comparing infection among vaccinated people on days 13 to 24 after the first dose against infection during days 0 through 12.
In the randomized trial, the estimated vaccine efficacy for severe Covid-19 (89% over the entire study period) was based on only 10 cases. Our study recorded 229 cases of severe Covid-19 – 55 in the vaccinated group and 174 in the unvaccinated group – resulting in an estimated efficacy of 62% for days 14 to 20 after the first dose, 80% for days 21 to 27, and 92% for 7 days or more after the second dose.
The large sample size in our study also allowed us to estimate vaccine efficacy for specific subpopulations that a randomized trial was not sufficient to evaluate. In the trial, the estimated efficacy for Covid-19 in people up to 55 years of age, 55 years of age, and 65 years of age or older 7 days after the second dose was 94 to 96%. We were able to study more precise age groups, and estimated that vaccine efficacy was similar for adults aged 70 or older and for younger age groups in the same time period.
A randomized trial estimated vaccine efficacy for patients with one or more coexisting conditions according to the Charlson Comorbidity Index.16 Specifically for patients with obesity or high blood pressure. These measures do not provide clarity regarding efficacy in patients with multiple coexisting conditions. We estimated the vaccine’s efficacy with respect to different numbers of coexisting conditions and found indications that the efficacy might be slightly lower among people with higher numbers of coexisting conditions.
Two factors make the current study uniquely suited for evaluating the efficacy of the BNT162b2 vaccine in a practical application: first, a rare combination of rich medical background data, Covid-19 PCR test results (for documented infection results), and patient follow-up—increasing data in both the community (for Covid outcomes). -19 symptomatic) and inpatient (for all other outcomes) – the HSBS has maintained an integrated data repository for more than half of the population of Israel, and updated it daily, for more than two decades; And second, the rapid pace and high uptake of the Covid-19 vaccine in Israel and the high rates of disease during the vaccination campaign. On the other hand, the rapid pace of the vaccination campaign contributed to the frequent oversight of data for matched unvaccinated controls, especially among those over 60 years of age (often only a few days after matching) and a corresponding decrease in the median follow-up period of the study.
Concerns have arisen about potential resistance of SARS-CoV-2 variants to Covid-19 vaccines17, 18 and neutralizing antibodies.19, 20 During the study period, an increasing proportion of SARS-CoV-2 isolates in Israel – up to 80% in the days prior to data extraction – were of the B.1.1.7 variant.21 Thus, this study estimates the average efficacy of the vaccine on multiple strains. Although we cannot provide a specific efficacy estimate for the B.1.1.7 variant, the plateau observed during later periods in the cumulative incidence curve of vaccinated subjects suggests that BNT162b2 vaccine is also effective for this variant, an observation consistent with previous reports that have shown Preserved neutralizing antibody titer.22 The B.1.351 variant was estimated to be rare in Israel at the time of data extraction.23
As with any observational study, our study may have been influenced by residual confusion due to differences between vaccinated subjects and unvaccinated controls, particularly with regard to health-seeking behaviour. We therefore performed rigorous matching on a wide range of factors that might be expected to confound the causal effect of a vaccine on different outcomes. After the matching process, we found a consistent pattern of similarity between groups in the days immediately preceding the 12th day after the first dose (the expected onset of the vaccine effect), which thus serves as a ‘negative control’24 a period (Figure 2and Figure S6 and Table S7). This similarity occurred despite a temporary increase in events among unvaccinated controls during the first days after the first vaccine dose, and most likely stems from the fact that subjects who choose to vaccinate on a given day feel healthy at the time of vaccination. Similarity of study groups in coexisting conditions and known risk factors for severe Covid-19 (Table 1 Figure S2) provides further evidence of interchangeability (ie no confusion). However, this rigorous matching process came at the expense of the final group not including approximately 34% of vaccinated subjects who met the study eligibility criteria. Limited matching of age and gender only would not be sufficient to eliminate early confounding (Fig. S6).
We also excluded populations with high internal variance in vaccination probability or outcome, such as health care workers, people held at home for medical reasons, and nursing home residents, to avoid residual confusion. Although the randomized trial was less likely to include people who were not healthy enough to comply with scheduled visits and the vaccination plan, it did not exclude health care workers.
To assess a potential selection bias that could stem from media censorship, where vaccinated controls feel satisfied around the time of vaccination, we performed a sensitivity analysis where they were kept in the unvaccinated group for a period of time defined differently for each outcome (Fig. ). This analysis showed results similar to those of the main analysis, indicating that any such bias was small in our analysis.
Finally, the history of symptom onset was not available for analysis. Alternatively, for infection results, the date was indicated on the swab collection date for the first positive PCR test. Given that there was likely to be a time gap between symptom onset and collection of swabs, the discrepancy observed in cumulative incidence plots of infection outcomes between vaccinated subjects and unvaccinated controls may be slightly delayed. In parallel, there may be a lower estimate of vaccine efficacy in each time window, as the estimate actually reflects a narrower window for the vaccine to be active. Since SARS-CoV-2 PCR testing is highly accessible in Israel and can be performed without referral within hours, we estimate this potential time gap and therefore the underestimation of vaccine efficacy is small. When interpreting efficacy estimates for more severe outcomes, longer median gaps (Figure S3) should be considered: 1 day for hospitalization, 5 days for severe Covid-19, and 11 days for Covid-19 death.
This study estimates the high efficacy of BNT162b2 vaccine in preventing symptoms of Covid-19 in an uncontrolled setting, similar to the vaccine efficacy reported in the randomized trial. Our study also indicates that efficacy is high for the most serious outcomes: hospitalization, severe illness, and death. Moreover, the estimated benefit increases in order of magnitude over time. These findings reinforce expectations that newly approved vaccines can help mitigate the profound global effects of the Covid-19 pandemic.