This study evaluates the effectiveness of the new BNT162b2 mRNA vaccine1 Against Covid-19 in a nationwide mass vaccination venue. The estimated vaccine efficacy during the follow-up period beginning 7 days after the second dose was 92% for documented infection, 94% for symptoms of Covid-19, 87% for hospitalization, and 92% for severe Covid-19. The estimated efficacy over days 14 to 20 (after a single dose) and days 21 to 27 (gradual shift between first and second vaccine doses) was 46% and 60% for documented infections, 57% and 66% for symptoms of Covid-19, 74% and 78% for hospitalization, 62% and 80% for severe Covid-19, and 72% and 84% for Covid-19-related death, respectively.
The first primary endpoint evaluated in the BNT162b2 randomized trial was symptomatic Covid-19. In both the randomized trial and our study, the cumulative incidence of symptomatic Covid-19 in the vaccinated and unvaccinated groups began to diverge around day 12 after the first dose.1 The estimated vaccine efficacy for Covid-19 symptoms beginning on day 7 after the second dose was 95% in the randomized trial, compared to 94% in our study. The estimated efficacy between the first dose and the second dose was 52% in the trial, compared to 29% in our study. This difference may reflect the high level of transmission in Israel during the study period,14 that affected both vaccinated subjects and controls equally during the first 12 days after administration of the first dose. To eliminate this distortion, we estimated the effectiveness of the first dose of vaccine against Covid-19 for the period from 14 to 20 days; The estimated efficacy was 57%.
The estimated efficacy for documented infection during days 14-20 was 46% in our study. A relatively similar efficacy of 51% was reported by Chodick et al. ,15th who evaluated a cohort from another healthcare organization in Israel and used a different study design comparing infection among vaccinated people on days 13 to 24 after the first dose against infection during days 0 through 12.
In the randomized trial, the estimated vaccine efficacy for severe Covid-19 (89% over the entire study period) was based on only 10 cases. Our study recorded 229 cases of severe Covid-19 – 55 in the vaccinated group and 174 in the unvaccinated group – resulting in an estimated efficacy of 62% for days 14 to 20 after the first dose, 80% for days 21 to 27, and 92% For 7 days or longer after the second dose.
The large sample size in our study also allowed us to estimate vaccine efficacy for certain subpopulations that a randomized trial was not sufficient to evaluate. In the trial, the estimated efficacy for Covid-19 in people up to 55 years of age, 55 years of age, and 65 years of age or older 7 days after the second dose was 94 to 96%. We were able to study more precise age groups, and estimated that vaccine efficacy was similar for adults aged 70 or older and for younger age groups in the same time period.
A randomized trial estimated vaccine efficacy for patients with one or more coexisting conditions according to the Charlson Comorbidity Index.16 Specifically for patients with obesity or high blood pressure. These measures do not provide clarity regarding efficacy in patients with multiple coexisting conditions. We estimated the vaccine’s efficacy with respect to different numbers of coexisting conditions and found indications that the efficacy might be slightly lower among people with higher numbers of coexisting conditions.
Two factors make the current study uniquely suited to assess the efficacy of the BNT162b2 vaccine in a practical application: first, a rare combination of rich medical background data, Covid-19 PCR test results (for documented infection results), and patient follow-up—higher data in both the community (for Covid outcomes). -19 symptomatic) and inpatient (for all other outcomes) – the HSBS has maintained an integrated data repository for more than half of the population of Israel, and updated it daily, for more than two decades; And secondly, the rapid pace and high uptake of the Covid-19 vaccine in Israel and the high rates of disease during the vaccination campaign. On the other hand, the rapid pace of the vaccination campaign contributed to the frequent oversight of data for matched unvaccinated controls, especially among those over 60 years of age (often only a few days after matching) and a corresponding decrease in the median follow-up period of the study.
Concerns have arisen about potential resistance of SARS-CoV-2 variants to Covid-19 vaccines17, 18 and neutralizing antibodies.19, 20 During the study period, an increasing proportion of SARS-CoV-2 isolates in Israel – up to 80% in the days before data extraction – were of the B.1.1.7 variant.21 Thus, this study estimates the average efficacy of the vaccine on multiple strains. Although we cannot provide a specific efficacy estimate for the B.1.1.7 variant, the plateau observed during later periods in the cumulative incidence curve of vaccinated subjects suggests that the BNT162b2 vaccine is also effective for this variant, an observation consistent with previous reports that showed Neutralizing antibody titers.22 The B.1.351 variant was estimated to be rare in Israel at the time of data extraction.23
As with any observational study, our study may have been influenced by residual confusion due to differences between vaccinated subjects and unvaccinated controls, particularly with regard to health-seeking behaviour. We therefore performed a rigorous matching of a wide range of factors that might be expected to confound the causal effect of a vaccine on different outcomes. After the matching process, we found a consistent pattern of similarity between groups in the days before day 12 immediately after the first dose (the expected onset of the vaccine effect), which thus serves as a ‘negative control’24 interval (Figure 2and Figure S6 and Table S7). This similarity occurred despite the temporary increase in events among unvaccinated controls during the first days after the first vaccine dose, and most likely stems from the fact that subjects who choose to vaccinate on a given day feel healthy at the time of vaccination. Similarity of study groups in coexisting conditions and known risk factors for severe Covid-19 (Table 1 Figure S2) provides further evidence of interchangeability (ie no ambiguity). However, this strict matching process came at the cost of not including approximately 34% in the final cohort of vaccinated subjects who met the study eligibility criteria. Limited matching for age and gender only would not be sufficient to remove early confusion (Fig. S6).
We also excluded populations with high internal variance in vaccination probability or outcome, such as health care workers, people confined to the home for medical reasons, and nursing home residents, to avoid residual confusion. Although the randomized trial was also less likely to include people who were not healthy enough to comply with scheduled visits and the vaccination plan, it did not exclude health care workers.
To assess a potential selection bias that could stem from media censorship, where vaccinated controls feel comfortable around the time of vaccination, we performed a sensitivity analysis where they were kept in the unvaccinated group for a period of time assigned differently for each outcome (Figure S7 and Table S5 ). This analysis showed results similar to those of the main analysis, indicating that any such bias was small in our analysis.
Finally, the history of symptom onset was not available for analysis. Alternatively, for infection results, the date was set to the swab collection date of the first positive PCR test. Given that there was likely to be a time gap between symptom onset and collection of swabs, the observed difference in cumulative incidence plots of infection outcomes between vaccinated subjects and unvaccinated controls may be slightly delayed. In parallel, there may be a lower estimate of the vaccine’s efficacy in each time window, because the estimate actually reflects a narrower window for the vaccine to be active. Since the SARS-CoV-2 PCR test is highly accessible in Israel and can be performed without referral within hours, we estimate this potential time gap and therefore the underestimation of vaccine efficacy is small. When interpreting efficacy estimates for more severe outcomes, longer median gaps (Fig. S3) should be considered: 1 day for hospitalization, 5 days for severe Covid-19, and 11 days for Covid-19 death.
This study estimates the high efficacy of the BNT162b2 vaccine in preventing symptoms of Covid-19 in an uncontrolled setting, similar to the efficacy of the vaccine reported in the randomized trial. Our study also indicates that efficacy is high for the most severe outcomes: hospitalization, severe illness, and death. Moreover, the estimated benefit increases in order of magnitude over time. These findings reinforce expectations that newly approved vaccines can help mitigate the profound global effects of the Covid-19 pandemic.