The world has bet the farm on vaccines as the solution to the pandemic, but the trials are not focused on answering the questions many might assume they are. Peter Doshi reports

As phase III trials of covid-19 vaccines reach their target enrolments, officials have been trying to project calm. The US coronavirus czar Anthony Fauci and the Food and Drug Administration leadership have offered public assurances that established procedures will be followed.[1,2,3,4] Only a “safe and effective” vaccine will be approved, they say, and nine vaccine manufacturers issued a rare joint statement pledging not to prematurely seek regulatory review.[5]

But what will it mean exactly when a vaccine is declared “effective”? To the public this seems fairly obvious. “The primary goal of a covid-19 vaccine is to keep people from getting very sick and dying,” a National Public Radio broadcast said bluntly.[6]

Peter Hotez, dean of the National School of Tropical Medicine at Baylor College of Medicine in Houston, said, “Ideally, you want an antiviral vaccine to do two things . . . first, reduce the likelihood you will get severely ill and go to the hospital, and two, prevent infection and therefore interrupt disease transmission.”[7]

Yet the current phase III trials are not actually set up to prove either (table 1). None of the trials currently under way are designed to detect a reduction in any serious outcome such as hospital admissions, use of intensive care, or deaths. Nor are the vaccines being studied to determine whether they can interrupt transmission of the virus.

Evaluating mild, not severe, disease

In a September interview Medscape editor in chief Eric Topol pondered what counts as a recorded “event” in the vaccine trials. “We’re not talking about just a PCR [polymerase chain reaction test]-positive mild infection. It has to be moderate to severe illness to qualify as an event, correct?” he asked.[8]

“That’s right,” concurred his guest, Paul Offit, a vaccinologist who sits on the FDA advisory committee that may ultimately recommend the vaccines for licence or emergency use authorisation.

But that’s not right. In all the ongoing phase III trials for which details have been released, laboratory confirmed infections even with only mild symptoms qualify as meeting the primary endpoint definition.[9,10,11,12] In Pfizer and Moderna’s trials, for example, people with only a cough and positive laboratory test would bring those trials one event closer to their completion. (If AstraZeneca’s ongoing UK trial is designed similarly to its “paused” US trial for which the company has released details, a cough and fever with positive PCR test would suffice.)

Part of the reason may be numbers. Severe illness requiring hospital admission, which happens in only a small fraction of symptomatic covid-19 cases, would be unlikely to occur in significant numbers in trials. Data published by the US Centers for Disease Control and Prevention in late April reported a symptomatic case hospitalisation ratio of 3.4% overall, varying from 1.7% in 0-49 year olds and 4.5% in 50-64 year olds to 7.4% in those 65 and over.[13] Because most people with symptomatic covid-19 experience only mild symptoms,[14] even trials involving 30 000 or more patients would turn up relatively few cases of severe disease.

In the trials, final efficacy analyses are planned after just 150 to 160 “events,”—that is, a positive indication of symptomatic covid-19, regardless of severity of the illness.

Yet until vaccine manufacturers began to release their study protocols in mid-September, trial registries and other publicly released information did little to dispel the notion that it was severe covid-19 that the trials were assessing. Moderna, for example, called hospital admissions a “key secondary endpoint” in statements to the media.[15] And a press release from the US National Institutes of Health reinforced this impression, stating that Moderna’s trial “aims to study whether the vaccine can prevent severe covid-19” and “seeks to answer if the vaccine can prevent death caused by covid-19.”[16]

But Tal Zaks, chief medical officer at Moderna, told The BMJ that the company’s trial lacks adequate statistical power to assess those outcomes. “The trial is precluded from judging [hospital admissions], based on what is a reasonable size and duration to serve the public good here,” he said.

Hospital admissions and deaths from covid-19 are simply too uncommon in the population being studied for an effective vaccine to demonstrate statistically significant differences in a trial of 30 000 people. The same is true of its ability to save lives or prevent transmission: the trials are not designed to find out.

Zaks said, “Would I like to know that this prevents mortality? Sure, because I believe it does. I just don’t think it’s feasible within the timeframe [of the trial]—too many would die waiting for the results before we ever knew that.”

Stopping transmission

What about Hotez’s second criterion, interrupting virus transmission, which some experts have argued[17] should be the most important test in phase III studies?

“Our trial will not demonstrate prevention of transmission,” Zaks said, “because in order to do that you have to swab people twice a week for very long periods, and that becomes operationally untenable.”

He repeatedly emphasised these “operational realities” of running a vaccine trial. “Every trial design, especially phase III, is always a balancing act between different needs,” he said. “If you wanted to have an answer on an endpoint that happens at a frequency of one 10th or one fifth the frequency of the primary endpoint, you would need a trial that is either 5 or 10 times larger or you’d need a trial that is 5 or 10 times longer to collect those events. Neither of these, I think, are acceptable in the current public need for knowing expeditiously that a vaccine works.”

Zaks added, “A 30 000 [participant] trial is already a fairly large trial. If you’re asking for a 300 000 trial then you need to talk to the people who are paying for it, because now you’re talking about not a $500m to $1bn trial, you’re talking about something 10 times the size. And I think the public purse and operational capabilities and capacities we have are rightly spent not betting the farm on one vaccine but, as Operation Warp Speed [the US government’s covid-19 vaccine plan] is trying to do, making sure that we’re funding several vaccines in parallel.”

Debating endpoints

Still, it’s fair to say that most of the general public assumes that the whole point of the current trials, besides testing safety (box 1), is to see whether the vaccine can prevent bad outcomes. “How do you reconcile that?” The BMJ asked Zaks.

“Very simply,” he replied. “Number one, we have a bad outcome as our endpoint. It’s covid-19 disease.” Moderna, like Pfizer and Janssen, has designed its study to detect a relative risk reduction of at least 30% in participants developing laboratory confirmed covid-19, consistent with FDA and international guidance.[21,22]

Number two, Zaks pointed to influenza vaccines, saying they protect against severe disease better than mild disease. To Moderna, it’s the same for covid-19: if its vaccine is shown to reduce symptomatic covid-19, it will be confident it also protects against serious outcomes.

But the truth is that the science remains far from clear cut, even for influenza vaccines that have been used for decades. Although randomised trials have shown an effect in reducing the risk of symptomatic influenza, such trials have never been conducted in elderly people living in the community to see whether they save lives.

Only two placebo controlled trials in this population have ever been conducted, and neither was designed to detect any difference in hospital admissions or deaths.[23] Moreover, dramatic increases in use of influenza vaccines has not been associated with a decline in mortality (box 2).[26]


By Peter Doshi, associate editor, The BMJ.

Sarah Tanveer helped research the design of studies and identify quotations, and Ulrich Keil provided comments on an early draft of this article.


  • Competing interests: I co-wrote an op-ed on this topic with Eric Topol, who is quoted in this article, I have been pursuing the public release of vaccine trial protocols, and I co-signed an open letter to the FDA calling for independence and transparency in covid-19 vaccine related decision making.
  • Provenance and peer review: Commissioned; externally peer reviewed.


    1. Shah A, 
    2. Marks PW, 
    3. Hahn SM. Unwavering Regulatory Safeguards for COVID-19 Vaccines. JAMA2020. doi:10.1001/jama.2020.15725. pmid:32766736 CrossRef PubMed Google Scholar
  1. Cavazzoni P, Marks P, Mayne S, et al. Senior FDA career executives: We’re following the science to protect public health in pandemic. USA Today. 2020 Sep 10. https://www.usatoday.com/story/opinion/2020/09/10/sound-science-to-meet-covid-challenges-fda-career-officials-column/5756948002.
  2. Office of the Commissioner. The FDA’s Oversight of Vaccines is Vital to Public Health. 2020. https://www.fda.gov/news-events/fda-voices/fdas-scientific-and-regulatory-oversight-vaccines-vital-public-health
  3. Hahn S. Remarks on the vaccine review process. 2020. https://www.fda.gov/news-events/speeches-fda-officials/dr-hahns-remarks-national-consumer-league-vaccine-review-process-09292020.
  4. Sanofi. Biopharma leaders unite to stand with science. Sep 2020. http://www.news.sanofi.us/2020-09-08-Biopharma-leaders-unite-to-stand-with-science.
  5. Palca J. What a nasal spray vaccine against covid-19 might do even better than a shot. NPR. 2020 Aug 28. https://www.npr.org/sections/health-shots/2020/08/28/906797539/what-a-nasal-spray-vaccine-against-covid-19-might-do-even-better-than-a-shot.
  6. Curwen T. Those coronavirus vaccines leading the race? Don’t ditch the masks quite yet. Yahoo Finance. 2020. https://finance.yahoo.com/news/those-coronavirus-vaccines-leading-race-120006184.html.
  7. Topol EJ. Paul Offit’s biggest concern about covid vaccines. 2020. https://www.medscape.com/viewarticle/936937.
  8. Moderna TX. Protocol mRNA-1273-P301, Amendment 3. 2020. https://www.modernatx.com/sites/default/files/mRNA-1273-P301-Protocol.pdf.
  9. Pfizer. PF-07302048 (BNT162 RNA-Based COVID-19 Vaccines) Protocol C4591001. 2020. https://pfe-pfizercom-d8-prod.s3.amazonaws.com/2020-09/C4591001_Clinical_Protocol.pdf.
  10. AstraZeneca. Clinical Study Protocol – Amendment 2 AZD1222- D8110C00001. 2020. https://s3.amazonaws.com/ctr-med-7111/D8110C00001/52bec400-80f6-4c1b-8791-0483923d0867/c8070a4e-6a9d-46f9-8c32-cece903592b9/D8110C00001_CSP-v2.pdf.
  11. Janssen Vaccines & Prevention BV. VAC31518 (JNJ-78436735) clinical protocol VAC31518COV3001 amendment 1. 2020. https://www.jnj.com/coronavirus/covid-19-phase-3-study-clinical-protocol.
  12. CDC. Coronavirus disease 2019 (COVID-19). 2020. http://web.archive.org/web/20200709001525/https://www.cdc.gov/coronavirus/2019-ncov/hcp/planning-scenarios.html.
  13. CDC. What to do if you are sick. 2020. https://www.cdc.gov/coronavirus/2019-ncov/if-you-are-sick/steps-when-sick.html.
  14. Moderna advances late-stage development of its vaccine (MRNA-1273) against covid-19. https://investors.modernatx.com/news-releases/news-release-details/moderna-advances-late-stage-development-its-vaccine-mrna-1273.
  15. National Institutes of Health. Phase 3 clinical trial of investigational vaccine for COVID-19 begins. 2020. https://www.nih.gov/news-events/news-releases/phase-3-clinical-trial-investigational-vaccine-covid-19-begins.
  16. Finn A, Malley R. A vaccine that stops covid-19 won’t be enough. New York Times. 2020 Aug 24. https://www.nytimes.com/2020/08/24/opinion/coronavirus-vaccine-prevention.html.
    1. Trogen B, 
    2. Oshinsky D, 
    3. Caplan A. Adverse consequences of rushing a SARS-CoV-2 vaccine: implications for public trust. JAMA2020;323:2460-1. doi:10.1001/jama.2020.8917. pmid:32453392 CrossRef PubMed Google Scholar
    1. Doshi P. Pandemrix vaccine: why was the public not told of early warning signs?BMJ2018;362:k3948doi:10.1136/bmj.k3948. FREE Full Text Google Scholar
    1. Avorn J, 
    2. Kesselheim A. Regulatory decision-making on covid-19 vaccines during a public health emergency. JAMA2020. doi:10.1001/jama.2020.17101. pmid:32870268 CrossRef PubMed Google Scholar
  17. International Coalition of Medicines Regulatory Authorities. ICMRA SARS-CoV-2 Vaccines Workshop #2 – Summary. 2020. www.icmra.info/drupal/en/news/22june2020/summary.
  18. Food and Drug Administration. Development and licensure of vaccines to prevent covid-19: guidance for industry. 2020. https://www.fda.gov/media/139638/download.
    1. Govaert TM, 
    2. Thijs CT, 
    3. Masurel N, 
    4. Sprenger MJ, 
    5. Dinant GJ, 
    6. Knottnerus JA. The efficacy of influenza vaccination in elderly individuals. A randomized double-blind placebo-controlled trial. JAMA1994;272:1661-5. doi:10.1001/jama.1994.03520210045030 pmid:7966893 CrossRef PubMed Web of Science Google Scholar
    1. Office of the Commissioner. FDA Insight. Vaccines (Basel)2020;COVID-19. https://www.fda.gov/news-events/fda-insight/fda-insight-vaccines-covid-19-part-1.Google Scholar
    1. Osterholm MT, 
    2. Kelley NS, 
    3. Sommer A, 
    4. Belongia EA. Efficacy and effectiveness of influenza vaccines: a systematic review and meta-analysis. Lancet Infect Dis2012;12:36-44. doi:10.1016/S1473-3099(11)70295-X pmid:22032844 CrossRef PubMed Web of Science Google Scholar
    1. Simonsen L, 
    2. Reichert TA, 
    3. Viboud C, 
    4. Blackwelder WC, 
    5. Taylor RJ, 
    6. Miller MA. Impact of influenza vaccination on seasonal mortality in the US elderly population. Arch Intern Med2005;165:265-72. doi:10.1001/archinte.165.3.265 pmid:15710788 CrossRef PubMed Web of Science Google Scholar

ORIGINAL SOURCE: “Will covid-19 vaccines save lives? Current trials aren’t designed to tell us”,
(Published 21 October 2020) Cite this as: BMJ 2020;371:m4037

^ Back to top ^


BREAKING: FDA announces 2 deaths of Pfizer vaccine trial participants from “serious adverse events”

First Pfizer coronavirus vaccines expected to land on Wednesday

COVID-19 Vaccine Bombshell: FDA Documents Reveal DEATH + 21 Serious Conditions As Possible Adverse Outcomes

FDA Briefing Document:

Positive association between COVID-19 deaths and influenza vaccination rates in elderly people worldwide

More than half in FDNY say they’ll refuse COVID-19 vaccine

COVID-19 vaccine recipients will not be exempted from self isolation:

Cancer risk associated with simian virus 40 contaminated polio vaccine

Which Industry Spends the Most on Lobbying?

Warning after two NHS workers have allergic reaction to Pfizer/BioNTech Covid vaccine

COVID-19: Four Pfizer vaccine volunteers develop Bell’s palsy

2019-2020 Preliminary In-Season Burden Estimate

Side effects from the COVID-19 vaccine means ‘your body responded the way it’s supposed to,’ experts say


Vaccine Safety to Remain Unclear Until Millions Get Their Shots

The three groups of people advised not to get the Pfizer COVID vaccine


Health Advisers Rename ‘Adverse Reactions’ to COVID-19 Vaccine

WHO Chief Warns Vaccine Won’t End Covid-19 Pandemic As Moderna, Pfizer Announce Early Successes

How Changing the Definition of Pandemic Altered Our World

October 2020 ACIP Meeting – Post-authorization safety monitoring plans

Related posts for further research:

The shocking reason why Pfizer’s coronavirus vaccine requires storage at -70C … because it contains experimental nanotech components that have NEVER been used in vaccines before

The shocking reason why Pfizer’s Coronavirus Vaccine Requires Storage at -70C … because it contains experimental nanotech components that have NEVER been used in vaccines before By Mike Adams (Natural News) You’re seeing the reports all over the news: Pfizer’s new coronavirus vaccine requires storage at -70C (-94F), which is much colder than the North…

Can These New COVID “Vaccines” Be Trusted?

The seven companies awarded vaccine contracts through “Operation Warp Speed” use new technologies never used before in vaccines for humans. Technologies that some say, will genetically alter those who receive it. As strange as this sounds that is just the beginning of a strange story. A great deal of secrecy surrounds the details of these…

New York Assembly Bill A11179: MANDATING COVID-19 VACCINE!


^ Back to top ^

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s