The slippery stats on flu vaccines

By Alan Cassels, November 2015

Are flu shots 60 percent effective? Or are they 3 percent effective in a good year and 1 percent in a bad year?

It’s the season when the flu sound bites are flying around like sneezes in a windstorm, threatening to infect anything in contact. Despite all the media discussion of mutating viruses, hand hygiene and anti-flu drugs, there is a common punchline to all the chatter: “The best thing you can do to protect yourself is get the flu shot.”

According to the US Centre for Disease Control the flu vaccine this year will be “50 to 60 percent effective in preventing flu in those who get their shots,” a level of effectiveness that often gets repeated by Canadian public health officials.

Closer to home, officials from the BC Centre for Disease Control are in the news, talking up the numbers and usually providing at least one photo op of an official rolling up her sleeve for the needle—literally taking one for the team.

Despite the campaign, the discussions on ways to best protect ourselves from viruses are narrow and shallow, with lots of unknowns. One surprising story originating from BC researchers last year said that people who get repeated vaccinations may actually undermine the vaccine’s effectiveness. 

The number that stands proud and tall this year is “60” as in “the vaccine reduces the risk by 60 percent!” 

Sixty sounds good. Impressive. Powerful. More than half! But that number is as uninformative as a used car salesmen plastering “60 percent off” signs on every car on the lot, without him ever telling us what the regular prices are.

People hearing “60 percent reduction” picture this inside their thought bubble: “If my risk of getting the flu this winter is 100 percent, the shot will reduce that to 40 percent. So instead of 100 people getting the flu, if everyone was vaccinated only 40 would get it. Hmmm. This 60 percent reduction sounds like great odds.”

Dr Danuta Skowronski at BC’s Centre for Disease Control echoed those reductions when she told Canadian Press in October that the shot could provide up to a 50 percent reduction in flu risk. She told CP: “If I were to give to you a discount coupon that gave you 50 percent off your grocery bill at the checkout stand, you would probably think that was great. And in the same way, a 50 percent reduction in influenza risk is really important.”

But, in terms of influenza, we still don’t know what the baseline is.

Centre for Disease Control vaccine effectiveness (VE) studies commonly measured laboratory confirmed flu illness that results in a doctor’s visit or urgent care visit as an outcome. So a VE estimate of 60 percent means that the flu vaccine reduces a person’s risk of developing a laboratory-confirmed flu illness that results in a visit to the doctor’s office or urgent care provider by 60 percent. 

Those are significant qualifiers, usually left unstated. Most of us don’t bother seeking lab tests or medical help when we come down with a flu-like illness. So we are left out of those stats completely.

Last year’s vaccine effectiveness, Skowronski told Victoria News, was about 23 percent (again this likely only refers to those lab-verified cases that didn’t end up in a doctor’s office or hospital). Calling it “stupendously bad,” she blamed it on an “under-performing H3N2 component of the vaccine,” that didn’t match the strain in circulation.

And that’s the nub of it: Predicting the dominant strains that will be circulating in any given year, 6-8 months in advance, is a crap shoot.

For more information, I called Dr Tom Jefferson of the Cochrane Collaboration in Rome. He is probably the world’s foremost expert on the evidence behind the influenza vaccine. The Cochrane Collaboration is a global independent network of researchers, professionals, patients and others interested in health. Free from corporate and governmental influence, its mission is to organize medical research information in a systematic way to allow health professionals, policy-makers, and the public to make evidence-based decisions.

Dr Jefferson reminds me that every flu season has over 200 circulating viruses which can cause influenza and influenza-like illness, all perfectly capable of making you headachy and feverish. The best evidence shows that up to 40 percent of acute respiratory infections and influenza-like illnesses have no recognizable cause, so none of these would be included in CDC-type vaccine effectiveness stats. Thankfully, most people who catch one of these viruses end up just fine and the risk of death or serious illness in otherwise healthy people is rare. 

When Jefferson and his colleagues published Cochrane’s March 2014 review, they found that under ideal conditions (when the vaccine matches the main viruses circulating that season) you need to vaccinate 33 healthy adults to avoid one set of influenza symptoms. This is what we’d call a NNV (Numbers Needed to Vaccinate) of 33. When the vaccine match isn’t very good (as it was last year) the NNV is about 100. Put another way, of 100 people vaccinated, 99 will see no benefit and one person will avoid one set of influenza symptoms. Flu vaccination did not seem to affect the number of people hospitalized or who lost working days. Overall this means the vaccine offers between one and three percent effectiveness. 

As for the magical 60 percent, Dr Jefferson didn’t mince words: “Sorry, I have no idea where the 60 percent comes from—it’s either pure propaganda or bandied about by people who do not understand epidemiology. In both cases they should not be making policy as they do not know what they are talking about.”

So what is Canada’s policy? Generally, all the official government-sanctioned advice lines up in a single sound bite: Get the flu shot. Canada’s Public Health Agency is currently revving up its annual flu campaign and provincial health authorities are readying their staff to deliver up to 12 million doses of the flu vaccine. In BC the vaccine is publicly subsidized (I never use the word “free” because taxpayers are still paying for it) for a wide variety of people: seniors, their caregivers, children, people who look after children, corrections officers, aboriginals, pregnant women, and, of course, health care workers, for whom the vaccine is not just recommended, it’s required.

For the 2015-16 flu season, BC Centre for Disease Control states that to date, BC has spent $9.1 million on flu vaccines, an amount that is almost double what it spent four years ago. When you add on what people pay privately for the vaccine, plus all the administration and clinic costs (the doctors, pharmacists and nurses’ salaries and fees to give you the needle), the overall cost might well be double. (No one from the BC Ministry of Health could tell me what all the flu vaccine administration costs add up to.) One thing is clear: this is a hugely expensive venture which probably exceeds $20 million per year.

Totally worth it, right? Hmmm. 

Most flu vaccine cheerleaders are not paid stooges to the drug industry; they are public health officials clearly wanting to do the right thing. They will say that even if the numbers are small—if it’s only effective in one to three percent of vaccinated people—this still means a lot of people are protected from getting a deadly flu. Maybe, but I wonder where there might be a bigger payoff for an investment of less than that $10 to $20 million.

Dr Lisa Polinsky also wonders. She’s a Victoria naturopathic physician who faces flu vaccine questions from her patients every autumn. Her take on immunity is that while the flu vaccine might be effective for the few strains of dominant virus floating around, she counsels her patients to think about the bigger picture of overall innate immunity. Her advice: “Don’t go narrowband, go broadband.” Clearly she’d like to see more research and emphasis on a variety of areas—perhaps Vitamin D or probiotics—that hold a lot of promise to strengthen overall human immunity to fight off viruses, while contributing to a person’s general wellbeing. 

She tells me, for example, that Vitamin D deficiency is poorly understood and we need way more understanding of what are optimal levels, whether people in our northern latitudes benefit from supplementation, or if adding vitamin D rich foods to our diets would help. In addition to that, she says, “There’s also a lot more we need to understand about the microbiome, essentially the bacterial colony that resides in your gut where 70 percent of your immune system is hosted.”

When people understand the very marginal benefits of the annual flu shot, as well as the rare risk of harm (previously reported in Focus, October 2012; with a response by BC’s Provincial Officer of Health in November 2012) it might start a different discussion. “Patients ask ‘what else can I do?’” says Dr Polinsky. “This is what starts the bigger conversation.” 

When you think of the tens of millions of dollars BC pours into its annual flu campaign, surely a wider public conversation over whether we need to continue funding an under-performing flu vaccine, delivered at high cost, is needed. Are other immune “protection” interventions worthy of public health dollars? Maybe once the public starts to understand that the meme “a flu shot is your best protection” is a gross exaggeration—some might even say a total con job—we can properly consider the next question: “now what?”

Alan Cassels is a pharmaceutical policy researcher in Victoria and author of the just launched book, The Cochrane Collaboration: Medicine’s Best Kept Secret.