Fact check: Are your chances of getting the flu after a vaccination only decreased by 1 to 2 per cent?

Updated Thu at 11:33am

The claim

Australia is well into the midst of winter, and with the cold weather comes flu.

Last year saw the highest levels of flu activity in Australia since the 2009 swine flu pandemic.

This year Australians wanting to be vaccinated against the flu mobilised in unprecedented numbers, forcing the Federal Government to order an additional 800,000 vaccines.

But the effectiveness of the vaccine was called into question by professor of Public Health at Bond University, Chris Del Mar, in an interview with Neil Mitchell on 3AW.

“It actually reduces the chance of getting flu by about 1 to 2 per cent — that’s laboratory confirmed influenza. It makes a tiny difference.”

So, are your chances of getting the flu after a vaccination only decreased by 1 to 2 per cent? RMIT ABC Fact Check finds out.

The verdict

Professor Del Mar’s claim is misleading.

The source of Professor Del Mar’s claim is a systematic review of studies measuring the efficacy of flu vaccine in preventing the flu in healthy adults.

That review found the flu vaccine decreased the rate of flu infection from 2.3 per cent to 0.9 per cent in healthy adults.

Experts contacted by Fact Check said while this showed an absolute drop in the infection of 1.4 percentage points, using these numbers to talk about effectiveness was misleading.

Professor Del Mar should instead have talked about the relative drop in infection, which is the effectiveness rate of the vaccine.

The very study quoted by Professor Del Mar found that the flu vaccine had an estimated effectiveness rate of 59 per cent — not 1 to 2 per cent.

That means for an individual who has been vaccinated, the risk of getting the flu is more than halved.

This is supported by other studies cited by the Australian Department of Health’s Immunisation Handbook, which found similar effectiveness rates.

What is the flu?

Influenza, or ‘the flu’, is a respiratory disease that causes symptoms including fever, cough, muscle aches, headache, nasal congestion, sore throat and fatigue.

It is estimated to cause an average of 13,500 hospitalisations and 3000 deaths in Australia each year.

Three types of influenza affect humans — type A, B and C. Influenza A and B are responsible for most infections in humans.

The flu is further classified into ‘strains’ of which there are many and which are constantly mutating and changing.

According to the Australian Immunisation Handbook, published by the Department of Health, the period of peak influenza circulation is typically between June and September for most parts of Australia, though flu could occur year-round in tropical regions.

How does the vaccine work?

New flu vaccines are developed in preparation for each flu season in both the northern and southern hemispheres.

Flu vaccines can either be inactivated — where the virus is essentially “dead” — or live attenuated, where the virus is alive, but weakened.

All flu vaccines currently available in Australia are inactivated.

Flu vaccines typically contain three to four strains of the flu — two type A strains and one or two type B strains.

As different strains of influenza circulate each flu season, a certain amount of guesswork goes into deciding which strains will be included in the vaccine.

In February and September each year, the World Health Organisation gathers experts and influenza centres from around the world in order to make recommendations on the composition of the next flu vaccine.

Once the World Health Organisation has made its recommendation, the Australian Influenza Vaccine Committee meets with the Therapeutic Goods Administration to confirm which strains will be included in the Australian vaccine.

In addition to the standard flu vaccine, this year there are enhanced flu vaccines available to elderly Australians.

These vaccines include three strains of the virus rather than the usual four, and are designed to target the strains most commonly affecting older people.

The context of the claim

Professor Del Mar’s interview on 3AW came days after a speech he made at the GPs Down Under Conference on the Gold Coast, where he questioned whether the flu vaccine was being oversold.

Multiple news outlets reported that in his speech Professor Del Mar pointed to a review “that found seasonal flu vaccines only reduced the rate of laboratory confirmed influenza cases by between 1 to 2 per cent”.

In an article in The Conversation, Professor Del Mar and his co-author, Professor Peter Collignon, again repeated the claim that flu infections are only reduced by one per cent.

“[A review] found the vaccine reduced the chance of getting laboratory-confirmed influenza from 23 cases out of 1,000 to 9 cases out of 1,000.

“While this seems to be a reduction of more than 50 per cent, that seems less optimistic expressed in absolute terms.

“The infection rate in adults drops from 2 per cent per year to 1 per cent. You could say that’s halved, but it effectively only drops by 1 per cent.”

The source of the claim

The review quoted by Professor Del Mar in the article in The Conversation is one of a series of three reviews into the effectiveness of the flu vaccine.

Those reviews, which look at flu in healthy adults, healthy children and the elderly come from Cochrane, an independent organisation that produces systematic reviews of medical research in order to make that research more accessible.

The Cochrane reviews referenced by Professor Del Mar compared a number of studies estimating the effectiveness of flu vaccine in order to come up with a single, overarching estimate of effectiveness.

“The first Cochrane review looked at the effects of the influenza vaccine in healthy adults from 25 studies conducted over single influenza seasons in North America, South America, and Europe between 1969 and 2009,” the article in The Conversation states.

What the report says

That Cochrane review found that without vaccination 23 healthy adults per 1000 would get the flu while for those who had been vaccinated nine per 1000 would get the flu.

In other words: “Inactivated influenza vaccines probably reduce influenza in healthy adults from 2.3 per cent without vaccination to 0.9 per cent.”

In their conclusions, the Cochrane reviewers state:

“Healthy adults who receive inactivated parenteral influenza vaccine rather than no vaccine probably have a one per cent lower risk of experiencing influenza over a single influenza season (2.3 per cent versus one per cent, moderate-certainty evidence).”

The review also states:

“Inactivated parenteral vaccines probably have 59% efficacy in preventing confirmed influenza.”

What that means for the infection rate

Experts told Fact Check the Cochrane review showed the risk of getting the flu was halved by the vaccine.

Dr David Price, a biostatistician at the University of Melbourne, told Fact Check: “What these numbers mean, is that for every 1000 people, 23 would get flu without a vaccine programme. However, with a vaccine programme, the number of people who would get flu reduces to 9 in 1000.”

The difference in these values, Dr Price explained, showed that for every 1000 individuals vaccinated, 14 flu cases were prevented.

“It appears as though Professor Del Mar may have misinterpreted this figure as the vaccine effectiveness, which it is not. The vaccine effectiveness is actually around 59 per cent,” Dr Price said.

“A fairer statement in referring to the difference between 0.9 per cent and 2.3 per cent would be to refer to a reduction of 1.4 percentage-points. When thinking about this difference as a relative difference… we can see that the total number of cases as a result of the vaccine has more than halved, and thus sounds far more effective than stating that the vaccine only reduces influenza by one to two per cent.”

Associate Professor Mark Chatfield, a biostatistician from the University of Queensland, told Fact Check:

“With two proportions, you can talk about absolute differences or relative differences or both. He talks about absolute differences. I think vaccine efficacy is more often expressed as a relative difference.”

Associate Professor Chatfield said both numbers should always be given in order to be clear when talking about two proportions; in this case Professor Del Mar should have specified the drop was from 2.3 per cent to 0.9 per cent.

“The public has probably been misled when [Professor Del Mar] gave the smaller absolute difference, rather than the larger relative difference, without saying whether he was meaning absolute or relative, which conveniently added to his argument.”

The Cochrane Review showed the flu vaccine halved the risk of getting the flu, Associate Professor Chatfield said.

Dr Rob Menzies, a senior lecturer from the School of Public Health and Community Medicine at UNSW, also agreed that the review showed the risk of getting the flu was more than halved.

“Using the figures in the systematic review, the vaccine reduced disease from 2.3 per cent to 0.9 per cent, which is a roughly 60 per cent reduction. This is very similar to other published estimates of effectiveness of flu vaccine of 59 per cent against laboratory-confirmed influenza in healthy younger adults.”

Is the Cochrane Review reliable?

The Cochrane Review found 2.3 per cent of people had laboratory confirmed influenza — flu confirmed by a diagnostic test.

Professor Allen Cheng, the director of the Infection Prevention and Healthcare Epidemiology Unit at Alfred Health, explained that using this figure as a baseline ignored unconfirmed flu infections.

“Not all patients get a diagnostic test after getting influenza infection, and the test isn’t completely sensitive, for example if the swab isn’t taken well [it may not give accurate results],” Professor Cheng said.

“How many people with flu like symptoms went to to the GP or hospital and got a flu test? Clearly not every person with the flu.”

Professor Robert Booy, head of the Clinical Research team at the National Centre for Immunisation Research and Surveillance, told Fact Check that with a lack of diagnostic testing, experts could only estimate that five to 10 per cent of the Australian population would get the flu in any given year.

He also had concerns about bias in the Cochrane reviews.

“They’re all done by the same people — the three reviews all came from the same stable so they always have the same biases,” Professor Booy said.

“There’s a concern that from the outset [the Cochrane reviewers] have had a subjective, nihilistic, negative view about the vaccine.”

This concern was shared by Professor Raina MacIntyre, who is the head of the Biosecurity Research Program at the Kirby Institute who told Fact Check:

“Despite the assertion that Cochrane reviews are ‘independent’ they are subject to the same personal biases as any other research study. Cochrane reviews can show opposite results based on who does them. They have been re-analysed (using the same data) by one group to get completely different results from another group.”

So, how effective is the flu vaccine?

Dr Menzies told Fact Check:

“The best authoritative source on vaccine effectiveness is the Australian Immunisation Handbook.”

The Australian Immunisation Handbook is developed by Australian Technical Advisory Group on Immunisation (ATAGI) and approved by the National Health and Medical Research Council (NHMRC).

According to the handbook:

“The efficacy and effectiveness of influenza vaccines of similar composition depends primarily on the age and immunocompetence of the vaccine recipient, and the degree of similarity between the virus strains in the vaccine and those circulating in the community.”

The handbook refers to a number of systematic reviews and studies into the efficiency of flu vaccine.

One such study, Influenza Vaccine Effectiveness in the United States during the 2015-2016 Season, found quadrivalent vaccines (vaccines with four strains of flu) to be 54 per cent effective.

A systematic review cited by the handbook found trivalent vaccines (three strains) to be 59 per cent effective, though this varied dependent on the season.

Professor MacIntyre added that the potential benefits of flu vaccine extend beyond preventing flu itself.

“There is also evidence that flu vaccine prevents heart attacks, because flu affects the clotting of the blood and the lining of the arteries through various mechanisms.

[The flu vaccine] can have a significant impact on preventing illness, hospitalisation and death, and experts around the world agree on this.”

Principal researcher: Ellen McCutchan

@EllenCutch

factcheck@rmit.edu.au

Sources

• Flu shot ‘oversold’: Medical expert says Australians better off beating the flu with masks, 3AW, 5 June, 2018
• The ‘flu’ vaccine: how it’s made and how it works, Queensland Health, 29 March, 2018
• 2018 seasonal influenza vaccines, Therapeutic Goods Administration, 27 March, 2018
• Fact Sheet: Influenza Vaccines, NCIRS, May, 2018
• The flu vaccine is being oversold – it’s not that effective, The Conversation, 5 June, 2018
• Vaccines for preventing influenza in healthy adults, Demicheli, V et al., 1 February, 2018
• Australian Immunisation Handbook, Department of Health, 1 August, 2017
• Influenza Vaccine Effectiveness in the United States during the 2015-2016 Season, Jackson M L et al., 10 August, 2017
• Efficacy and effectiveness of influenza vaccines: a systematic review and meta-analysis, Osterholm M T et al., 26 October, 2011
• Influenza Vaccination of Pregnant Women and Protection of Their Infants, Shabir A. et al., 4 September, 2014
• Influenza vaccination during pregnancy for prevention of influenza confirmed illness in the infants: A systematic review and meta-analysis, Nunes M C et al., 4 March, 2018
• Most people don’t benefit from vaccination, but we still need it to prevent infections, The Conversation, 8 June, 2018
• Building Influenza Surveillance Pyramids in Near Real Time, Australia, Craig B Dalton et al., November, 2013