Great post, particularly on the protective effect of the flu vaccine for cardiovascular outcomes, which I didn't know about.
But...given the p value of the large RCT (https://pubmed.ncbi.nlm.nih.gov/34459211/) is 0.04 and the upper limit of the confidence interval is a hair's breadth from the line of no effect, does this need further replication before taken as gospel?
I note this trial was in a population who weren't that elderly (mean age 60) but who did have a recent MI which presumably needs to be taken into consideration when extrapolating?
On the large RCT, I agree that it shouldn't be taken as evidence alone and that there's uncertainty around how large the reduction is.
But some more relevant points: marginal p-values are mainly a problem if there's p-hacking or publication bias. With this trial, the details were pre-registered on clinical trials.gov (http://clinicaltrials.gov/show/NCT02831608) and you can see they followed the same protocols. Also, most of the outcomes had moderate effects and were significant, rather than just one or two.
In general, I think that since there's a clear mechanism (heart attacks and strokes -> death), it seems reasonable to extrapolate that it reduces all-cause mortality (supported by this trial) even though I'm less sure about the magnitude.
Re: the recent MI point, this was likely a way to increase power in the study - people with a recent MI would be more likely to have an MI during the trial, so it would require fewer people to see a difference between vaccine and placebo if there was an effect.
Great post, particularly on the protective effect of the flu vaccine for cardiovascular outcomes, which I didn't know about.
But...given the p value of the large RCT (https://pubmed.ncbi.nlm.nih.gov/34459211/) is 0.04 and the upper limit of the confidence interval is a hair's breadth from the line of no effect, does this need further replication before taken as gospel?
I note this trial was in a population who weren't that elderly (mean age 60) but who did have a recent MI which presumably needs to be taken into consideration when extrapolating?
Would be interested to hear your thoughts!
Thanks!
On the large RCT, I agree that it shouldn't be taken as evidence alone and that there's uncertainty around how large the reduction is.
But some more relevant points: marginal p-values are mainly a problem if there's p-hacking or publication bias. With this trial, the details were pre-registered on clinical trials.gov (http://clinicaltrials.gov/show/NCT02831608) and you can see they followed the same protocols. Also, most of the outcomes had moderate effects and were significant, rather than just one or two.
In general, I think that since there's a clear mechanism (heart attacks and strokes -> death), it seems reasonable to extrapolate that it reduces all-cause mortality (supported by this trial) even though I'm less sure about the magnitude.
Re: the recent MI point, this was likely a way to increase power in the study - people with a recent MI would be more likely to have an MI during the trial, so it would require fewer people to see a difference between vaccine and placebo if there was an effect.
Thanks - interesting that the original sample size calculation for was 4400 but cut short due to covid.
I guess if they got to this number we would have more certainty.
I wish someone would give you and Stuart like $10 million to found a new science magazine.
How common is asymptomatic flu?