Professor Patrick Holford of Teesside University and Head of Science and Education at Biocare enjoys his own special reality. You may recall that in The Great Gatsby, Fitzgerald describes newspaper reports of his hero’s murder as “grotesque, circumstantial, eager and untrue”. HolfordWatch feels the same way when reading some of Holford’s more outrageous interpretations of scientific literature.
Is “outrageous” a bit strong? No. Holford repeatedly demonstrates breathtaking ignorance of statistics yet has an unswerving belief in his own ability to interpret those of scientists and researchers who do understand statistical techniques and tools. Morever, he represents his mis-information as expertise to the surprisingly gullible lifestyle journalists who, too frequently, cover stories that are best left to journalists with some understanding of science or medicine. And, not least, he promotes himself as an expert to the public and sows disinformation.
It is no wonder that the statistical analyses in the Food for the Brain Child Survey 2007 are as poor and worthless as they are.
We offer you one straightforward example where Holford not only mis-interprets simple summary statistics but branches out to claim:
It really makes me question the integrity of the authors and the journal. Let’s explore that for a minute with a ‘conspiracy theory’ hat on. Last week pharmaceutical drug sales topped $600 billion. The number one best seller was Lipitor, a stin [sic] drug for lowering cholesterol… [Don't be fooled by the Omega 3 scam]
Just what is it that has provoked Holford’s faux theatrics and lead him to suggest the donning of tinfoil hats? It is his interpretation of summary data from Hooper et al’s systematic review of Omega 3 for mortality, cardiovascular disease and cancer.
It’s my job to read and analyse the whole paper and it soon becomes clear there is something very fishy going on…
[T]he authors…analyse all the studies using fish derived sources of omega 3. They include twelve studies, nine of which show a benefit, one of which shows no effect and two show a very small negative effect. However, combined, in the way that this review analyses the data, this seemingly obvious beneficial effect doesn’t come up as significant.
What I find particularly deceptive I that this obvious skew is not even discussed in the research paper. It really makes me question the integrity of the authors and the journal…
[Don't be fooled by the Omega 3 scam]
Well, if that is all that is underlying Holford’s concerns then maybe he should start planning a very classy apology to the authors of that systematic review and the BMJ. I’m not sure what scale of apology is appropriate when you have questioned the integrity of the authors on such insubstantial grounds but you might have your own opinions on that.
The reason that “this obvious skew is not even discussed in the research paper” is because it doesn’t exist. The fact that it doesn’t exist is apparent within half a second to anybody who actually understands how to interpret a forest plot (aka confidence interval plot or blobbogram). If you already know how to do this, please skip the next section, if not, please bear with us for a couple of minutes which is all it will take for you to understand this unbelievably simple form of data summary.
the blobbogram at a glance
You have systematically sorted through your records and other relevant sources and come up with a number of trials that meet your criteria. For each of those trials, you display your findings as a blob (your measured effect or point estimate of the effect of your intervention). You draw a horizontal line that extends either side of your blob to represent the results and their confidence intervals. The longer the line, the more uncertain that result and, conversely, the shorter the line, the more certain the result. The diamond represents the combined results of all the trials.
The vertical line marks the point about which the horizontal lines would cluster if the two treatments (typically an active treatment and a control group) that are compared in the trials had similar effects: it is the marker for no effect. If a horizontal line touches (or crosses) the vertical line, then that individual trial did not report results that show a clear difference between the treatments (this is in bold because it is important later). The value of this vertical line is 1: if a horizontal line neither touches nor crosses it, then the result is individually statistically significant.
And finally, when the diamond (the combined results of all the trials) is positioned nicely to the left of the vertical line it indicates that the treatment under consideration is beneficial. A diamond that is positioned to the right, of the vertical line would indicate that the control did more harm than good or was of greater benefit than the intervention.
In the above example, it is clear to see that three blobs have horizontal lines that cross the vertical line (their results are not statistically significant) and two blobs are positioned to the left of the vertical line and their confidence intervals do not cross the vertical line (their results indicate benefit and are statistically significant).
Some gratuitous comments that you don’t need to interpret this diagram but are generally useful. You may also have noticed that line for trial 4 has a blob on the left but crosses the line. This shows that the value of the point estimate looked as if it were indicating a benefit for the intervention, but the confidence interval (interval estimate) shows that if the trial were to be repeated, then the value might lie to the right of the vertical line, or anywhere along that range: it is not statistically significant. You may also notice that trial 3 has a long line; this indicates a wide confidence interval, and, even in the point estimate had been on the left of the line, you could not have claimed any significance for the results i) because it crosses the line of no effect (the vertical) and ii) you would have misgivings because of the range of the confidence interval. (And, although the blob was to the right of the line, this is not statistically significant.)
Back to the rough guide to blobbograms, you will see above that the diamond is nicely to the left of the vertical line which confirms that, overall, the treatment depicted in our illustration was of some benefit and that this aggregate is statistically significant.
back to holford’s analysis
The following diagram differs from the above because each of the horizontal lines has the name of a trial to the left of it, and the numerical range of the confidence interval for its results to the right (this is useful when something is so close to the line that you need to check the range to see whether it includes 1, and therefore crossed the vertical line and is therefore not statistically significant). We should also mention that the size of a blob in a horizontal line relates to the relative weighting of that study. E.g., the number of participants in that trial: crudely, a large blob shows a higher number, and a smaller blob a smaller number.
We shall repeat Holford’s analysis of this diagram of the impact of marine Omega 3 on mortality (consult the BMJ’s full figure rather than excerpt to see the table legends etc; click our thumbnail to see a larger version of the relevant section of the summary data).
[The authors] include twelve studies, nine of which show a benefit, one of which shows no effect and two show a very small negative effect. However, combined, in the way that this review analyses the data, this seemingly obvious beneficial effect doesn’t come up as significant.
Is Holford’s interpretation what you see? No? It looks to us as if 10/12 of those trials either touch or cross the vertical line and their results indicate no benefit. (GISSI-P is the 7th line and looks like it might touch, but, if you consult the confidence interval on the right, you will see that it does not cross 1 or the vertical line.)
Unsurprisingly, the diamond which is the aggregate of these trials, is towards the left of the line but also crosses it which is why the authors gave their nuanced conclusion. So, Holford is mistaken: there were 2 trials (not 9) with a positive (and significant) result and the others did not show a significant benefit. The “obvious skew” is in Holford’s interpretation, not in data summary nor in Hooper et al’s nuanced interpretation of the results.
The first trial, Burr, has a wide blob (large number of participants), a short horizontal line (you can be confident in the results) and lies to the left of the vertical line; it shows a positive effect. However, the second trial, Kaul, obviously has a small number of participants (small blob) and a lengthy horizontal line that crosses the vertical line. So, even though the point estimate of the blob is to the left of the vertical line, the range of values along which the true value may lie is large which means that the results are not significant.
At a glance, you can see for yourself that Holford’s interpretation of this diagram was egregiously wrong.
As for that “combined, in the way that this review analyses the data, this seemingly obvious beneficial effect doesn’t come up as significant” remark, we have a simple observation. There is no statistical rehabilitation camp where you can send data until you can compel them to report the result that you want. You can analyse those data any way that you like and, as long as you are paying due care and attention to the meaning of significance and confidence intervals, you will never get a different result. You can not distort results that are not significant to make them so. For those who have found that the little excursion into statistics has sapped their concentration and would prefer an aphorism, you can not make a silk purse out of a sow’s ear.
Once more, we refer you to the fine paper by Kruger and Dunning that explains why so many people have no grasp for the limits of their own competence or any appreciation for the relative competence of others.[2] However, this stark competence gap is the more remarkable when one considers that one of the authors whom Holford is criticising here is Professor Carolyn Summerbell, one of his ‘colleagues’ at the University of Teesside.
Aside from the substantial apology that he owes the authors of the systematic review, Holford’s lack of understanding of this basic statistical tool raises some uncomfortable questions. Just what skills will he impart to the young minds for which he will be responsible at the University of Teesside? Will he try and persuade any students that he supervises that they should interpret a blobbogram in the same way that he does?
Update 11 April In the comments, Dr Aust points us towards von Schacky and Harris discussion of the null conclusion of the Cochrane Review. There are some commenters such as Truswell who question whether the Cochrane Review process is appropriate for assessing what are essentially observational and cohort studies. These are legitimate issues that are soberly and thoughtfully discussed. On the other hand, we have Holford, criticising the authors on the basis of his interpretation of this figure and questioning the integrity of the review authors and the journal. And then we have his explanation that this is part of a drive to protect the market share of statins.
Overall, as the authors remark, many people (themselves included) expected an unequivocal result that would indicate an attractive, statistically validated relative risk rather than the (frankly) unimpressive RR of 0.87 with a confidence interval that mandates a note to the inclusion of unity. There are some researchers such as Brignell who advocate the strong position of discounting any RR that has a point estimate of less than 2 or, if risks are reduced, be below 0.5. Others argue that unimpressive RRs are acceptable for epidemiological findings where the numbers of population involved are so large that even a less than stellar RR will benefit large numbers of people so the question is then one of whether it is justified by risk, benefits and cost.
Beyond this, of course, we have to mention John Ioannidis: Why Most Published Research Findings Are False. Ioannidis writes:
The smaller the effect sizes in a scientific field, the less likely the research findings are to be true.…[R]esearch findings are more likely true in scientific fields with large effects, such as the impact of smoking on cancer or cardiovascular disease (relative risks 3–20)…than in scientific fields where postulated effects are small…(relative risks 1.1–1.5)… [I]f the majority of true genetic or nutritional determinants of complex diseases confer relative risks less than 1.05, genetic or nutritional epidemiology would be largely utopian endeavors.
Notes
[1] Hooper et al. strongly recommend that interested parties should read the full Cochrane Review from which they extracted some of the data and findings for this systematic review. Omega 3 fatty acids for prevention and treatment of cardiovascular disease (Review) (pdf) They we had already anticipated and assessed most of the subgroupings and individual outcomes that BMJ commenters suggest in the original Cochrane review.
As stated in the paper, the authors emphasise:
The overall result from the meta-analysis was a relative risk suggesting benefit of omega 3 intake on mortality, but a confidence interval including unity (total mortality RR 0.87, 95% CI 0.73 to 1.03 with significant heterogeneity) so that we are not quite so sure.
[2] You may be interested to read an overview of the Justin Kruger and David Dunning paper that discusses how difficulties in understanding one’s own incompetence can lead to inflated self-assessments.
What’s even more amazing is that when they then shared the performance of other participants with the people who performed poorly (hoping that they would then adjust their self-perception downward) people who scored poorly failed to adjust their self-perception of their performance. In other words, they are completely unaware of their own [in]competence, and can’t detect competence in others.
It really it a very helpful paper that explains many otherwise inexplicable actions.
A Photon in the Darkness likewise offers a helpful discussion of this paper: The Arrogance of Ignorance
32 responses so far ↓
Why Do Mainstream Media Promote Patrick Holford? « Holford Myths: what is the problem with Nutritionist Patrick Holford? // April 10, 2008 at 2:18 pm
[...] some thoughtful and spirited commentaries about the level of Holford’s scholarship that cast doubt on his ability to understand the research that he reads and cites with such abandon. Holford has written odd items about My Right to Be Called a [...]
superburger // April 10, 2008 at 5:17 pm
Do you know if Professor Patrick Holford of Teeside University, or his staff, read HolfordWatch?
I’m surprised that someone with a BSc in Experimental Psychology would apparently not know the basics of stats - imagine that type of thing is useful to budding psychologists.
I wonder if he will see fit to correct his website on the basis of HW pointing out his apparent errors.
If he does that’s excellent - but rather awkward for Prof Holford.
If he doesn’t I wonder how the more erudite employees of Teeside University will react to their visiting professors interpretation of statistics.
eveningperson // April 10, 2008 at 5:30 pm
I seem to recall it being pointed out (I don’t have the reference to hand but it was probably by Thomas Gilovich or Massimo Piattelli-Palmarini, who have both written accessible books on cognitive errors) that people seem to naturally feel that more and more insignificant (or even zero) results sum to more ’significant’ evidence for the desired conclusion. The ‘no smoke without fire’ effect.
In this case, Holford’s ’seemingly obvious beneficial effect’ will feel as if it trumps any valid statistical analysis.
dvnutrix // April 10, 2008 at 5:49 pm
Well, no lesser a person than Professor Holford himself has informed us that he is too busy writing books and helping people to be healthier to trouble himself with accuracy (I paraphrase, slightly).
Yes, there was rather more stats in the course than the young Holford enjoyed but his degree certificate is in plain ordinary Psychology, not Experimental Psychology. I don’t know why he cleaves to the Experimental claim…
Stephanie Fox claims that she and Holford will only entertain documented claims. She is such a whizz in the publishing industry that apparently blogs don’t count and links don’t count as documentation.
It is so funny - she obviously has no awareness of Holford’s PR company’s actvities in re: the wiki disaster of 2007 or the suspicions about Hickory and whether there is another sockpuppet in play or another meat puppet…
dvnutrix // April 10, 2008 at 5:52 pm
If Stephen Colbert does that sort of truthiness, it’s funny.
When a visiting professor at a UK university does it, it is inexpressably sad.
jdc325 // April 10, 2008 at 6:11 pm
“I wonder if he will see fit to correct his website on the basis of HW pointing out his apparent errors.”
They’ve amended something today actually - the title of the journal that carried the Hardman and Hart paper had been incorrectly rendered on the “Patrick’s Full Responses” webpage as ‘the Journal of Nutrition and Food Science’, rather than as ‘Nutrition and Food Science’. [The distinction is important, as the Journal of Nutrition and Food Science is a peer reviewed journal, indexed on Pubmed, whereas Nutrition and Food Science is not - which most of you were probably aware of before I was]
Which is why I was surprised to read of their stance here: “Stephanie Fox claims that she and Holford will only entertain documented claims. She is such a whizz in the publishing industry that apparently blogs don’t count and links don’t count as documentation.”
Perhaps Patrick’s errors are so numerous that his colleagues/staff simply don’t have time to correct them all? Which makes you wonder why Patrick puts so many of his thoughts in the public domain where they can be challenged.
dvnutrix // April 10, 2008 at 7:58 pm
jdc325 - rats, do you think we are enabling him in some bizarre humilation ritual?
If you are interested in another classic that comes under the heading of “making stuff up and hanging in on the nearest hapless journal paper”, might we interest you in Holford’s arguments for chromium and cinnamon?
Scroll down and enjoy the Kahn - it really is a classic.
Ben Goldacre // April 10, 2008 at 8:26 pm
Even by the standards of Professor Patrick Holford from Teesside University that really is a quite spectacular misunderstanding of basic undergraduate evidence based medicine.
anon // April 10, 2008 at 8:35 pm
Please be careful with this. Holford is not incorrect to say “results indicate no benefit”, if one interprets this as saying that their point estimate (center of their interval) is not close to the null value.
Of course, that’s not what a meta-analysis uses; its input is the range of values each study estimated were consistent with their respective data - their confidence interval.
I don’t think Holford understands the difference - but I don’t think he’s alone in that.
dvnutrix // April 10, 2008 at 8:55 pm
Hi anon - Holford said:
He is using his interpretation to “question the integrity of the authors and the journal…”. Do you think that his perspective is sustainable?
Gary Davis // April 10, 2008 at 10:53 pm
Generally, I like your thoughtful analysis of misinterpretation of statistical results. However, I was a little concerned when you wrote: “You draw a horizontal line that extends either side of your blob to represent the results and their confidence intervals. The longer the line, the more uncertain that result and, conversely, the shorter the line, the more certain the result.”
Because this is somewhat vague it is hard to argue with. However, if it means, as I suspect it does, that a particular small 95% confidence interval argues for a “more certain” result, then I think that’s wrong.
The general misconception is this: a particular 95% confidence interval does not, in general, contain the sample, population mean 95% of the time re-sampling is carried out. It might contain that mean around 80% of the time, but in the case of a binomial proportion it could be as low as 20%.
The confidence interval is a random variable, and on one instance tells you anything at all about about the certainty of the result.
dvnutrix // April 10, 2008 at 11:48 pm
Sure - that is a simplification for an over simple diagram. If you can provide a non-frightening link for interval estimates that is suitable for inclusion in a blog post I would be very pleased to pop it in.
The main point of this post is that this chap is a visiting professor who was using his interpretation of this diagram to question the integrity of the authors of the systematic review and the journal. Yet, his interpretation is flawed.
Mine is a trifle bland because I didn’t want to get frighten off the nervous by laying out all of the complexities. I have, before now, written blog posts that not only have extensive references and footnotes but further references for the footnotes. I’m trying to simplify but it is important to do that without sacrificing the necessary level of understanding to cope with the example.
I would be grateful for a better, simpler re-write of any sections that are raising alarms or “Hmms”. Or, “Yes, I see your point but would prefer this wording”.
dvnutrix // April 11, 2008 at 12:13 am
As an example of the sort of maths one typically encounters from Holford, I present another example where he was abusing people for failing to highlight what seemed to him to be a striking finding.
Just to choose one in an area in which Holford claims particular expertise, we looked at a section in his Sept. 2007 100%health newsletter: Is Taking Vitamin C For A Cold A Waste Of Time? (pp 4-5). Holford gives an overview of the recent Cochrane Review on the topic of vitamin C and colds: pdf for whole report; plain-language summary. Holford wrote:
And yes, that would have been an amazing headline and one that he would rightly lament the newspapers and other mainstream media somehow missing. Just think of the potential productivity increases at work and in schools if children could experience “up to a month” of fewer ‘cold’ days per year. Except, the authors didn’t write any such thing. Those figures do not “equate to up to a month less ‘cold’ days per year”. Holford’s ability to interpret and communicate numbers has long ceased to be amusing and become irritating.
Take a moment to think about it. If the reduction of “up to a month” is truly a reduction of 13.6% then think of how many days a year those children would have to have a cold to make that reduction true. What does your instinct tell you on this point without even engaging your mental arithmetic skills? Do you have a vague feeling that this would imply that these unfortunate children would have to experience more than 200 days of cold symptoms a year in order to experience a 13.6% reduction that amounts to “up to a month”.
What does the primary source, the actual Cochrane Review (pdf) say?
So, that’s a pooled estimate that is extrapolated from the relevant trials; the authors refer to 28 days as the upper estimate of cold morbidity in children under the age of 12 rather than the general range, and the reduction might be 4 days from that upper estimate. Just to belabour the point, 4 days is nothing like “up to a month”.
Yet, such mathematical interpretations are being promoted by a visiting professor in a UK university and used as a basis to attack the integrity of others or as foundation for allegations of bias. And, these ‘facts’ are being circulated in a newsletter that he charges membership fees to receive and that he promotes as being ‘the truth’.
anon // April 11, 2008 at 2:30 am
Hello dvnutrix
No, I don’t think Holford’s argument is tenable; his statement about Omega 3 being “obviously” beneficial is ridiculous. But to argue this successfully, one has to avoid over-egging things.
The description above makes a (literally) bold statement that because the meta analysis doesn’t “cross the vertical line” the indication is of “no benefit”.
You have to be a pretty ardent Neyman-Pearsonite to go along with that. The meta-analysis, of the available data, supports a small but beneficial effect - although it doesn’t make this “obvious” as Holford writes. The data do not support very big negative effects, or very big positive ones. The data are certainly also consistent with no effect - but that’s about as accurate as they allow you to be.
Now, if I was the FDA and terrified of making a Type I error in a final, binary decision, I’d accept the “no benefit” choice. But I’m not; and as nasty side-effects and/or massive cost of Omega 3 seem implausible, a more careful weighing of Type I/II errors seems appropriate.
More work also seems appropriate to figure out the possible confounding effect of publication bias in the smaller studies, or systematic bias in the big ones - for example why aren’t GISSI and Burr 2003 more concordant?
anon // April 11, 2008 at 2:38 am
sorry, typo with “does cross the line”
Dr Aust // April 11, 2008 at 9:21 am
Another discussion of the Hooper et al meta-analysis can be found in the paper by von Schacky and Harris here.
superburger // April 11, 2008 at 9:29 am
as an aside, I find it odd the Professor Holford of Teeside University did not contribute his insights to the BMJ’s rapid/rabid responses section, as he has been known to in the past (with predictable consequences.)
(Perhaps HW could remind readers of a time where Professor Holford has accused another scientist of vested interests, whilst regrettably forgetting to mention his own)
fascinating as his website may be, and as large as his mailing list no doubt is, Professor Holford must surely have a duty as a scientist (for he is very clear that he is, himself, a scientist) to raise these concerns in the published academic literature.
Rigorous, free intellectual discourse is the cornerstone of good science and Professor Holford should be encouraged to bring his observations of ‘obvious skew’ to the attention of his fellow scientists.
Perhaps Professor Holford has raised his allegations of “obvious skew” with Professor Summerbell, as their paths cross (literally, if not intellectually) on the campus of Teeside University?
jdc325 // April 11, 2008 at 11:07 am
Slightly off-topic, but: “…and as nasty side-effects and/or massive cost of Omega 3 seem implausible…”
Hm. Isn’t there at least a theoretical danger that there could be an increased risk of haemorrhagic stroke (or of bleeding to death after an accident)? That at least *seems* plausible to me, as n3 fatty acids reduce platelet aggregation - although Gerster seemed to think the concern over haemorrhagic stroke risk was unfounded.
I agree that the cost of fish oil should not be ‘massive’ - but could it be more cost-effective to get one’s Omega-3 from eating fish as part of a healthy balanced diet? The fish purchase would be part of your regular grocery bill, rather than involving an extra layout of cash.
Actually, you could eat 100g of tinned salmon per day for about £6 per month and that would provide ~1.1g EPA/DHA. The equivalent in fresh mackerel would be about £9 per month. The equivalent in capsules would cost £10 per month. Eating fish would seem to be a much more cost-effective way of obtaining Omega-3 than is consuming capsules. It’s funny how people are often prepared to spend more on pills than on food: http://www.badscience.net/?p=299
dvnutrix // April 11, 2008 at 12:09 pm
I have a lecturer somewhere who groaning and saying, “I brought them up to be more hugger-mugger than proscriptive, where did it all go wrong?”
Are you OK with the later re-word of that point which is:
The authors of the review were surprised by their own results and commented on it in the Rapid Responses that follow.
The authors do comment on the various studies as the inclusion of Burr 2003 excited some controversy.
Overall my point is that there were no grounds for questioning the integrity of the authors and the journal.
Woobegone // April 11, 2008 at 12:11 pm
Ah, but you have to consider that the long term safety of mackerel has yet to be established. There’s evidence that mackerel consumption raises the risk of choking on a fish bone, and it’s been linked to your kitchen smelling a bit fishy all day.
dvnutrix // April 11, 2008 at 12:17 pm
Superburger, where you at all thinking of the turmeric discussion ? Challenged about his conflicts of interest, Holford attacks Prof Colquhoun, arguing that Colquhoun “has so far not felt it relevant to mention his own competing interests and financial involvements with the pharmaceutical industry”. Now, this might lead one to believe that Colquhoun is hiding his interests in the evil pharmaceutical industry - in order to have a go at a nutritionist. However, Colquhoun didn’t declare any competing interests because, um, he doesn’t have any (unlike Holford). Colquhoun explains that his “research has never been funded by the drug industry, but always by the Medical Research Council or by the Wellcome Trust“.
And yes, enquiring minds are agog to know what the excellent and estimable Professor Summerbell makes of Holford feeling compelled to ‘question her integrity and that of the journal’.
dvnutrix // April 11, 2008 at 12:33 pm
jdc325 - rather excitingly, Dr Damian Downing proposes a different conspiracy theory to Holford’s in the Rabid Responses:
And yes, I would agree that eating fish looks like it would be a better benefit and more efficient use of one’s budget.
dvnutrix // April 11, 2008 at 12:35 pm
Woobegone - I respectfully point you towards gravad lax. No actual cooking as opposed to sitting around waiting for it to cold cook during which time it can be covered to keep out the smells.
dvnutrix // April 11, 2008 at 12:51 pm
Dr Aust - there are some well-argued reasons to dispute the overall result of this review and the controversy is understandable. It is legitimate to argue the toss (as Truswell did) about whether the Cochrane review process is appropriate for assessing this sort of issue.
However, I don’t consider it legitimate to start questioning people’s integrity based on Holford’s interpretation of that blobbogram. Nor was it a sufficiently robust point to sustain his extended riff into the statins conspiracy.
LeeT // April 11, 2008 at 1:35 pm
“Actually, you could eat 100g of tinned salmon per day for about £6 per month”
You can get two Salmon fillets in the Sainsbury’s Basics range for less than £2 pack. Own brand tins of sardines cost 35p. To be on the safe side a couple of Omega 3 eggs as well. That works out at about £7.20, more interesting than tinned salmon.
Perhaps we could have an Omega 3 dinner party?
oily // April 12, 2008 at 3:19 pm
But don’t forget the 2004 advice on oily fish http://cot.food.gov.uk/pdfs/fishreport2004full.pdf
draust // April 12, 2008 at 6:13 pm
Agreed 100% about Holford. He doesn’t understand the stuff he cites, and more dangerously still he doesn’t understand that he doesn’t understand it (Kruger & Dunning etc). More dangerous still, he is given a forum to promulgate his understanding-free certainties by credulous journalists who don’t understand that Patrick doesn’t understand that he doesn’t understand, or indeed understand… etc etc.
I think the point about the Fish Oil thing specifically is that a scientist reading the Hooper et al review and its responses would have thought something like:
“Hmm.. interesting. There is obviously a bit of “wiggle” here, and dispute about what the various studies do and don’t mean or imply - plus a confusion between 2ndary and primary prevention. So all in all it looks like fish oils might do something, but we don’t really understand under precisely what conditions and for which people… so there would be no grounds to recommend them for everyone…
So all in all, I think personally I’ll just keep on eating fish twice a week because it tastes good.
Plus when they say “more studies are needed”, that’s really true since there is some indication more studies might really tell us something useful.”
This last one is an important distinction, I think. The line “more and better studies are needed” appears at the end of vast nos of Cochrane and similar systematic reviews and meta-analyses, but it doesn’t always means the same thing. How one views it depends on what I would call the “prior probability” of the thing under test.
For instance: “studies of distant healing… …available studies small and often poorly controlled… no clear evidence of benefit… more better studies are needed”
…would mean to most scientists “Seriously, don’t waste your time, the trials that have been done show nothing and the utter implausibility of it means only an Alt-Nutjob would go any further down this dead end”.
But it is dead right, I think, that PH’s thought process on looking at the meta-analysis is not the same as a scientist’s. His seems quite clearly to be more:
“Omega-3 fish oils are and good for you. I know this because they are natural. And I read some studies that someone said said they were good, so it must be true AND I can tell people it is “based on evidence”. So if this review argues against this, the authors must be Shills of Big Pharma and have fixed the results, though I don’t really understand how…
Ah, but here in the comments thread some pro-fish oil person says this, which I can cut and paste…”
In essence, then, PH reads these reviews in exactly the same way as the journalists who swallow his schtick. As you and Holford Myths have set out so clearly, PH’s “achievement” is managing to convince the media that he is a bit like a scientist, while actually he is nothing like a scientist and a lot like a (bad) journalist.
Patrick Holford and Scientific Accuracy and Attention to Detail: He Needs It « Holford Watch: Patrick Holford, nutritionism and bad science // May 18, 2008 at 2:19 am
[...] regularly accuses others of inaccuracy and questions the integrity of leading researchers such as Professor Carolyn Summerbell (coincidentally, a senior academic at the University of Teesside; we should point out that Holford [...]
Patrick Holford Claims to Eat Like a Gorilla. Seriously?!? « Holford Watch: Patrick Holford, nutritionism and bad science // June 17, 2008 at 5:02 pm
[...] Former Visiting Professor Patrick Holford and Head of Science and Education at Biocare has his own special and alternate reality. A personally-constructed reality in which he has a worthwhile opinion when [...]
Patrick Holford Refers to Himself As An Expert (No-One Else Was Volunteering) « Holford Watch: Patrick Holford, nutritionism and bad science // June 22, 2008 at 10:23 pm
[...] work for Optimum Nutrition Made Easy. Parts 1, 2, 3 and 4. Regular readers are accustomed to Holford’s alternate reality (and part 2) so will not be surprised that he refers to himself as an expert in [...]
Patrick Holford, Dr Richard Halvorsen and The Vaccination Schedule As a Risk for Asthma « Holford Watch: Patrick Holford, nutritionism and bad science // August 11, 2008 at 3:53 pm
[...] people’s potential conflict of interest that he identifies them when none exist and even indulges in a conspiracy theory when it serves his purpose. However, although Holford mentions that Halvorsen operates the private [...]
Down for Everyone or Just Me: Use It People, Use It Before Generating Conspiracy Theories « Holford Watch: Patrick Holford, nutritionism and bad science // November 22, 2008 at 4:02 pm
[...] to use an archive service such as Wayback and learn about URL validation. If you could throw in the ability to interpret blobbograms, systematic reviews etc., that would be a bonus. But, baby [...]
Leave a Comment