Holford Watch: Patrick Holford, nutritionism and bad science

We are going to look at something interesting about the relationship between Food Is Better Medicine Than Drugs (FIBMTD) and the Food for the Brain Child Survey 2007. The former was written by Holford & Burne, and the latter by Holford & Fobbester.

The Food for the Brain Child Survey 2007 (pdf) is irredeemable. There is nothing to be done to salvage the report because the data collection and analyses are seriously flawed. There are serious issues concerning the survey questions and this is yet another object lesson in why research must be conducted using standardised and validated questionnaires. The data were farcical even before the revelation that Professors Holford, Philip Cowen and David Smith (amongst others) want us to believe that something of statistical significance can emerge from a comparison of 2 outlier groups from an unrepresentative survey of 10,222 children. Outlier groups of 32 and 42 children which, when added together, make up 0.72% of the children (or 2.36% or so if you are just looking at the SAT group, but the point holds that these 74 children are the entire prop for the dietary recommendations).

Don’t even trouble your head as to whether there has been a clustering technique to take account of any children from the same families. Every survey taker wants to highlight 0.72% of their total unrepresentative sample and pronounce on public and research policy on the back of that. However, more about this at another time and if you are interested, you might consult our multi-part review of the Food for the Brain Child Survey 2007 (references below).

So, we offer our comments as part of a commentary on how not to report research and the way in which proponents of nutritionism mislead the public; the kind view of this would be that it is episodic incompetence. However, you can decide for yourself whether there is an enduring trend by looking at the entire corpus of someone’s work. We invite you to do that: we may even have some pointers available on our Holford Myths page.

We are astounded that Holford claims to be unaware of problems concerning the literature overview in the FFTB report. The literature overview is spectacularly disappointing: we offer our summary. What is especially depressing about this lamentable overview is that it looks to have been adapted from Chapter 16 of Food Is Better Medicine Than Drugs (FIBMTD) by somebody who had absolutely no degree of understanding for the comparative nuance that the author of that section used, nor for the ballpark relevance of the literature that is cited in the context of a discussion about children with ADHD rather than the general population of children (as per the FFTB Child Survey 2007). We disagree with FIBMTD’s extravagant claims of “proof” and some of the nuance is wrong. But given the standards with which we are familiar, it is peculiarly gratifying to see that, at some point, somebody understood that distinctive groups of children who participated in the studies that they cite. We shall look at the FIBMTD version because it is instructive and gives a clue as to how some people fail to understand nuance. And fail to understand that some terms are not semantically equivalent and can’t be used just because you consider them to sound more science-y - they do sound more science-y, and that’s a problem if it really can’t be justified.

If you want to see what the FFTB said, and our response, consult: our recent response or our original FFTB: Part 1. For this post, we are highlighting the difference between the original text written for FIBMTD and the text as it appeared in FFTB. Chapter 16 of Patrick Holford and Jerome Burne’s Food Is Better Medicine Than Drugs (FIBMTD).

Now, thanks to the ground-breaking research of pioneer researchers…we have proof based on a series of double-blind trials of the benefit from supplying extra essential fats to children with learning and behaviour problems….[Text cites refs 2 and 3.] [Emphasis added.] [pg. 322]

Yes and no. We loathe the use of the word “proof” and would doubt that the authors of the papers under discussion would approve of it: “proof” is a substantial over-statement of the quantity and quality of the evidence (Equazen were reprimanded by the ASA and received a severe drubbing for submitting these papers as part of their argument for efficacy). These issues should have been self-evident to Holford, who frequently lauds his own scholarship (he used to have a band of respectable scholars who also endorsed his work but this is ever dwindling). However, unlike FFTB, we must highlight that FIBMTD does not indicate or imply that these findings can be generalised to the wider population of children who do not have developmental complications or issues. (For the full detail of our objections: FFTB Part 1.)

To date, 12 double-blind studies on vitamins and IQ have been carried out, and 10 out of 12 show a clear improvement…[Ref 1] [Emphasis added.] [pg. 322]

We have expressed our disagreement about the numbers and dispute that 10/12 showed a “clear improvement” but we celebrate the fact that FIBMTD did not make the FFTB mistake of reporting this as “significant improvement”. In the interest of fairness, we should also state that FIBMTD, like FFTB, also fails to comment on Benton’s caveats about generalising the results to the wider population of children. (For the full detail of our objections: FFTB Part 1.)

FFTB states that “a significant proportion of children with ADHD may have unidentified food and chemical sensitivities”. One of the supporting references for that statement is also cited in FIBMTD.

[Carter et al] placed 78 hyperactive children on a “few foods diet”, eliminating both chemical additives and common food allergens. During this open trial, the behaviour of 59 of the children (76 per cent) improved. The researchers then secretly reintroduced the foods and additives that had provoked reactions for 19 of the children. The children’s behaviour rapidly became worse and so did their performance in psychological testing. [Ref 4][pg. 325]

Well, like the curate’s egg, this summary is good in parts. However, like the FFTB, FIBMTD rather overlooks the fact that many of these children had neurological disabilities or associated allergic problems in addition to hyperkinesis (there were substantial co-morbidities). Plus, the children had already been referred to a specialist dietary clinic and their parents were interested in pursuing a dietary approach for the management of their children’s symptoms. Moreover, Carter et al state:

It is still not clear how generally applicable such a treatment might be within a general group of hyperactive children. [pg. 568] [Emphasis added.]

A rather important nuance that FIBMTD misses, along with FFTB, to be fair. Like with the Benton summary. Neither the Holford & Burne permutation nor the Holford & Fobbester one draw your attention to this: we mention this because Holford has a track record of accusing scientists and researchers of suppressing relevant nuances or information and cover-ups. And we emphasise it because the back-jacket of FIBMTD assures us that this is a “brilliantly researched, authoritative book, based on solid scientific trials”. And because Holford claims that a cadre of professors reviewed and approved each section of FIBMTD.

Holford Watch could express itself at some length about the dubious claims, references and assertions in Chapter 16 of FIBMTD: Solving Attention and Learning Problems Ritalian vs making kids smarter. However, to do so would be off-topic for this post.

For now, we invite you to reflect upon the remarkable fact that it looks as if Holford & Fobbester drew upon FIBMTD for their literature overview for FFTB Child Survey. In doing so, not only did they fail to pick up on the errors in that work (which means that they can’t have checked the primary sources), they failed to understand some of the nuance, and they produced a version that introduced more error. And Holford is not at all reluctant to accuse others of failing to check primary sources or misinterpreting information. You can read a fine example of Holford’s adroitness with appropriate nuances and sources in his discussion of statins in FIBMTD.

And, just to be clear, Holford and Burne offer you an entire section in which they coach you to “encourage your doctor to practise food medicine”. They anticipate the objections that you might meet. E.g., pg. 378:

Objection 2: There is no evidence these treatments work.
Response:…it is just not true. Any properly trained nutritional therapist will be just as keen on basing treatment on the evidence as a doctor. In fact, you might argue that the diet and exercise approach is more firmly based in the evidence.

…the evidence upon which the doctors have based their prescribing often turns out to be faulty. Of course, they are not going to take kindly to this kind of comment from you, but you could show them this book, including the list of references, or even offer to lend it to them…

Actually, judging by the content of The Rise of the Lifestyle Nutritionists: Part 1; it seems that properly credentialled nutrition scientists and doctors won’t take those kinds of comments or references from their primary source - Holford. And, in the light of a remarkably extensive catalogue of errors, and frankly odd enthusiasms (e.g., Q-Link, dowsing for nutritional deficiencies), who can blame them?

References

[1] Benton D, Micro-nutrient supplementation and the intelligence of children. Neurosci Biobehav Rev. 2001 Jun;25(4):297-309.
[2] Richardson AJ, Puri, B. A randomized double-blind, placebo-controlled study of the effects of supplementation with highly unsaturated fatty acids on ADHD-related symptoms in children with specific learning difficulties. Prog Neuropsychopharmacol Biol Psychiatry. 2002 Feb;26(2):233-9.
[3] Richardson AJ, Montgomery P. The Oxford-Durham study: a randomized, controlled trial of dietary supplementation with fatty acids in children with developmental coordination disorder. Pediatrics. 2005 May;115(5):1360-6.
[4] Carter CM, Urbanowicz M, Hemsley R, Mantilla L, Strobel S, Graham PJ, Taylor E. Effects of a few food diet in attention deficit disorder. Arch Dis Child. 1993 Nov;69(5):564-8.

Notes

We used the paperback 2007 edition of FIBMTD in this post.