Holford Watch: Patrick Holford, nutritionism and bad science

Here at Holford Watch, we like the idea of good research into nutrition. So, we were intrigued when we received an e-mail from Holford’s e-mail mailing list - telling us about a new article in the Nutrition Journal by Block et al: Usage patterns, health, and nutritional status of long-term multiple dietary supplement users: a cross-sectional study link [pdf]. We read it with interest. And we were really disappointed in the quality of this research. Holford argues that:

this study strongly suggests that popping a handful of supplements a day is likely to add years to your life and life to your years.

However, the quality of the paper in question does not allow such confidence in any conclusions based on it. There are a number of problems here (ironically, just the type of techniques that drug company researchers are sometimes accused of using for nefarious means). In particular:

• There is no attempt to control for the diet of those analysed. While they compare multiple supplement users to non-supplement users, these populations may not be comparable. For example, do non-users eat more junk food than users: if multiple users were committed enough to ‘wellness’ to swallow an average of 17 supplements each day for 20 years or longer (p. 5 and 11) they may be less likely than usual to be gorging on doughnuts.
• The researchers used a sample of multiple users “who completed [their] questionnaires and were free of cancer” (p. 5). This firstly works to exclude the c. 60% of multiple users who did not complete their questionnaires - some may not have replied due to ill health or death - and the researchers chose not to investigate reasons for non response. The exclusion of those with cancer will also have influenced results (this type of thing is not uncommon, but there should have been discussion of the problems caused).
• Much of the data offered is surrogate measures rather than actual health outcomes. So, for example, the researchers show that taking vitamin pills raises blood serum levels of the vitamin (will the next issue of the journal contain a paper proving that the pope’s catholic?) However, such things are only really worthy of note if they actually influence health outcomes - and the evidence suggests that in many cases they do not have a positive effect.
• Along similar lines, Holford is excited that the study found that “four times as many people not taking supplements (45%) had elevated homocysteine levels (above 9 µmol/l - averaging 9.6) compared to the multiple-supplement takers (11% - averaging 6.1).” However, while we know that taking certain supplements will lower Hcy levels, there is not good evidence that lowering Hcy levels in this way will improve your health.
• People who take that many supplements for such a long time are unusual: generally, it is hard to get people to comply with a treatment regime to nearly such an extent. The NYT carries an illuminating discussion of epidemiology and the confounding effect of compliance. It appears that “people who comply with their doctors’ orders when given a prescription are different and healthier than people who don’t.” Is this also the case with people who take a lot of supplement pills? I would like to know whether these people are, for example, also more likely to comply with prescriptions of medicine from a qualified Dr, more likely to comply with recommendations for regular exercise, etc.
• Because the multiple users had been taking so many supplements for so long, this suggests that they were - at worst - financially stable for 20+ years: if you’re broke, money tends to go on pills before multiple vitamin pills. How has this benefited their health?
• The multiple users had been using pills from Shaklee Corporation for 20+ years. This makes me worry as to how generalisable the conclusions of the paper are: for example, does Shaklee advertise in health food shops or vegan publications? Are their products often ‘prescribed’ by nutritionists? I could go on… The article doesn’t even say if multiple users were, for example, disproportionally vegan - which makes it very hard to interpret the results.

As I’ve said before, I’m not a nutritionist (I’m a very naughty boy, apologies to Monty Python). These problems jumped out at me after a quick reading of the article, and mean that I don’t think it demonstrates anything much - except, perhaps, from providing additional evidence for the fairly banal finding that supplementing with vitamin and mineral pills will generally increase serum levels of what you’re supplementing. I’m surprised that Holford - given his glittering professorship at Teesside University, and his criticisms of the ways in which drug companies can distort scientific data - seems to have failed to spot these issues. While the researchers should be commended for openly declaring their links to the Shaklee Corporation and their Shaklee Corporation funding under Competing Interests (p.20), I am surprised that Holford did not see these competing interests as worthy of mention in his mailing list e-mail. I’m even more surprised that the reviewers and editors for the Nutrition Journal failed to insist that the article’s authors address these problems in their discussion of their results.

We’re going to look at some other aspects of the paper another day - but, for now, it’s worth noting how many aspects of this paper are problematic, and asking how it got published in its current form.

Update 05.12.07: Nature offers The Ghost of Research Past which is an instructional overview of popular nutritional beliefs that persist although clinical trials have long since contradicted observational studies that suggested a clinical benefit (explored more fully in Tatsioni, Bonitsis & Ioannidis).
Chicago Flame has the most recent example of this sort of thinking. “For college students who generally have a diet that may be limited by money, and may lack sufficient variety to provide all the micronutrients in the amount needed, a good generic multi-vitamin/mineral supplement (One-A-Day type) may be a good idea. They are safe, they are inexpensive, and they provide ‘insurance’ on a daily basis for a somewhat unbalanced diet,” said Dr. Robert Reynolds, Ph.D. and associate professor in the Department of Human Nutrition.

Update 17.12.07: Does Industry Sponsorship Undermine the Integrity of Nutrition Research/ and Relationship Between Funding Source and Conclusion Among Nutrition-Related Science Articles