Looking over the Durham fish oil (non)trial again, I noticed something I had missed: they have previously claimed to be looking at three quite different things in order to determine whether the pills are effective. They started out (back in 2006) with a press release where they promised to measure efficacy by comparing students’ actual GCSE results to their predicted results. Not terribly promising methodologically, but at least Durham were explicit about what was going to be measured. Except that, in their response to a 2008 FOIA request, Durham stated that
All that was being sought was the number of children taking up the offer of capsules and a comparison of GCSE results from the previous year
To me, that sounds rather different. Then, to make matters worse (as we have already noted) Durham eventually chose to measure how fish oil takers compared with ‘matched pairs’ from the same year group. Again, this is a different test of whether the pills work. Those of you wondering why Durham are taking so long to release these results might also be interested to know that the 2006 press release promised that
the first test of the supplement’s effectiveness will be when they sit their ‘mock’ exams this December.
It is good practice to decide what you are measuring – and how you are going to judge the efficacy of a treatment – before starting a trial: otherwise, it can be tempting to change what is measured in order to generate the ‘right’ result. While I cannot know why Durham have discussed plans to measure the ‘success’ of fish oil in this (non)trial in three ways since 2006, this gives yet another reason why it is imperative that Durham release their data ASAP: so that the claims in their press releases can be scrutinised.
Rather than Durham playing third time lucky with tests of efficacy, could this be a case for Einstein creature?
Keep looking, Dave Ford, you’ll find that false positive somewhere!
This is precisely why trials need to be registered and researchers need to state up front what they will be examining, how they will collect the data, and they methods they will use to analyse it.
Of course, all will become clear at this Saturday’s FFTB when Dr Portwood presents the results of the Durham studies.
Are you going along to listen, dvn? I think you should…all in the name of “research”, of course.
If I had £300+ going spare and couldn’t think of a better use for it, then, who knows. We were bitterly disappointed not to be offered comp tickets to the event, from the Holford seminar on Friday pm through to the ticker-tape, triumphalist finish on Sunday. Maybe FFTB thought we might mention their Child Survey or apply the same scrutiny to some of their other work, or that carried out in Durham.
We’re cut to the quick, but somehow, we shall survive.
“Of course, all will become clear at this Saturday’s FFTB when Dr Portwood presents the results of the Durham studies.”
I think I just felt my heart sink.
Changing the measure of success so many times makes it clear just how seriously we should take any results announced by Dave Ford or Madeleine Portwood. Once would have been enough for me to write their conclusions off as unreliable. Measuring things two or three different ways until you find the result you like best is not science, it’s nonsense.
Apparently, the Food for the Brain conference is now fully booked. So – in the absence of a media pass for us – we won’t be able to attend…
Of course, we’d be very interested to know what is discussed. If anyone’s planning on going – or finds this page googling the topics discussed after the conference – please do let us know.
Still gnawing away at alternative medicine you Brave New World alpha hopefuls :0)
Please tell me that you do know there IS a direct relationship between EFA’s and brain function you are not so tunnel visioned as not to acknowledge this I hope
Hey why don’t you guys give it a try – might ease the anger etc you never know, do your own trial, you don’t even have to tell each other find out for yourself how effective it is, certainly clears the cobwebs for me.
Admin edit: “certainly clears the cobwebs for me” – judging by the quality of your insights/comments, that is a very personal perception and it’s good that it comforts you to hold it.
Lateral – surely the interesting research which has been done about the role of EFAs in the body makes the pure quality of the Durham (non)trial even worse. Durham could have contributed to our knowledge of EFAs if they had done the research properly, but have instead merely muddied the waters furthers.
By the way, dubious trials (sorry, initiatives) on children do make me angry. I think that’s an entirely appropriate response; it certainly hasn’t been ameliorated by my (relatively high) EFA consumption.
Pingback: Durham (non)trial fails to show any benefit to fish oil (Equazen EyeQ) supplementation « Holford Watch: Patrick Holford, nutritionism and bad science