ADD / ADHD, Dore Achievement Centres, Dyslexia, Science

ADHD, Biofeedback, Good Study Design and the Placebo Problem

Mind Hacks points to a good article on the current state of research into the effectiveness of biofeedback on ADHD. Its conclusion that biofeedback is a promising but unproven treatment is fair one.

What is interesting about the article is that it talks about what makes a good research study and delves into the problems of having a control group. In medication trials the control group takes a placebo pill, an inert sugar pill, whilst the study group takes the real medication. Both pills look the same and none of the medical staff handing out the pills know who is getting what. This minimises the confounding variables in the experiment.

In biofeedback studies, the problem is what to do you do with the control group. The treatment typically consists of 30 minutes a day using a computer and a biofeedback device. The control group needs to have a dummy treatment that is identical so that no one knows who is getting the real treatment but yet is guaranteed to have no effect. By its nature, biofeedback is an interactive process so the control group must have some sort of interactive experience. A device that just randomly responses to the the biofeedback would be quickly spotted.

This problem, when applied to a treatment such as the Dore program, becomes even more significant. Dore is series of exercises forming a 12 month program of twenty minutes a day. What activity could be used as a placebo that isn’t immediately recognised as the placebo? The only possibility would be to give one group the proper Dore exercises whilst the control group gets a random selection of exercises. But, the random exercises are too much like the real thing and there is a chance they help develop the cerebellum. Though they certainly would not be as effective it would introduce an unknown variable into an experimental set-up designed to remove unknowns.

A secondary problem is that children are assessed every six weeks under Dore using sensitive balance and eye-tracking devices. The child gets regular empirical evidence that the treatment is having an effect long before any improvements are seen in academic work. Obviously the users of the placebo should not see any improvement and this could have a significant effect on the child’s general confidence.

There is a way to conduct trials in these situation. You have two treatment groups, A and B plus a control, Group C. Group A gets the treatment whilst Group B gets a different activity, for example one-to-one help for 20 minutes a day. Group C gets no treatment. After Group A has received the treatment, the groups switch so Group B get the treatment and Group A gets the other activity. Once Group B has completed the treatment the experiment ends.

Both Groups A and B should of made more academic progress than the control Group C because of the one-to-one help they received and confounding variables such as the Placebo and Hawthorne effects. If the treatment worked better than the one-to-one help, then Group A would be expected to show academic improvements during the first period greater than Group B. This progress would slow down in the second period after the two groups switched roles and Group B would catch-up. If the treatment did not work, Group B would be ahead after the first period thanks to the one-to-one help and Group A would catch-up in during the second period.

There are some obviously difficulties in using this experimental design with Dore. Firstly it is a year long treatment. That means the whole experiment will last two years. To allow for people to move schools or drop out without having a major impact on the statistics, each group needs to be quiet large. Given that for the two treatment groups, there is a lot of investment of time in doing the exercises or taking children to the one-to-one sessions, the drop-out rate is likely to very high. Each group would probably need to start with about 50 people.

The cost of all this is significant. One-to-one teaching everyday for 50 people for year, twice, won’t be cheap. Nor will provision of the Dore treatment. Overheads in managing the experiment, tracking the academic performance of the children all add up. A gold standard experiment like this costs tens of thousands of pounds. Of course if Wynford Dore pays for the experiment then it won’t be an independent study but no one else will fund the experiment. The only dyslexia research body with that sort of funding is the department at York University. It is run by Professor Snowling who is rabidly opposed to Dore so its unlikely that any funding will come from there.

Another problem with an experiment on Dore is an ethical one. Asking a child to take part in a drug trial for four weeks is OK because if the treatment has no benefit then the child isn’t effected in anyway. For a Dore trial, the child has to spend twenty minutes a day for a year. If the treatment doesn’t work then the child has lost a huge amount of time and effort that could of been spent on more established therapies and the child would of fallen even further behind academically. This is major problem as the basis of any ethical experiment is that in no way, regardless of the experiment’s outcome, should the subjects experience any detrimental effects.

There are good reasons to criticise Dore’s scientific research and similar research by other alternative treatment but it has to be seen in context. The practicalities of an effective study that proves in one go the treatment works are both difficult and expensive. Dore and others are stuck in a catch-22 situation where if they pay for research there will be immediate accusation of bias but if they don’t pay for it, no one else will. Finally the ethical issues make the whole feasibility of an experiment doubtful.

Source: How Strong is the Research Support for Neurofeedback in Attention Deficits?


  1. FrazzleDazzle

    Very well said, Chris! Thank you very kindly for taking the time to explain the difficulties and issues with research regarding therapies such as these.

    If one would look at similar treatment therapies performed by OT’s and vestibular rehab specialists, physical therapists, other types of rehab specialists, (all whom are quietly doing their work without anyone screeming down their necks that it is not scientifally-proven), you won’t find much research and/or documentation out there concerning large groups either. As one could imagine, that would be impossible. Rather, you will find more individual case studies and documentaries of the more interesting or unusual cases. Dore should be regarded as no different because of the nature of the treatment, as you so well noted.

    Your work here is very well-appreciated! 🙂

  2. myomancy

    Thanks for kind words.

    It is remarkable how little is scientifically proven.

    Ritalin, because it was created in the 1950s, had no evidence of its effectiveness on ADHD before doctors started prescribing it in the 80s. I’m not even sure there was any evidence that Ritalin was safe to use with children at that point.

    Education is even worse. Have you ever seen a scientific study on the effectiveness of schools? Our school days and lessons are no different than 1908 or even 1808. Its seems very doubtful that when schools as we know them first came into existence that they just happened to hit on the perfect way to educate kids.

  3. DORE don’t even give stats on the sort of things you’d expect to be basic housekeeping, such as % who do not complete the programme. That wouldn’t need a trial, just a look through their accounts.

    Similar problems come up in medical research all the time, and there are well established ethical & procedural ways to deal with it. As I understand it you’d want to show that the control group was not worse off than appropriate ‘standard care’. So you could have one group getting the usual school support. To control for the effects of attention, expectation, etc., you’d need some treatment taking about the same amount of time & effort.

    I was reading an interesting study today on dyslexia, where they had one group using whole-word reading, one group using a phonics approach, and one with no specific reading training but instead looking at ‘classroom survival skills’ such as organisation & planning.

    Until you’ve demonstrated a disproportionate advantage of one treatment it is ethical to test them – the alternative is that every child is a guinea-pig to no benefit.
    Medical trials will often have a separate ethics committee who will be the only people with access to un-blinded data, so if part way through the trial one treatment is already clearly much better, the trial can be stopped & all groups given the most effective treatment.

    Finances – the Welsh Assembly are having a look at SpLDs just now, & I’ve written to them suggesting that the costs of doing a big head-to-head trial over a few schools would be outweighed by the benefits of Welsh children having the most appropriate intervention after the trial is done.

  4. myomancy

    Tom – Treated / untreated with what? on who?

    Brainduck – It is odd that Dore does not publish even basic stats leaving me to assume the drop out rate is high, at least 50%.

    There would be other ways for Dore to gather useful data. Say the SAT scores for the 3 years before treatment plus the 1 year after treatment. There should be a notable improvement on an individual level and this data set could use the national average results as a baseline / control. Not an ideal study as the nay-sayers would claim any improvement is nothing to do with Dore but any sensible person would see it as good evidence.

  5. FrazzleDazzle

    We have the pre- and post- AIMS scores (state tests comperable to SATS in the UK). Within months of starting the program, my son had quite a leap from all of his previous years in reading/comprehension/fluency, etc. I have his score sheets posted on his blog. I do not see that as maturing, I do not see that as a placebo, it is pure objective stuff and I am so very proud of him and his hard work this last year on the program.

Comments are closed.