Black and White Thinking

A recent paper in the official journal of the American Academy of Pediatrics is attracting attention because it claims to refute a widely established treatment of an optical complaint, or visual stress in a large number of children and some adults, especially sufferers of dyslexia.
Stuart J. Ritchie, MSc, Sergio Della Sala, MD, PhD, & Robert D. McIntosh, PhD (2011) Irlen Colored Overlays Do not Alleviate Reading Difficulties. Pediatrics 128(4). doi:10.1542/peds.2011-0314
The condition causes the individual to experience unusual mobility in printed matter, i.e., the words fade, wobble, vibrate, move around, spin, jump off the page. This has been termed Scotopic Sensitivity Syndrome (SSS), because the effect on the words is caused by the glare from the white spaces between them (you’ll have to imagine the text in the following pictures jiggling and dancing about, or get a better idea of what it’s really like using an animated interactive app.).
While coloured light (photopic) reception is handled by the cone cells in our eyes, scotopic reception for white light is associated with our rod cells. SSS has also come to be known as Meares–Irlen Syndrome, or sometimes just Irlen Syndrome, after the original researchers who discovered that placing a coloured acetate over the reading material markedly improved the subject’s reading comfort and ability. It’s staggering that, until diagnosed, children who have lived with this all their lives think this is how the rest of the world sees writing. I mean, who told them otherwise?
Helen Irlen clearly has a vested interest in the adoption of her treatment, and indeed, when aware the condition exists, schools are keen to introduce the coloured overlays for children with reading/learning difficulties. This suggests that while Irlen products are not the only ones available or even suitable for the purpose, there is an obvious additional financial incentive for their uptake.
Furthermore, the treatment is controversial, largely because the exact clinical causes of the condition are controversial. Much of this doubt prevails because there have been very few good experiments that control for complicating factors and placebo effects. In response, the correct skeptical and scientific approach is to design a good experiment that can investigate the most relevant issues to identify what it is that’s in the treatment that works. In doing so, much needed detail can be revealed about the condition. Plus, it can be established whether Helen Irlen and her Institute are profiting via deception, which is the obvious suggestion made by a paper refuting the efficacy of coloured overlays.
However, whether down to a placebo effect or actual clinical intervention, the fact that there is at least one person who has benefitted clearly indicates that there is a positive difference to be made. And be assured, there are countless testimonials from those marvelling at the immediate comfort while reading, brought about by using overlays.
This is where there are good grounds for disagreeing with the motivations for Ritchie et al.‘s research. Even if the Irlen treatment is unproven in a controlled experimental conditions, rather than attacking an intervention that is clearly doing good, perhaps they would have been better targeting their efforts and research funding (arguably some of which is donated by the tax-payer via ESRC backing of Ritchie’s studies) on something more constructive. Simply put, a single trustworthy testimonial from someone who claims they have benefitted from using coloured overlays makes a mockery of any statistics that claims there is no benefit in using them. That is irrefutable.
The question follows, why didn’t Ritchie et al. manage to discern any effect? As above, their claim that there isn’t one is easily disproved – ask anyone who has benefited. Therefore, the reason must be because,
1. their method is flawed,
2. their statistical analysis is wrong,
3. there is another factor.
The care and attention to their method seems admirable; it was designed to generally acceptable standards for this sort of trial. Noticeable though is that it involved assessment of the children by an Irlen specialist, who applied their diagnostic procedure to prescribe the overlay colour for each child. If then the trial was attempting to assess the effectiveness of this prescription, then all it really achieved is a measure of that practitioner’s ability, which rather undermines the objectivity of the experiment.
Their stats are straightforward and appropriately applied, although by their own admission, they lack the sample size to test one of the aspects of Irlen diagnostics that interest them, but not the main point regarding the benefits of colour overlays.
This then leaves “another factor”.
Young research students are generally ambitious and knocking the pedestal out from under an established leader in their discipline is a fun game. I’ve been there myself. Ritchie is clearly a rising star. It is he who was the field operative for this experiment. I do hope that ambition or procedural error have not led to an unintentional mix up over the allocation of overlays. Using prescribed colours matches the treatment to the specifics of an individual’s condition (as a function of light wavelength and optical neurobiology), so, receiving the wrong overlay would be tantamount to no overlay, a clear overlay, or any other inappropriately random (not prescribed) overlay, which is of course what the experiment found.
Anything other than an unintentional error is naturally unthinkable.
I am no psychologist, but regardless of the reasons for the experiment failing to identify any benefit, it does not take an expert to sit next to a child who has experienced reading discomfort, and therefore learning difficulties at school, who immediately lights up with excitement and disbelief because, as soon as an overlay is introduced, they report that the words stop jumping around and wiggling on the page. This is a real and tangible effect. Not necessarily a measurable effect, but a subjective result still, and only inasmuch that the degree of improvement is difficult to quantify.
Suddenly that child is able to sit still and concentrate on text for protracted periods. Suddenly that child is able to do their school work and home work. Suddenly that child doesn’t have reason to see themselves as a failure because the educational system is too unresponsive to detect this condition.  Suddenly that child realises what they are having to deal with on the page is not the same for the majority of others. Suddenly that child can feel normal and alike their friends. That child is my eldest daughter.
Far, far better had Ritchie et al. applied their expertise to discovering an improved system for treating this condition rather than spreading uncertainty about a system that is proven to have positive effects even if not statistically measurable, but one that is changing lives for the better. Their conclusion is downright irresponsible and unhelpful as it is certain to dissuade primary educators from at least attempting a course of action that just might alleviate reading difficulties for many young children, for example, a screening programme for every primary school child,
Our results suggest that Irlen colored overlays have no immediate effect on reading in children with reading difficulties, even among those in whom [Irlen Syndrome] has been diagnosed. On the basis of these results, and the lack of convincing evidence from previous studies, we recommend that parents, schools, health care professionals, and government bodies carefully consider the totality of the evidence before expending time, resources, and hope on this controversial treatment.
A simple informal discussion with parents of children benefitting from the use of coloured overlays would have suggested a more valid approach. If they had really want to make a contribution to the understanding of this condition, an obvious and easy method would have been to print their test text on to coloured paper of a range of hues, in order to investigate when, how and why the distracting glare of black-on-white occurs.
————–
Ritchie et al. have just had their follow-up paper accepted for publication:
Stuart J. Ritchie, Sergio Della Sala, & Robert D. McIntosh (in press) Irlen Colored Filters in the Classroom: A One-year Follow-up. Mind, Brain, & Education.
————–
UPDATE My 10-year-old daughter was tested for scotopic sensitivity yesterday by a professional optometrist with decades of experience. Her reading rate was immediately improved 36% using a purple coloured overlay, the colour found most appropriate for her specific requirements. The benefit my daughter has found, along with the many other children who have benefited likewise when accurately assessed, does suggest Ritchie’s study was lacking as they were unable to detect similar in 61 children of the same age (7-12 years) and with the same condition. This makes their concluding remarks even more inaccurate and irresponsible.
About these ads

About JFDerry

Writer. Darwin, science & more. 4 books: Piospheres, Darwin in Scotland, Serial Killers. Current project is THE DISSENT OF MAN. Born near London, raised near Primrose Hill and in Lincolnshire, and studied at the Universities of Bangor, York and Edinburgh for degrees in Biochemistry, Bioelectronics and Biological Computation, and a PhD in African Ecology. Mainly working in British and African universities, but also in Spain, Brussels, Mongolia and Australia, to date, publication history is mostly in academic journals, on aspects of computational biology, pastoralism and on Charles Darwin and evolution. However, also written for several national newspapers, various governments, several major record labels and independent book publishers. Fiction has appeared at the Edinburgh International Book Festival and poetry is at the Human Genre Project. Lives in Edinburgh, with partner and their two daughters.
This entry was posted in Uncategorized and tagged , . Bookmark the permalink.

7 Responses to Black and White Thinking

  1. Anonymous says:

    So what level of evidence would be required for you to be persuaded that a given intervention was not effective, at least to the average recipient?

  2. JFDerry says:

    Dear Anonymous (please identify yourself if you really want an equal discussion),

    It’s obvious that the study in question failed to detect the benefits reported by so many recipients; something was clearly amiss with the experiment for it not to achieve a measure of the immediate benefits reported. Part of the problem in quantifying this effect is that there are no average recipients because individual responses are specific (as a function of light wavelength and optical neurobiology, as stated above), and due care may not have been observed. Ultimately the study fails on design, because it is only testing the ability of the Irlen practitioner: sample size=1, degrees of freedom=0.

  3. “A single trustworthy testimonial from someone who claims they have benefitted from using coloured overlays makes a mockery of any statistics that claims there is no benefit in using them. That is irrefutable.”

    As the old cliché goes, the plural of anecdote is not ‘data’. Would you also make the same claim about the efficacy of acupuncture, chiropractic or homeopathy?

    A cursory glance at Wikipedia reveals that it’s not just Ritchie et al. who have found no evidence for coloured overlays working:

    http://en.wikipedia.org/wiki/Scotopic_sensitivity_syndrome#Lack_of_Scientific_Proof

    But let’s not allow evidence to get in the way of assertions.

  4. JFDerry says:

    I do not claim the syndrome is a discrete condition. It may well be linked with other treatable disfunction, but that does not justify removing a way to ease the symptoms.

    Okay, so you’ve got a broken leg. Here’s a cast but you’re not allowed a walking stick as well.

    The only proof you need that overlays work is to ask a child, “does that help?”

    (PS: There are more reliable sources of academic information than Wikipedia)

  5. EdPsychPhD says:

    As the author of this blog suggests, there are “other factors” that were not taken into account in Ritchie’s methodology that could be factors in his lack of positive findings. Aside from a small sample size, Ritchie admitedly had over half of his subjects who suffered from “uncorrected optemetric problems.” Irlen always requires that optemetric problems first be corrected for before testing for the correct color, as color will not help problems associated with the eye (only with the brain). All those subjects should have been thrown out of the study. Second, and even more importantly, the average age of child participating in the study was around 9 years old. These were early elementary students, identified as struggling readers, who as a result had very likely failed to acquire the foundation of reading and phonics skills required for fluid reading and comprehension. Ritchie did not allow for any intervention period where children would be allowed to actually acquire reading skills now that the barrier preventing them from acquiring those skills (i.e., Irlen Syndrome) had been removed. Irlen overlays are not a replacement for reading skills and reading practice, they merely remove the distortions and discomfort that make acquiring those reading skills more difficult. The expectation that he would see immediate effects on reading fluency and comprehension from very young children who lacked basic reading skills, is where he failed most in his methodology. Other placebo controlled studies with young children have chosen alternative methodology, such as delaying treatment to a control group, while offering treatment to the test group for a specified period of time, then assessing change in reading fluency and comprehension for both groups. Ritchie’s study is not the first placebo controlled double blind experiment out there on Irlen. Given his small sample size, and flawed methodology, it does seem somewhat irresponsible as an academic researcher to make claims like he did at the end of his paper telling people to beware of the method and not to waste their money. That statement alone show’s his immaturity as a researcher. More seasoned researchers would suggest the need for further research to confirm their findings, given the long history of positive research out there that actually exists.

  6. JFDerry says:

    EdPsychPhD,

    thank you for your excellently reasoned comment. The trial was undoubtedly naïve and lacked a comprehensive understanding of the associated factors. The metric was inappropriate, expecting some sudden change in ability when learning up to that point in time had been hampered by scotopic sensitivity. The concluding message is dangerously irresponsible and mean. Even if not a direct treatment, the use of overlays moderates the visual stress. Sadly, the follow up paper carries the same damning message but fails equally as it builds on the original. The work has all the signs of an inexperienced, ambitious researcher and reflects badly on the more senior academics involved, who should know better.

  7. Thony C. says:

    Anecdotal evidence is not data but at least it lays the basis for further thought. I come from a family with a large collection of so-called learning difficulties. At least four of us, myself included, are very obviously severely ADD and I have two nephews, different parents, and one niece, again a different set of parents, who are all severely dyslexic. All of them officially diagnosed and all received special tuition and other assistance in school. I myself am severely dysgraphic. I first came across color overlays in the case of my niece, who is the youngest of the people mentioned here, and I know for a fact that her reading skills improved measurably after she started using them. Now this of course proves nothing but it does suggest that the study critiqued here by J F might not be the last word on the subject.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s