371778_low CROP 1

New Research Casts Doubt on CogMed for ADHD

Bookmark and Share

The Working Memory Training bandwagon could soon be screeching to a halt, at least when it comes to much-promoted viability for ADHD, given results from two recent studies that undercut previously made claims.

This comes as no surprise to me. Years ago, I participated in a list serve of ADHD professionals. When I dared to question the validity of new, unproven programs such as CogMed (which is discussed below), I was roundly castigated by psychologists who were using it in their practice.

Never mind the bias created by the profit motive; there was certainly insufficient evidence back then (circa 2008) to merit offering it to children or adults with ADHD.  From what I could tell among the list members who also had ADHD, they reported no improved function in life, but they enjoyed the training. All in all, it sounded like an expensive distraction to me.

More importantly, there is always an opportunity cost to pursuing these unproven treatments instead of evidence-based interventions such as medication and behavioral-therapy as well as structural supports (calendars, timers, visual organizing systems, etc.). Will an adult lose a job or a marriage? Will a child fail a grade or suffer other continuing losses to self-esteem?

As it turns out, my skepticism was well-founded. While initial studies were favorable, more recent research counters those studies’ conclusions.

Thanks to David Rabiner,  Associate Research Professor at Duke University’s Department of Psychology  Neuroscience, I can share with you (below) a sophisticated analysis of the research.  Dr. Rabiner has long performed the excellent service of parsing the research around ADHD in his newsletter, Attention Research Update. You can subscribe to his free newsletter here and read through the substantial archives once you are subscribed

I appreciate his clear writing style, but research terminology can be complex for the average reader; this analysis might be “too much information” for some. So, here is the bottom line,  as Dr. Rabiner writes below:

“What can we conclude from this work? Despite promising initial reports suggesting that  (CWMT) is a potentially effective treatment for ADHD, these studies significantly undercut this conclusion. This does not mean that there is no utility to CWMT, however, particularly for individuals with demonstrated working memory deficits. If one’s treatment goal is to enhance working memory, CWMT may have real value. If the goal is to bring ADHD symptoms under control, however, these findings indicate that for most children with ADHD, CWMT would not currently be considered a reasonable substitute for medication and/or behavior therapy.”

Here is the full report from Attention Research Update:

There are several reasons why it is important to develop evidence-based ADHD treatments in addition to medication and behavior therapy.

  • Not all children benefit from medication, some experience intolerable side effects, and many continue to struggle despite the benefits provided by medication.
  • Behavior therapy can be difficult for parents to consistently implement, and does not generally reduce behavior difficulties to normative levels.
  • Furthermore, although both treatments can help manage ADHD symptoms, they generally do not induce changes that persist after treatment ends.
  • Finally, despite numerous studies documenting the short- and intermediate term benefits of medication and behavior therapy, their impact on children’s long-term success remains to be clearly documented.

In response to these limitations, researchers have shown growing interest in whether cognitive training—generally done via computer—can induce more lasting changes in children’s ability to focus and attend. One approach that has shown promise in helping youth with ADHD, and which is now widely available, is Working Memory Training.
Working Memory Training is based on findings that Working memory (WM)—the ability to hold and manipulate information in mind for subsequent use—is frequently compromised in youth with ADHD and may contribute significantly to symptoms of inattention. WM deficits also to contribute to the academic struggles that many children with ADHD experience. Developing an intervention to enhance WM in children with ADHD could thus be extremely helpful.
Several published studies suggest that WM training is a promising intervention for children with ADHD. In one (see helpforadd.com/2005/march) for a detailed review, children with ADHD were randomly assigned to high intensity (HI) or low intensity (LI) WM training. The HI treatment involved performing computerized WM tasks, e.g., remembering the sequence in which lights appeared in different portions of a grid, recalling a sequence of numbers in reverse order, where the difficulty level was regularly adjusted to match the child’s performance by increasing or decreasing the items to be recalled. This is called ‘adaptive’ training because the difficulty level adapts to match the child’s performance and children are consistently challenged to expand their working memory capacity.

In the LI condition, the tasks were similar but the difficulty remained low throughout, i.e., the number of items did not increase when children responded correctly. For these children, their working memory capacity was not consistently challenged and was not expected to grow as a result. This was considered the control condition.

Each group trained 30-40 minutes per day, 5 days per week, for 5 weeks with training supervised by parents. Parents were supported through weekly phone calls with a trained coach whose role was to help make sure training was implemented as intended.

Results indicated that immediately after treatment, as well as 3 months later, children in the HI group showed improved WM performance compared to LI children. Furthermore, parent reports indicated significant reductions in ADHD symptoms, particularly inattentive symptoms; these reductions remained evident at 3 months. However, no benefits in ADHD symptoms were evident in reports provided by children’s teachers. Given the importance of improving attention in the classroom, this was a significant limitation.

A subsequent study (see helpforadd.com/2012/july) also used random assignment to HI vs. LI training, and observed the impact on children’s behavior in a controlled classroom setting. Results indicated significant reductions in off-task classroom behavior among children with ADHD who received HI training. This partially addresses concerns about failure to find teacher reported benefits in other studies. Children also showed gains in non-trained measures of WM.

Results from these studies, along with several others, suggest that Working Memory Training (the specific training system used in these studies was Cogmed Working Memory Training, i.e., CWMT) yields benefits in non-trained measures of WM and reductions in parent-report inattentive behavior. However, no study has found benefits in teacher reported behavior and symptoms.

Concerns about the evidence base for CWMT

A significant limitation in the evidence-based for using CWMT to treat youth with ADHD is the absence of teacher-reported benefits. In addition, some researchers question whether the LI training is an adequate control condition. This is because although children in HI and LI training complete the same number of trials each session, the LI training takes less time each session because it does not become more difficult. Thus, the conditions differ in ways other than whether difficulty level adjusts to match the child’s performance.

Some have also suggested that parents of LI children may become aware that their child has been assigned to the control group. If parent are not truly “blind” to condition, it could explain parent-reported benefits that have been found. For these reasons, some have suggested that CWMT should be regarded as no more than a “possibly efficacious” treatment for ADHD and not considered a ‘first-line’ treatment like medication and behavior therapy.

Results from 2 recent trials

Two recently published studies provide important new data on the efficacy of CWMT for ADHD:

Study #1:

Working memory training in young children with ADHD: A randomized controlled trial (van Dongen-Boomsma et al., 2014) was conducted with 51 5-7-year old children with ADHD in the Netherlands. Similar to the studies summarized above, children were randomly assigned to HI vs. LI training. Training consisted of 25 sessions of 15 minutes 5 days a week for 5 weeks; this is the recommended training schedule for younger children. Training was conducted in children’s home and supervised by parents. Training was conducted in children’s home and supervised by parents.

A certified coach contacted parents each week to evaluate the performance and motivation of the child using a standardized questionnaire. Neither child, parents, or coaches knew which condition the child had been assigned to. Because coaches were kept blind to children’s condition, and thus did not receive detailed information on how children were progressing through the exercises, they were unable to provide coaching support to parents as is done in regular clinical practice.

Outcome measures included neurocognitive assessments, parent and teacher reports of ADHD symptoms, and a global assessment of functioning made by study clinicians. Results indicated benefits of HI training on only 1 of 25 outcome measures, a measure of verbal working memory. Importantly, no training related differences were found for parent, teacher, or clinician ratings. The authors conclude that their findings cast “…doubt on the claims that CWMT is an effective treatment in young children with ADHD.”

Study #2:
A randomized clinical trial of Cogmed Working Memory Training in school-age children with ADHD: A replication in a diverse sample using a control condition.  (Chacko et al., 2013). In this second randomized controlled trial, 85 7- to 11-year old-children with ADHD were assigned to HI or LI CWMT.

Training consisted of 5 30-45 minutes per week for 5 weeks; this is the typical session length for children in this age range. Additional trials were added to LI training sessions as needed so that the length of LI and HI training sessions were more comparable. Also unlike the prior study, coaches had complete access to children’s training data so that they could oversee parents as is done in standard clinical practice using CWMT. Outcome measures included parent and teacher ratings of ADHD symptoms, standardized assessments of working memory, computerized assessments of attention, and academic achievement testing.
As reported prior studies, children receiving active training showed significant gains in working memory compared to control children. This was true for both visuo-spatial and verbal working memory.

However, computerized tests of attention showed no significant difference between the groups. The same was true for parent and teacher ratings of ADHD symptoms as well as for measures of academic achievement.

Based on these largely negative results, the authors conclude that CWMT should not be used as a treatment for ADHD.

Summary and Implications
Results from these 2 randomized-controlled trials do not support CWMT as a first-line treatment for ADHD. In both studies, there was evidence that training produced gains in some non-trained measures of working memory. However, improvements in parent or teacher ratings of behavior were absent. Given the adverse impact of core ADHD symptoms on academic and behavioral functioning, this is a significant limitation.

In the first study, one could argue that coaches could not use detailed records of children’s training performance to guide their coaching calls with parents, which may have undermined the training effectiveness. This was not true of the second study where coaching supervision was provided in the standard manner. This second study was also the largest trial of CWMT for ADHD conducted to date and the sample size was sufficient to detect meaningful treatment effects if they were there.

What can we conclude from this work? Despite promising initial reports suggesting that CWMT is a potentially effective treatment for ADHD, these studies significantly undercut this conclusion. This does not mean that there is no utility to CWMT, however, particularly for individuals with demonstrated working memory deficits. If one’s treatment goal is to enhance working memory, CWMT may have real value. If the goal is to bring ADHD symptoms under control, however, these findings indicate that for most children with ADHD, CWMT would not currently be considered a reasonable substitute for medication and/or behavior therapy.

One final comment. I think it is important to note that many clinicians are using CWMT with children who have ADHD and many have reported that they are obtaining good results. A number of these are clinicians that I know and respect, and it is difficult to reconcile the negative results reported here with outcomes that are reported by many clinicians who use Cogmed in their practice. This is an example of where research findings differ from clinical impressions, and I don’t think it is possible to conclude with complete certainty that one is right and the other is wrong. However, if one looks to the research to make decisions about treatments to recommend for children with ADHD, routinely recommending Cogmed would be inconsistent with the current research base in my view.

Bookmark and Share

Tags: , ,

  1. Kathryn’s avatar

    For an in depth discussion of the entire body of Cogmed research, please visit …..

    —–

    Sorry, Kathryn with Pearson, owner of Cogmed. I don’t allow advertising on my blog, especially the kind that comes disguised as a comment.

    Pearson has an advertising and marketing budget, and I’m sure my blog’s readers have had ample exposure to it.

    Reply

  2. Oren Mason MD’s avatar

    Gina, your readers should probably know how attractive systems like CogMed are to professionals who treat ADHD. The idea–as a businessman–that I could buy a system, hire a tech to run it and earn money without using any of my own time is very attractive. It takes an ethical stance to resist the allure of CogMed and similar “adjunctive therapies”.

    CogMed was worth a try; it would have been wonderful had it worked. The hope that one little tug on the Gordian knot of ADHD might untie the whole thing is very seductive and very wrong. So far, the adjunctive therapies that actually work are very time intensive: cognitive behavioral therapy, exercise, IEP’s, and meditation.

    Thanks for making the evidence clear on this. We need to ditch the ineffective therapies and keep moving forward with effective ones.

    Oren Mason MD
    founder, director Attention MD
    author “Reaching For A New Potential”

    Reply

    1. Gina Pera’s avatar

      Thanks, Oren.

      I’m glad that you connected the dots instead of me.

      The mere implication of it in that listserv drew a firestorm of personal attacks on me.

      Which is why I left it in disgust. They were not ADHD professionals.

      Best,
      g

    2. Danielle’s avatar

      Gina,

      Thanks for posting this. ADHD isn’t the only condition I have and I’m glad it’s being studied in girls now. I wonder just how many women my age weren’t diagnosed until adulthood because of the different presentation. I remember getting in trouble a lot when I’d be given a list of tasks to do verbally, like change the cat litter, take out the garbage… And actually those were always the first two on the list and to this day I don’t know a single thing after that. I was not permitted to write it down because “you should be able to remember this.”

      The funny thing about that is I had no problem remembering my homework assignments from school. I would copy the problem numbers or chapters to read off the blackboard and not need to look at what I wrote most of the time. Generally, it wasn’t simple like “1-20.” It was more like “1-15 odd numbers only, 23, 27, 30-36, 49, 55-60.” It got complex. Back then, my memory was great and I was recently told that I remember more detail than most people and adding details is a something people do when they are lying (this little tidbit from someone who knows me just well enough to make that observation explains a lot).

      But if you aren’t paying attention to something, you can’t form the memory. I don’t know how engaging CogMed is, but if it holds their attention and works for improving working memory, it’s going to show improvement in working memory. Writing down my homework assignments made me pay attention and if I could remember the assignments without looking, I probably could have remembered that list of chores if I could write them down, crumple up the paper and burn it. I just wasn’t interested in chores right after dinner when maybe my homework wasn’t even done yet and I wanted to finish it because I couldn’t play until it was done. I’m willing to bet that a kid with ADHD can remember just as much about his favorite video game as a kid without ADHD. Change that to the most boring class at school and that memory vanishes in the kid with ADHD. Why? YOU CAN’T FORM MEMORIES IF YOU AREN’T PAYING ATTENTION! It’s very close to asking for a sequence of numbers the person never received.

      I have been in pain since I was 17. That’s over half my life and I no longer work so my brain isn’t stimulated the way it used to be either. There is at least one study out there (I would have a citation if I could handle a screen bigger than an iPhone) showing that being in pain for more than 6 months causes cognitive just like the aging process. This was done on people in pain for 6 months and I did the math and it put me at my parents age. My husband has been saying he feels like he is getting dumber. We are at that turning point and this study was 6 months of pain. I’m just shy of 20 years of pain! I’m also on multiple meds that affect my memory. I’ve always thought it was just the meds making me feel dumber. I’ve had many discussions with my psychiatrist about my memory and sometimes he tells me straight up I probably can’t remember something because I wasn’t paying attention so there’s not enough information to form a memory.

      A few years ago, a friend told me about Lumosity. When I’m well enough, I do it every day because it’s fun and I like justifying video games somehow. But I also hope there is something to the neuroplasticity thing and it’s helping me. They let you know what percentile you’re in compared to other Lumosity users and my scores are actually pretty high and most of them have gone up with regular use. Part of it has been learning the games but they have made it adaptive now so that’s less of a factor. There is one exception. My score for attention isn’t bad but it’s far lower than any of the others and doesn’t really improve despite the fact that I like the games. If I wasn’t treated for ADHD, I’m sure that score would be horrible.

      I was able to maintain good grades using intellect to compensate for ADHD. I was often more interested in what was out the window than what the teacher was saying. I relied heavily on the textbook, hated doing 50 math problems where there were 3 problems with different numbers several times and I got it after 2 or 3, struggled with some subjects but managed to do okay, often had my head on my desk not looking at the blackboard but somehow when a teacher called on me, managed to pick up my head just long enough to give the right answer and then go back to my nice state of half-sleep. Perhaps my motivation for paying attention at all was that a couple correct answers would get them to stop trying to embarrass me and I was tired. I learned to hide my figeting because my parents didn’t like it. Really the biggest way my ADHD showed at school was junior year math, which I liked. We were seated alphabetically and I ended up in the front row right at the teacher’s desk next to a kid named Brian. I didn’t know him before this class. He was a senior and I was a year ahead in math. We would talk throughout the whole class, voices low enough not to disturb the class. The pace was boringly slow for both of us and the teacher stopped all his attempts at keeping us from doing this when we would not only answer the question but add on the final answer to what was 1/4 of the way through being taught because we had already worked it through and yes we were talking about other things but both doing very well in the class. It’s probably lucky we were in the corner. That’s something I could have been in trouble for but wasn’t. I think I got lucky a lot with stuff like that and my fear of getting in trouble was greater than my lack of impulse control and teachers at my school gave warnings and saw me as a good kid so the only call home was for reading an “inappropriate” book during free reading in 8th grade English and then “lying” about my mother giving it to me to read and the teacher confiscated it. I waited for that call with great anticipation. No one liked that teacher and my mom wanted her book returned to me. It was a romance novel with an innocuous name and cover and with no smartphones or computers in the classroom I can guess what kind of book that teacher read. I had just finished reading “Satanic Verses” because I wanted to know what was in it that made people try to hunt down the author and kill him and I guess that book was fine but not the one my mom wanted me to read.

      My problems were actually mostly at home. I do have some of the boy traits but now that girls are being studied I can REALLY look back and see it. I mean my parents had to PAY me to shut up. And my rates went up quickly and I still never made much money. That’s probably obvious. Recently my uncle wanted a turn to talk and did something much more effective. He handed me a shiny metal object and that kept me quiet much longer. I’m not talking a quarter either. This was interesting. It had a complex design and texture and looked different when I tilted or turned it. I wanted to keep it but I knew I couldn’t. I ran my fingers over it to feel the textures and looked at it from every angle. I even asked to see it again after I gave it back. I really wanted this but knew he could get in a lot of trouble if he let me keep it (it wasn’t a weapon or anything). It was OK for him to have but even though it became a souvineer of previous work, he probably can’t give it away even though, to quote him, “it’s useless.”

      I no longer question my diagnosis of ADHD. I have the symptoms. I can look back and see them from childhood when I didn’t have a ton of other health issues and got by on luck, fear and intelligence. The meds help. But most of all, I am an adult and was given a shiny metal object to calm me down and it worked!

      I’ve been paying for a Lumosity subscription for a long time, maybe 5 years. I don’t always use it as much as I should, but I use it regularly often enough to keep it, and I’ve gone for long periods doing a training session a day, sometimes playing more because it’s fun. They’ve improved it a lot and there are new games being added often now and it’s adaptive, so learning a game (like with bird watching, for example, I didn’t know the names of most of the birds so it took more turns to get letters when I first started but after I learned the names of the birds, my scores for that game became more reflective of how much it was helping because I didn’t have birds where I couldn’t figure out what it was without all the letters). Other than an initial bump in my score and percentile from just learning the games, my attention has stayed flat while my scores for other areas creep up during periods when I’m using it nearly every day, even memory, which is impaired by meds. So I can see how memory would show improvement if the program is able to hold your attention if memory is also measured by something that holds your attention. But I got a new neurologist recently and he did a mini neurocognitive eval as part of his initial testing that involved distracting me and his conclusion was “you are good at math and your memory sucks.” My response was “I didn’t need to pay you to get that information.” He was trying to be funny and so was I and both of us had valid points. I was there for something else. But as far as Lumosity is concerned, my memory is good and the only reason for that is it holds my attention and doesn’t translate at all into real life. So I have Lumosity measuring both working memory and attention under these artificial conditions and it says my memory is improving (with expected drops from certain med changes) and everything is generally improving, all except my attention, which it has undividedly once I start, so while the two are related, I really don’t see how improving working memory would improve attention and I have data on myself that doesn’t translate into the real world, just like CogMed didn’t translate into the real world, showing everything but attention improving.

      I actually don’t even understand the hypothesis, actually, at least not in the real world. Attention is needed for memory. I don’t see how improving memory has any effect on attention. But then again I can walk around with a fake smile for months and not get any happier from the act of smiling, only from not spreading around my bad mood. I have had muscular injuries from mentally traumatic events get successfully treated after time has passed and it brings up emotions surrounding the event. I may be making a cognitive connection before I realize it. But you can smile without being happy. You can act happy without being happy. You can’t remember data you didn’t get. So I really don’t even see how this could work indirectly.

      Reply

      1. Gina Pera’s avatar

        Hi Danielle,

        Thank you for one of the most interesting comments I can imagine on this topic!

        Gina

Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>