Jan 13, 2017

Brain Training – the “Controversy”

The Meriam-Webster defines training as: “the skill, knowledge, or experience acquired by one that trains."  In combination with the word “brain” it becomes a redundant and awkward term, similar to brain-based learning, which also makes no sense as all learning is brain-based. The same holds true for brain-training. Nobody would, for instance, use the compound leg-based running.

In Simons’s (Simons et al., 2016) opinion brain-training is an invention of companies selling cognitive software, making it more appealing to the public – kind of a “hard science” backup. Most often the brain-training term is used to describe commercially available programs, such as CogMed, CogniFit, Lumosity, BrainHQ, Fast ForWord, to name just the most known ones. In contrast,  interventions used in lab-research are predominantly labeled as cognitive training.

The so-called “controversy” relates to statements signed by two groups of scientists. The first one that was released on October 20th 2014,  dubbed  A Consensus on the BrainTraining Industry from the Scientific Community, was put forward by the Max Planck Institute for Human Development and the Stanford Center on Longevity signed by 70 researchers in the field. The main message was that there is a discrepancy between the state of the art in scientific research on cognitive training, on the one hand, and the claims made by the industry advertising commercial cognitive-training software, on the other.

The second one was signed by 133 scientists with the following objections to the first one:
“We agree strongly with parts of your statement, agree substantially with other parts, but are compelled to sign this letter to express our concern that many readers of your statement might wrongly conclude that there is no evidence that any cognitive training regimen can improve cognitive function.”

In the introduction to a recent lengthy paper by Simons et al. (2016) the statement – that in our opinion started mainly as an opposition to false claims about benefits of cognitive training programs advertised by software companies,  mostly Lumosity – is presented as being just about the question: Can intelligence be increased?  NO – the first group, versus YES –  the second group.

In this context it is further worth mentioning that Jaeggi was one of the signers of the first consensus statement, which is surprising given that she was one of the authors of the paper suggesting that training on an adaptive dual n-back task can increase fluid intelligence (Jaeggi et al., 2008). The studies that she and her colleagues conducted afterwards provided similar conclusions (Jaeggi et al., 2010; 2011; 2014; Au et al., 2015; 2016; Buschkuehl et al., 2012;2014;  Katz et al., 2014). The strongest evidence that the so-called controversy was actually invented later and was initially designed to oppose the claims of software companies, was the recent discussion between two opponents: Melby-Lervåg and Hulme (2016 a) on the NO side and Au, Buschkuehl, Duncan, and Jaeggi (2016) on the YES side. We discuss this issue, along with the problem of biased inclusion/exclusion of studies in meta-analyses, which in turn influence conclusions about the malleability of intelligence, in our forthcoming book Increasing Intelligence (Academic Press, Elsevier, release date: February 2017).


In the book we reviewed 17 meta-analyses about the influence of cognitive training on intelligence. We concluded that far transfer effects on measures of fluid intelligence are small and most often obtained after n-back training or training using multi-factorial interventions targeting multiple cognitive skills.

Let's go back to my suggestion that certain meta-analyses have been characterized by biased study selection. Most meta-analyses implying no far transfer effects of training on intelligence have excluded the COGITO study (Schmiedek et al., 2010; 2013; 2014). In this study, although having a no-contact control group, more than 200 individuals were assigned to the training group. The number of participants contributes to the estimated power a study adds to a meta-analysis. In the most recent meta-analysis by  Melby-Lervåg et al. (2016 b), the reason for excluding the COGITO study was: “Did not involve working memory training” (p. 516, Fig.1). The inclusion criteria were further specified: “In cases where an intervention had multiple components, the working memory tasks had to constitute at least 50% of the intervention” (p.517). Schmiedek et al. (2010) included 12 different intervention tasks classified as perceptual speed (6 tasks), working memory (3 tasks) and episodic memory (3 tasks). The 6 memory tasks were adaptive in presentation time which was adjusted in relation to individual performance on trained tasks. In the three episodic memory training tasks the participants had to remember lists of words, pairs of numbers and words, and the position of 12 colored real world objects in a 6 x 6 grid. In our opinion the study would qualify to be included in the meta-analysis by Melby-Lervåg et al. (2016), since 50% of the training tasks actually were WM tasks. It goes without mentioning that the conclusion of the meta-analysis was: “We conclude that there is no evidence that working memory training yields improvements in so-called far-transfer abilities” (p., 526).

Similarly, Simons et al. (2016) described their review paper as “…a comprehensive review of the brain-training literature, one that examines both the quantity and the quality of the evidence according to a well-defined set of best practices” (p., 103). However, the authors in support of the NO-effect do not mention data that opposes their viewpoint. In support of their claim that WM training shows only near transfer effects, Simons et al. (2016; p., 111) cite studies on neural plasticity, for instance, that violinists displayed selective neural growth in the right motor cortex, corresponding to the use of their left hand to finger the strings. On the other hand, they fail to notice the extensive review of brain activation patterns during performance on n-back tasks by Owen et al. (2005). The review provided robust evidence for the activation of 7 brain areas in all n-back variants. Moreover, the identified brain activation patterns during n-back performance also correspond with those that are crucial for intelligent behavior identified by the two of the most influential theories of the relationship between brain structure/function and intelligence – the P-FIT (Jung and Haier, 2007) and the multiple-demand system (Duncan, 2010). The study by Kearney-Ramos et al. (2014) replicated the findings described by Owen et al. (2005).

Furthermore, similar changes in brain structure and function in relation to WM training were reported by several recent studies. Román et al. (2016) showed that adaptive n-back training influenced changes in cortical thickness and surface area. In yet another study by Thompson et al. (2016), it was shown that intensive working memory training produced functional changes in large-scale frontoparietal networks which led the authors to conclude that their study “…provides insight into the adaptive neural systems that underlie large gains in working memory capacity through training” (p., 575). A more humorous perspective on the controversy is presented in a book by Dan Hurley, Smarter: The New Science of Building Brain Power (2015, New York: Plume, Penguin Group). I particularly like the reference to Randall Engle’s position, which underscores my standpoint in an amusing way.
Given that researchers are humans there is probably no meta-analysis without bias – the same holds true for our blog. Let me clarify our standpoint in the controversy: We think that intelligence can be increased, but at the moment only in controlled laboratory experiments, for the purpose of verifying or rejecting findings based on correlational studies, thereby widening our understanding of the construct of intelligence. This idea is best summarized in the contributions of authors provided at the International Seminar: Advances on Intelligence Research: What should we expect from the XXI Century? Held in Madrid, April 7-8, 2016 (see the contribution by Santarnecchi).


References
Au, J., Buschkuehl, M., Duncan, G. J., & Jaeggi, S. M. (2016). There is no convincing evidence that working memory training is NOT effective: A reply to Melby-Lervåg and Hulme (2015). Psychonomic Bulletin & Review, 23(1), 331–337. http://doi.org/10.3758/s13423-015-0967-4
Au, J., Sheehan, E., Tsai, N., Duncan, G. J., Buschkuehl, M., & Jaeggi, S. M. (2015). Improving fluid intelligence with training on working memory: a meta-analysis. Psychonomic Bulletin & Review, 22(2), 366–377. http://doi.org/10.3758/s13423-014-0699-x
Buschkuehl, M., Hernandez-Garcia, L., Jaeggi, S. M., Bernard, J. A., & Jonides, J. (2014). Neural effects of short-term training on working memory. Cognitive, Affective, & Behavioral Neuroscience, 14(1), 147–160. http://doi.org/10.3758/s13415-013-0244-9
Buschkuehl, M., Jaeggi, S. M., & Jonides, J. (2012). Neuronal effects following working memory training. Developmental Cognitive Neuroscience, 2, S167–S179.
Duncan, J. (2010). The multiple-demand (MD) system of the primate brain: mental programs for intelligent behaviour. Trends in Cognitive Sciences, 14(4), 172–179. http://doi.org/10.1016/j.tics.2010.01.004
Jaeggi, S. M., Buschkuehl, M., Jonides, J., & Perrig, W. J. (2008). Improving fluid intelligence with training on working memory. Proceedings of the National Academy of Sciences, 105(19), 6829–6833.
Jaeggi, S. M., Buschkuehl, M., Jonides, J., & Shah, P. (2011). Short- and long-term benefits of cognitive training. Proceedings of the National Academy of Sciences, 108(25), 10081–10086. http://doi.org/10.1073/pnas.1103228108
Jaeggi, S. M., Buschkuehl, M., Shah, P., & Jonides, J. (2014). The role of individual differences in cognitive training and transfer. Memory & Cognition, 42(3), 464–480. http://doi.org/10.3758/s13421-013-0364-z
Jaeggi, S. M., Studer-Luethi, B., Buschkuehl, M., Su, Y.-F., Jonides, J., & Perrig, W. J. (2010). The relationship between n-back performance and matrix reasoning — implications for training and transfer. Intelligence, 38(6), 625–635. http://doi.org/10.1016/j.intell.2010.09.001
Jung, R. E., & Haier, R. J. (2007). The Parieto-Frontal Integration Theory (P-FIT) of intelligence: Converging neuroimaging evidence. Behavioral and Brain Sciences, 30(02), 135. http://doi.org/10.1017/S0140525X07001185
Katz, B., Jaeggi, S., Buschkuehl, M., Stegman, A., & Shah, P. (2014). Differential effect of motivational features on training improvements in school-based cognitive training. Frontiers in Human Neuroscience, 8. http://doi.org/10.3389/fnhum.2014.00242
Kearney-Ramos, T. E., Fausett, J. S., Gess, J. L., Reno, A., Peraza, J., Kilts, C. D., & James, G. A. (2014). Merging Clinical Neuropsychology and Functional Neuroimaging to Evaluate the Construct Validity and Neural Network Engagement of the n-Back Task. Journal of the International Neuropsychological Society, 20(7), 736–750. https://doi.org/10.1017/S135561771400054X
Melby-Lervåg, M., & Hulme, C. (2016 a). There is no convincing evidence that working memory training is effective: A reply to Au et al. (2014) and Karbach and Verhaeghen (2014). Psychonomic Bulletin & Review, 23(1), 324–330. http://doi.org/10.3758/s13423-015-0862-z
Melby-Lervag, M., Redick, T. S., & Hulme, C. (2016 b). Working Memory Training Does Not Improve Performance on Measures of Intelligence or Other Measures of “Far Transfer”: Evidence From a Meta-Analytic Review. Perspectives on Psychological Science, 11(4), 512–534. https://doi.org/10.1177/1745691616635612
Owen, A. M., McMillan, K. M., Laird, A. R., & Bullmore, E. (2005). N-back working memory paradigm: A meta-analysis of normative functional neuroimaging studies. Human Brain Mapping, 25(1), 46–59. https://doi.org/10.1002/hbm.20131
Román, F. J., Lewis, L. B., Chen, C.-H., Karama, S., Burgaleta, M., Martínez, K., … Colom, R. (2016). Gray matter responsiveness to adaptive working memory training: a surface-based morphometry study. Brain Structure and Function, 221(9), 4369–4382. https://doi.org/10.1007/s00429-015-1168-7
Schmiedek, F., Lövden., M., & Lindenberger., U. (2010). Hundred days of cognitive training enhance broad cognitive abilities in adulthood: findings from the COGITO study. Frontiers in Aging Neuroscience. http://doi.org/10.3389/fnagi.2010.00027
Schmiedek, F., Lovden, M., & Lindenberger, U. (2013). Keeping It Steady: Older Adults Perform More Consistently on Cognitive Tasks Than Younger Adults. Psychological Science, 24(9), 1747–1754. http://doi.org/10.1177/0956797613479611
Schmiedek, F., Lövdén, M., & Lindenberger, U. (2014). Younger adults show long-term effects of cognitive training on broad cognitive abilities over 2 years. Developmental Psychology, 50(9), 2304–2310. http://doi.org/10.1037/a0037388
Simons, D. J., Boot, W. R., Charness, N., Gathercole, S. E., Chabris, C. F., Hambrick, D. Z., & Stine-Morrow, E. A. L. (2016). Do “Brain-Training” Programs Work? Psychological Science in the Public Interest, 17(3), 103–186. https://doi.org/10.1177/1529100616661983
Thompson, T. W., Waskom, M. L., & Gabrieli, J. D. E. (2016). Intensive Working Memory Training Produces Functional Changes in Large-scale Frontoparietal Networks. Journal of Cognitive Neuroscience, 28(4), 575–588. https://doi.org/10.1162/jocn_a_00916


6 comments:

  1. Excellent summary of this weird domain. We should remember what Rich Haier acknowledges on his recently published book: "the ultimate purpose of all intelligence research in to enhance intelligence". We, intelligence researchers, will not stop until finding the way to this desirable end. Congratulations for this interesting blog.

    ReplyDelete
    Replies
    1. Roberto, thank you for your positive comment. I agree, we won’t stop until we increase it. This probably is our evolutionary determination, but probably also a necessity facing the threat of intelligent machines

      Delete
  2. Norbert - many thanks for your contributions to this domain. I have been been an avid reader of your work (and that of Dr. Colom) over the last several years while completing a working memory training trial for my PhD work. Two points in response to your above summary - one technical, one philosophical:

    The Technical: My understanding is that the meta-analytic neuroimaging work of Owen et al. (2005; and additionally, that of Rottschy et al., 2012) represents functional activation in response to n-back task completion, but not repetitive 'training' with the n-back task, per se. Very few studies seem to have investigated pre- vs. post-training activation for n-back type tasks (Thompson et al. 2016 being one that has), and perhaps no published work yet has examined pre- versus post-training activation for a transfer task (i.e. Raven's or some other test of fluid intelligence) in addition to the training task itself. Our group is hoping to make contributions in this regard very soon.

    The Philosophical (a borrowed musing from my dissertation discussion): At root, improving cognition is an inherently behavioural endeavour, and must be evaluated primarily on the basis of its behavioural effects, through traditional methods of the behavioural sciences. In doing so, we cannot forget that the ultimate aim of improving intelligence is not to change synapses or axons, nor to gain points on psychometric tests of theoretical measures of one construct or another. Rather, through these latent and intermediary metrics, the ultimate aim of improving intelligence is that which is so succinctly summarized by the brain training industry's ‘deceptive’ advertising: “improve[ing] memory, attention, processing speed and problem solving skills so you can feel more confident in your abilities” (quoted in Redick et al., 2013; pp. 359). In other words, improving cognition is only valuable to the extent that it is able to benefit our behaviour, and our subsequent ruminations of it. William James (1881) expresses this most eloquently:

    The current of life which runs in at our eyes or ears is meant to run out at our hands, feet, or lips. The only use of the thoughts it occasions while inside is to determine its direction to whichever of these organs shall, on the whole, under the circumstances actually present, act in the way most propitious to our welfare. …or in planer English, perception and thinking are only there for behaviour’s sake. (reprinted in James, 1960, pp. 114)

    The present working memory training literature fails this basic test. For all the theorizing about potential effects of working memory training on mediating mechanisms or constructs (i.e. synapses, or myelination, or arborisation, or WM capacity), the observed effects fall short of reliably altering participants’ behaviour in ways ‘propitious to their welfare’ – placebo effects notwithstanding. In James’ terms, the current of working memory training tasks that run in at participants eyes and ears do not appear to reliably or meaningfully run out at their hands, feet, or lips despite potentially causing some interesting phenomena amongst intermediate constructs whilst inside.

    ReplyDelete
    Replies
    1. Dear Cameron, thank you for your comment. About the technical part. There was another study I know (Langer, et al. 2013; http://dx.doi.org/10.1016/j.cortex.2013.01.008). The training was based on Oberauer’s conceptualization of the Intelligence WM overlap – relational integration, defined as the ability of building new relations between elements and thereby creating structural representations (Oberauer, et al., 2008; https://doi.org/10.1016/j.intell.2008.01.007). They used pre post high density EEG recordings and graph theory based analysis of connectivity. But you probably know this study.

      I agree with the ideas in your second part. In the lab we most often forget them, just looking for changes in test scores or biomarkers. Although, at the moment the interventions probably do not allow for more. We can just hope that they will reveal the mechanisms underlying cognition and by that make it possible that interventions will have an impact on the brain to move us in the right direction (e.g., behavior).

      Delete
  3. If you want to increase your memory and reading skills then you must choose quantum speed reading course. This course enables you to read faster than normal. To know more about Quantum speed reading course and their techniques in detail, please visit our website.

    ReplyDelete
  4. Very interesting blog. It would be very good to hear about binaural beats. They have been very trendy since a couple of years. Do you plan on writing on them at some point?
    Another topic I would love to see discussed is the (probably negative) influence of screen exposition on intelligence.
    Best regards.

    ReplyDelete