Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Distinct reinforcement learning profiles distinguish between language and attentional neurodevelopmental disorders

Distinct reinforcement learning profiles distinguish between language and attentional... Background Theoretical models posit abnormalities in cortico-striatal pathways in two of the most common neu- rodevelopmental disorders (Developmental dyslexia, DD, and Attention deficit hyperactive disorder, ADHD), but it is still unclear what distinct cortico-striatal dysfunction might distinguish language disorders from others that exhibit very different symptomatology. Although impairments in tasks that depend on the cortico-striatal network, includ- ing reinforcement learning (RL), have been implicated in both disorders, there has been little attempt to dissociate between different types of RL or to compare learning processes in these two types of disorders. The present study builds upon prior research indicating the existence of two learning manifestations of RL and evaluates whether these processes can be differentiated in language and attention deficit disorders. We used a two-step RL task shown to dis- sociate model-based from model-free learning in human learners. Results Our results show that, relative to neurotypicals, DD individuals showed an impairment in model-free but not in model-based learning, whereas in ADHD the ability to use both model-free and model-based learning strategies was significantly compromised. Conclusions Thus, learning impairments in DD may be linked to a selective deficit in the ability to form action-out - come associations based on previous history, whereas in ADHD some learning deficits may be related to an incapacity to pursue rewards based on the tasks’ structure. Our results indicate how different patterns of learning deficits may underlie different disorders, and how computation-minded experimental approaches can differentiate between them. Keywords Attention-deficit/hyperactivity disorder, Developmental dyslexia, Two-step task, Model-based vs. Model- free reinforcement learning Background Developmental dyslexia (DD) and Attention-deficit/ hyperactivity disorder (ADHD) are two of the most com- mon neurodevelopmental disorders. Dyslexia is char- *Correspondence: Yafit Gabay acterized by difficulties in acquiring reading, writing, ygabay@edu.haifa.ac.il and spelling skills, whereas ADHD is characterized by Department of Special Education, University of Haifa, Haifa, Israel inattention, impulsivity, and hyperactivity symptoms. Edmond J. Safra Brain Research Center for the Study of Learning Disabilities, University of Haifa, 199 Abba Khoushy Ave, Haifa, Israel Traditionally, DD has been suggested to arise from Department of Cognitive Sciences, University of Haifa, Haifa, Israel phonological impairments [87] but domain-general The School of Psychological Sciences, Tel Aviv University, Tel Aviv, Israel accounts postulate sensory [46] or procedural learning Sagol School of Neuroscience, Tel Aviv University, Tel Aviv, Israel impairments [65, 98, 99] in its etiology, thus providing © The Author(s) 2023. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/. The Creative Commons Public Domain Dedication waiver (http:// creat iveco mmons. org/ publi cdoma in/ zero/1. 0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data. Nissan et al. Behavioral and Brain Functions (2023) 19:6 Page 2 of 14 a mechanistic account for the diverse range of linguistic both DD and ADHD individuals are impaired in learn- and nonlinguistic symptoms observed in this disorder. ing information integration categories [49, 88] which are ADHD has been associated with an executive function believed to be acquired via striatal-based RL mechanisms deficit [4], but a growing body of evidence points to key [3]. Finally, both DD [38] and ADHD individuals [41] deficits in motivational/reward-related processes as well are impaired in probabilistic RL tasks when task condi- [7, 36, 37, 59, 69, 73, 77, 80, 89]. There is a high comor - tions favor striatal-based memory engagement rather bidity between these two childhood neurodevelopmental than hippocampal-based memory engagement, similar disorders [105], including shared symptoms such as tem- to a pattern observed among patients with striatal dys- poral processing impairments [22, 94], executive function function [30, 33]. Notably some studies revealed intact deficits [56], and procedural learning deficiencies [1, 110, RL in ADHD, but such findings are mostly found is tasks 34, 54, 57]. in which feedback is deterministic [47, 61] or in studies Despite decades of research, the neurocognitive basis using relatively simple tasks with low number of stimuli of these two disorders is still highly debated and the rea- [14, 48, 58]. son for the overlap is not yet fully understood. Recent advances in the research of comorbidity prompt a change Model‑free vs. model‑based RL from single deficit models to multiple models of develop - Nevertheless, we still do not have a clear understanding mental neuropsychology. According to the multiple defi - of RL phenomena in both DD and ADHD or whether cit model [70], there are multiple probabilistic predictors they are characterized by distinct/shared RL mecha- of neurodevelopmental disorders across levels of analyses nisms. Recent advances in the field of neurocompu - and comorbidity arises due to shared risk factors. tational models of cognition suggest that RL cannot Interestingly theoretical and empirical findings in the be considered a unitary phenomenon. Rather, people research  of DD and ADHD implicate abnormalities in employ different computational strategies when solving cortico-striatal pathways in both disorders [64, 99]. In RL problems. One of these involves learning stimulus– DD, cortico-striatal  disruption [10, 51, 76, 103]  is pre- response contingencies which, after formation, are less sumed to affect the ability to acquire skills, procedures sensitive to outcome and reward (Yin & Knowlton, 2006). and stimulus–response associations acquired incremen- A more prevalent account of learning describes goal-ori- tally [24, 65, 97, 98]. Since language learning critically ented learning by focusing on learning outcome-action depends upon these domain general abilities [21, 97], contingencies. Here, outcome-action contingencies can impaired striatal-based learning is presumed to disrupt be based solely on recent history and presumed to arise the typical course of reading, writing, and spelling skills computationally from model-free (MF) learning. The in those with DD. In ADHD anatomical and functional MF system learns the expected value of actions through abnormalities within the striatum [11] have been sug- prediction errors, which quantify the difference between gested to give rise to impulsive behaviors [45] and neu- the worth of actual and expected outcomes. In addition, robiological models of ADHD posit that the deficit in action-outcome contingencies can be updated through striatal-based learning and memory is likely to arise model-based (MB) RL, which operates by learning a from dopamine dysfunction within the neostriatum [78]. predictive model of multiple world states and action- Recent evidence points to the right caudate as a shared outcome probabilities, and updating action-outcome neural substrate that is likely to be affected in both disor - contingencies by incorporating this information and ders [64]. planning an action course by using this model to evaluate the different outcomes prospectively over multiple future Reinforcement learning world states [13, 15, 20, 107]. Here MB is likely to involve The cortico-striatal network is responsible for reinforce - learning state values based on planning processes [100]. ment learning (RL), the process in which individuals It has been shown that animals and humans use a learn by trial and error to make choices that exploit the mixture of RL processes [13, 15, 20, 107, 111]. Limiting likelihood of rewards and minimize the occurrence of computational resources by concurrent task [66, 67] or penalties [91]. Therefore, based on the notion of cortico- inducing stress [66, 67] hinders MB but not MF learning, striatal abnormalities in both disorders, RL is likely to somewhat in line with observations that learning based be affected as well. Consistent with this assumption, RL on stimulus–response associations is resistant to distrac- deficits have been documented in DD [38, 42, 63, 72, 88] tion [32, 109]. The ability to use MB strategies follows a as well as in ADHD [35, 39, 49, 61, 95]. Impairments have developmental trajectory, as in children MF learning is been observed across RL tasks involving probabilistic more dominant than MB learning [15]. Furthermore, MF feedback such as the Probabilistic Selection Task [35, 63] learning has been shown to be sensitive to core compo- and the Weather Prediction Task [39, 42]. Furthermore, nents of executive functions, such as working memory N issan et al. Behavioral and Brain Functions (2023) 19:6 Page 3 of 14 and cognitive control [66–68]. Finally, in psychiatric dis- state to the next into their planning and decisions in the orders there is an imbalance between the ability to use first step. Such computations, MF association and MB MF vs. MB learning, such that those who have disorders planning, may be uniquely disturbed in DD and ADHD. associated with compulsivity and impulsivity tend to be Krishnan et  al. [53] argued that cortico-striatal dys- impaired in their ability to use MB learning strategies functions have been noted in both language and [43, 102]. Neurobiologically, these two types of learning psychiatric disorders (such as ADHD) and raised the pos- strategies are presumed to rely upon partially distinct sibility that different computational models may explain neural substrates within the basal ganglia. It has been the behavioral learning profile in each disorder. They suggested that the dorsal lateral striatum subserves MF specifically speculated that in developmental language learning whereas the dorsal medial striatum underlies disorders compared to psychiatric disorders (includ- MB learning [44]. Despite this evidence, however, hip- ing ADHD) learning impairments will be less apparent pocampal damage in humans hampers MB learning but when learning state values (the overall reward that one not MF learning [101]. Furthermore, although basal gan- expects when choosing the state as the starting point). glia dopamine levels affect stimulus-response learning However as learning state values is common in MF and and hence are likely to affect MF learning [29], recent MB learning [90], learning state values based on planning evidence points to the possibility that basal ganglia dopa- processes may distinguish between language and atten- mine levels influence the ability to use MB but not MF tional disorders. This notion is consistent with ample learning strategies [82]. Notably, however, computational evidence showing that those with ADHD, but not those stimulations reveal that tonic dopamine levels influence with DD, exhibit planning deficits and prefer immediate the exploitation-exploration behavior trade-off when small rewards to delayed larger rewards [5, 16, 79, 85, learning values is based on previous reinforcement his- 95]. Therefore, one could predict that MB learning will tory [50]. be selectively disrupted in ADHD. On the other hand, deficits in the MF association are likely to be impacted The present study in both disorders, as shown by evidence pointing to an The purpose of the present study was to examine RL impaired ability to learn reinforcement contingencies in behavior in two of the most common yet very differ - DD based on recent history [42, 63, 88] and ADHD [35, ent neurodevelopmental disorders. The theoretical and 39, 49, 61, 95]. empirical body of research points to cortico-striatal abnormalities in both disorders (for a review see [99], Results which may lead to RL difficulties. RL has been studied Data analysis in both ADHD and DD, but there has been no attempt Power analysis to dissociate between different types of RL processes. To determine whether the current study was adequately Although a previous study revealed that methylpheni- powered, we performed an a priori power analysis. Based date increased risk taking in people with ADHD [62], we on prior research, we computed an effect size of  d = 0.65 are aware of no studies that directly examined MB vs. MF for the key group difference in model-based learning [82]. RL learning in ADHD or DD. Likewise, there has been Using the software package G*Power [23] with power little attempt to compare RL in these neurodevelopmen- (1 − β) set at 0.80 and α = 0.05, one-tailed, we determined tal disorders. The two-step task (TST; [13]) represents a that a sample size of 30 per group was required. u Th s, the recently popular approach to creating a task that differ - current study was adequately powered. entiates between MF learning and MB processes and has been tested in a substantial number of studies in humans Screening (e.g., [18, 66, 67, 82, 102, 106, 107]). In this task, a partici- We excluded individuals who stayed with the same pant is required to make two decisions, each taking him response-key for more than 95% of the trials (0 were closer to the outcome stage where a reward is revealed. excluded) or had more than 25% implausible quick reac- TST allows a differentiation between two types of com - tion-times in either the first or second stage (< 150  ms; putations that may lead to impairments in reward-ori- 1dys, 4 ADHD were omitted). For the remaining ented behavior. The first is the MF effect of outcome on respondents we omitted from analysis trials with implau- decisions, by which actions that were rewarded may not sible reaction times (< 150  ms), and the first trial in the be sufficiently enhanced or associated with reward, lead - task (2.48%). ing to a weak association between actions and rewards. The second is the MB effect in which the likelihood that Modal based vs. model free learning a path will lead to a reward is learned. Here, participants Each clinical population group (DD/ADHD) was tested may not incorporate the probabilities of moving from one against its own control group (neurotypcials matched to Nissan et al. Behavioral and Brain Functions (2023) 19:6 Page 4 of 14 the DD group and neurotypicals matched to the ADHD regression, where transition (rare vs. common) and group group, respectively) and each clinical and control group (DD/ADHD vs. control) were entered as fixed effects pre - were matched by age, gender, and non-verbal intelli- dicting second-stage RTs. The regression included an gence. Analyses were performed using  R  (Team, 2020). additional random effect of participants on the intercept Mixed-effect logistic regression models were conducted parameter. using the lme4 package [8]. For both experiments we used the following analyses: Experiment 1: DD vs. controls To assess whether the groups differed in their ability First stage MF vs. MB effects to use MF vs. MB strategies, we evaluated the effect of Table 1 shows the results of this model and Fig. 1A illus- events on each trial (trial n) on the first-step decision in trates the effects. We observed a significant main effect of the subsequent trial (trial n + 1). The two key predictors previous outcome [χ2 (1) = 114.61, p < 0.001] on partici- in trial n were whether or not a reward was received and pants’ choices, showing that participants were more likely whether this occurred after a common or rare transition to stay with their first-stage choice when the previous to the second stage. We evaluated the impact of these trial was rewarded vs. unrewarded, across groups. This events on the chance of repeating the same first-stage effect is indicative of model-free learning across groups. choice in trial n + 1. A pure model free agent is likely to We further found that group modulated this effect, as repeat a first-stage choice that results in reward regard - evident by a significant previous outcome × group inter- less of the previous transition type, predicting a positive action [χ2 (1) = 8.08, p = 0.004], such that the DD group main effect of reward on first-stage stay probabilities. showed a smaller influence of previous outcome on A pure model-based agent, on the other hand, evalu- first-choice stay probability. We also observed a signifi - ates first-stage actions in terms of second-stage alterna - cant previous outcome × previous transition interaction, tives they tend to lead to. To examine the contribution [χ2 (1) = 13.424, p < 0.001], indicative of model-based of these two systems (i.e., MF vs. MB) we calculated a learning. The three-way interaction of reward × transi- mixed effect logistic regression, where previous out - tion × group was not significant [χ2 (1) = 1.52, p = 0.21], come (rewarded vs. unrewarded), previous transition suggesting that people with DD tended to evaluate first- (rare vs. common), group (DD/ADHD vs. control), and stage actions in terms of the second-stage alternatives all related interactions were entered as fixed effects pre - associated with them, similar to how neurotypicals evalu- dicting the probability that the participant would repeat ated them. the same choice (stay probability). We further included in this analysis (and in all further mixed-effects regression analyses), a random effect of participants on the intercept Second‑stage MB effect parameter [31]. Table 2 shows the results of this model and Fig. 1C illus- As an additional measure of model-based abilities, we trates the effects. We found a significant main effect of analyzed second-stage reaction times (RTs) as a function transition [χ2 (1) = 611.35, p < 0.001], where choices fol- of transition (rare vs. common). A previous study showed lowing a rare transition were slower than those follow- that greater deployment of model-based strategies in the ing common transitions. None of the remaining effects first stage led to shorter RTs after common vs. rare tran - with group were significant. This observation is consist - sitions [81]. Thus, the effect of transition on second-stage ent with the finding that those with DD did not differ RTs can serve as an additional estimate for model-based from matched neurotypicals in their ability to use MB involvement [12, 17]. We calculated a mixed effect linear strategies. Table 1 Results of the mixed-effects model of first-stage MF and MB effects Chisq Df Pr (> Chisq) CI (95%) Reward_oneback 114.62 1.00 < 2.2e-16 − 1.12 − 0.30 Transition_oneback 1.69 1.00 0.19 − 0.41 0.21 Group 1.63 1.00 0.20 − 1.04 0.49 Reward_oneback:transition_oneback 13.42 1.00 0.00 − 0.21 0.71 Reward_oneback:group 8.08 1.00 0.00 − 0.69 0.43 Transition_oneback:group 0.99 1.00 0.32 − 0.55 0.21 Reward_oneback:transition_oneback:group 1.52 1.00 0.22 − 0.42 0.77 N issan et al. Behavioral and Brain Functions (2023) 19:6 Page 5 of 14 Fig. 1 Performance of DD/ADHD and controls on the two-step task. A, B Y-axis represents the probability of repeating the same first-stage choice as a function of the transition in the previous trial (common versus rare) and of the outcome (rewarded versus unrewarded). C, D Y-axis represents second-stage reaction times (RTs) as a function of transition (rare vs. common) and group (DD/ADHD vs. controls) Table 2 Results of the mixed-effects model of RT Experiment 2: ADHD vs. controls First‑stage MF vs. MB effects Chisq Df Pr (> Chisq) CI (95%) Table 3 shows the results of this model and Fig. 1B illus- Transition 611.35 1.00 < 2e-16 126.10 232.26 trates the effects. We observed a significant main effect Group 2.67 1.00 0.10 − 142.31 9.89 of previous outcome [χ2 (1) = 92.603, p < 0.001], indica- Transition:group 0.09 1.00 0.76 − 76.84 73.47 tive of model-free learning across groups. However, group modulated this effect, as evident by a significant Table 3 Results of the mixed-effects model of first-stage MF and MB effects Chisq Df Pr (> Chisq) CI (95%) Reward_oneback 92.60 1.00 < 2.2e-16 − 0.39 − 0.12 Transition_oneback 4.17 1.00 0.04 − 0.29 0.05 Group 0.07 1.00 0.80 − 0.20 0.67 Reward_oneback:transition_oneback 15.97 1.00 0.00 − 0.15 0.35 Reward_oneback:group 8.08 1.00 0.00 − 0.53 − 0.15 Transition_oneback:group 0.10 1.00 0.75 − 0.47 0.02 Reward_oneback:transition_oneback:group 4.75 1.00 0.03 0.07 0.77 Nissan et al. Behavioral and Brain Functions (2023) 19:6 Page 6 of 14 disorders. Consistent with previous studies, neurotypical previous outcome × group interaction [χ2 (1) = 8.077, participants in both Study 1 and 2 exhibited a typical use p = 0.01], such that the ADHD group showed a smaller mixture of MF and MB strategies in the two-step task. influence of previous outcome on first-choice stay prob - However, the performance of young adults with DD and ability. We also observed a significant previous out - ADHD differed relative to matched neurotypicals. come × previous transition interaction [χ2 (1) = 15.967, Our results show that compared to matched controls, p < 0.001], indicative of model-based learning. The triple individuals with DD and individuals with ADHD were interaction of reward*transition*group was significant less likely to repeat a choice that was rewarded com- [χ2 (1) = 4.755, p = 0.029], such that ADHD participants pared to neurotypicals. However, those with ADHD but exhibited a reduced MB behavior (i.e., smaller previous not those with DD were less affected by MB considera - outcome × previous transition interaction) compared to tions in their decisions compared to neurotypicals. Sup- neurotypicals. porting this observation, those with ADHD but not those with DD exhibited reduced expectation violation effects, Second‑stage MB effect as reflected by a reduced RT difference between common Table 4 shows the results of this model and Fig. 1D illus- and rare transitions as another indication of lower MB trates the effects. We found a significant main effect learning. of transition [χ2 (1) = 340.94, p < 0.001], where choices The observation of impaired model-based RL in ADHD following a rare transition were slower than those fol- is consistent with previous findings showing that the lowing common transitions. Importantly, there was a sig- ability to use MB strategies is disrupted in disorders nificant transition by group interaction [χ2 (1) = 29.551, characterized by striatal dopamine dysfunction, such as p < 0.001], such that the transition effect (slower Parkinson’s disease [82] and broadens it to populations responses in rare compared to common states) was that are also associated with striatal dopamine alterations higher in the control group compared with the ADHD and impulsive tendencies, such as ADHD. The results are group, consistent with lower deployment of model-based especially consistent with previous findings showing tem - strategies in the first stage for the ADHD compared to poral discounting in those with ADHD [5, 16, 79, 85, 95]. the control group. To test whether both groups exhib- The impaired ability of people with ADHD to use MB ited a transition  effect despite the differences in magni - strategies could arise from several reasons: First, ADHD tude of the effect as indicated by the interaction, pairwise participants can have difficulties/are slower at generating contrasts were calculated using the emmeans function complex internal models of task environments. Another from the emmeans package [60]. Two pairwise contrasts possibility is that they are able to generate internal mod- for the levels of Transition (rare vs. common) were cal- els but fail to exert the cognitive effort required to follow culated for each group using the output of emmeans as these mental models. Finally, it can be the case that MB input for the function contrast together with the Bonfer- learning is overwhelmed by the absence of automatic roni correction for multiple comparisons. The effect of control routines that are normally provided by the MF transition (slower responses in rare cases compared to system, rendering MB learning less effective in ADHD. common states) was significant for both groups (ADHD: The latter possibility, however, is inconsistent with the estimate = 88.8, SE = 27.9, z. ratio = 3.18, p = 0.0015; TD: results of the DD group that demonstrated preserved estimate = 173, SE = 27, z. ratio = 6.410 p < 0.001). MB learning despite impaired reward effect relative to neurotypicals. Future studies are undoubtedly needed in General discussion order to understand the reduced model-based behavior RL impairments have been implicated in both DD and we observed in those with ADHD. The observation of ADHD [35, 39, 42, 49, 61, 63, 88, 95]. Here, we aimed impaired MF and MB learning in ADHD is consistent to determine how different RL types (MF vs. MB) are with neurobiological models of ADHD positing impaired affected in these two most common yet different neu - RL mechanisms [35, 78, 96]. Although these models dif- rodevelopmental disorders, and whether shared and dis- fer in their level of explanation [60] all assume that RL tinct learning profiles could be observed across the two processes are likely to be impaired in ADHD. The pre - sent findings add to this theoretical body of research by pointing to the possibility that RL deficits in ADHD can - Table 4 Results of the mixed-effects model of RT not be conceived as a unitary phenomenon but that two Chisq Df Pr (> Chisq) CI (95%) distinct types of RL processes are likely to be affected in Transition 340.95 1.00 < 2.2e-16 35.26 142.36 this disorder. Despite differences in the ability to use MB Group 2.67 1.00 0.10 − 203.55 − 6.63 strategies in the ADHD and DD groups, a similar previ- Transition:group 29.55 1.00 0.00 6.45 157.96 ous-outcome main effect impairment was observed in N issan et al. Behavioral and Brain Functions (2023) 19:6 Page 7 of 14 both groups compared to neurotypicals. There are several DD consistent with recent observations (Gabay, Roark & possibilities for explaining the reduced previous-outcome Holt, [112]). Procedural learning plays an important role main effect we observed in the two groups. First, such an in language acquisition [97] including the ability to form effect could be explained by noise or an increased ten - sound categories [26, 55]. Impaired category learning via dency to explore the environment [92], which could rea- procedural learning mechanisms could therefore influ - sonably be associated with decreased use of MF strategies ence the ability of people with DD to form precise pho- [28]. This possibility is consistent with recent findings nological representations with negative effects on reading showing that ADHD symptoms are negatively correlated and phonological skills [40]. with win-stay scores [74]. Indeed, computational stimu- Taken together, the present findings reveal an interest - lations reveal an effect of altered dopamine levels on the ing dissociation between attentional and language devel- exploration-exploitation trade-off. As such, altered dopa - opmental disorders. A common deficit in MF association mine levels in ADHD could give rise to such trade-off, may lead to learning impairments in both disorders. Such consistent with neurobiological models of ADHD [35, impairments may be related to attenuated effect or detec - 78, 96]. Notably, increased exploration in DD is less con- tion of outcome valance, or to problems in associating sistent with recent findings showing similar win-stay and the reward with its preceding actions, especially linking lose-shift scores in DD compared to neurotypicals in a it to actions that are twice removed from the outcome probabilistic reinforcement learning task [63]. Another (first-stage decisions). However, the two disorders show possibility is that the ability to learn reinforcement con- different effects of MB mechanisms. While the DD group tingencies based on the recent outcome history is more showed an intact MB representation of the path lead- disrupted in neurodevelopmental disorders compared ing to outcome and the ability to dynamically use this to typical populations [35, 39, 42, 49, 61, 63, 88, 95]. In information when making planning decisions, i.e., think- this regard, some have speculated that MF learning has ing ahead, ADHD participants did not incorporate this notable parallels with procedural learning and that hip- information. This may be because of inappropriate rep - pocampal-based learning is more equivalent with model- resentation of transition probability (i.e., of the path) or based behavior [19]. Considering this, the present results by failing to incorporate this information in decisions. resonate with theoretical models positing a procedural This distinction between planning ahead and updating learning dysfunction in DD alongside intact hippocam- backwards may be a characteristic of other deficiencies pal-based learning abilities [65, 98, 99]. Furthermore, between these two disorders, to be explored in future at first glance the observation of impaired MF and MB studies, and may call for different interventions. Such learning in ADHD is inconsistent with theoretical and findings could be interpreted in light of the multiple defi - empirical research positing impaired striatal-based learn- cit model of developmental disorders, according to which ing in ADHD alongside spared hippocampal-based learn- every developmental disorder involves multiple cognitive ing [6, 41, 45, 99]. However, MB learning is also likely to risk factors [70]. Based on this notion, it may be the case involve additional neural substrates and in particular the that impairments in model-free RL may be one of the dorsolateral prefrontal cortex [86], which has been shown key risk factors for DD and ADHD [71] but that the MB to be affected in ADHD [27]. Therefore, it can be the case learning deficit is related to the defining neuropsycholog - that RL that rely on the dorsolateral prefrontal cortex as ical features of ADHD but not of DD. well are more likely to be affected in ADHD [49], rather The two-step task is one of the most common than RL that are mostly associated with greater activation paradigms that has been suggested to differentiate in hippocampal-based structures [41]. Further studies are between  MF learning and MB processes  and has been required to explore this possibility. tested in a substantial number of typical and impaired A further major contribution of the present study to populations. Nevertheless, caution is warranted in inter- previous literature is the examination of types of strate- preting behavioral performance in this task, as several gies employed by participants with DD during learning. modifications to this paradigm could affect the relative The results of the present study suggest that learning contribution of each system to behavior. For example, it deficits observed in DD might arise from impaired effi - has been shown that MF RL can produce behavioral pat- ciency in using MF-based strategies. Our study therefore terns in the two-step task that could be interpreted as highlights the importance of studying not only learning MB RL [2]. Furthermore, providing explicit instructions deficits in DD but also use of strategies that might have led participants to make primarily model-based choices a role in them. Since rule-based learning may be analo- with little model-free influence [25]. However, in the cur - gous to MB RL and procedural-based strategy may be rent study, we found that ADHD and DD showed distinc- analogous to model-free RL [68], the ability to use proce- tive deviation from the behavior of control participants dural-based strategies should be selectively disrupted in in the same task. This suggests that, to some extent, the Nissan et al. Behavioral and Brain Functions (2023) 19:6 Page 8 of 14 two-step task used here can differentiate between learn - memory  (Digit span test; Wechsler, 1997 [104]), rapid ing processes and provide an informative insight into automatized naming skills (RAN tests;[9], phonological how such learning processes are impaired in different processing (phoneme segmentation, phoneme deletion, neurodevelopmental disorders. It will be important to and Spoonerism), reading skills  [83, 84], and attentional direct future investigations to examining variants of the functions (ASRS; [52]. two-step task in ADHD/DD in order to more precisely These tests were used to assert group differences in understand the nature of MF/MB processes in these neu- reading and phonological abilities. The results, shown rodevelopmental conditions. in Table  6, indicate that the groups did not differ in age, To conclude, in the present study we compared differ - cognitive abilities, or attentional skills, but compared to ent types of RL across DD and ADHD participants and the control group the DD group displayed a profile of their matched controls. Our results show a shared cog- reading disability compatible with the symptomatology nitive deficit in MF learning across participants with DD of developmental dyslexia. This group differed signifi - and ADHD relative to neurotypicals, alongside a deficit cantly from the control group on both rate and accuracy in MB learning that was selectively disrupted only in the measures of word reading and decoding skills. The DD ADHD group. These results suggest that distinct RL pro - group demonstrated deficits also in the three key pho - files can distinguish between language and attentional nological domains: phonological awareness (Spooner- disorders. ism, phoneme segmentation, phoneme deletion), verbal short-term memory (digit span), and rapid naming (rapid automatized naming). Methods Experiment 1: Participants with DD and neurotypical Experimenet 2: Participants with ADHD and neurotypical participants participants Sixty-six university students (35 with DD, 15F and 31 Sixty-five university students (35 with ADHD; 23F and controls, 18F) took part in the study. All participants 30 controls; 22F) took part in the study. All participants were university students in Israel, from families with were university students in Israel, from families with middle to high socioeconomic status. All participants middle to high socioeconomic status.  All participants were screened for being native Hebrew speakers, had no were screened for being native Hebrew speakers, had no history of neurological disorders and/or psychiatric dis- history of neurological disorders and/or psychiatric dis- orders, had normal or corrected-to-normal vision and orders, had normal or corrected-to-normal vision and normal hearing. The inclusion criteria for the DD group normal hearing. The inclusion criteria for the ADHD was (1) a formal diagnosis by a licensed clinician; (2) the group included (1) a formal diagnosis of ADHD by an absence of a formal diagnosis of attention deficit hyper - authorized clinician; (2) positive screening for ADHD activity disorder (ADHD) or a specific language impair - based on the adult ADHD self-report scale (ASRS; [52], ment; (3) a score below the clinical cutoff on  the adult namely a score > = 51; (3) the lack of a formal diagnosis ADHD self-report scale (ASRS); (4) a score below a 1SD of a comorbid developmental disorder such as develop- local norm cut-off for  phonological decoding [108]; (5) mental dyslexia; (4) a cognitive ability score within the a cognitive ability score within the normal range > 10th normal range > 10th percentile  Raven score. The con - percentile  Raven score [75]. Based on these criteria, trol group was composed of individuals with no history three participants with DD were excluded from the final of learning disabilities who exhibited no difficulties in sample. The control group was composed of individuals attentional skills (e.g., did not receive a positive score with no history of learning disabilities who exhibited no of ADHD based on the ASRS) and was matched in age, difficulties in reading (e.g., were above the reading cut - gender, and nonverbal intelligence (assessed by the Raven off  (non-word reading) and was matched in age, gender, test) to the DD group. The Institutional Review Board of and nonverbal intelligence (assessed by the Raven test) to the University of Haifa approved the study (no. 18/099), the DD group. The Institutional Review Board of the Uni - which was conducted in accordance with the Declara- versity of Haifa approved the study (no. 18/099), which tion of Helsinki, with written informed consent provided was conducted in accordance with the Declaration of by all participants. Participants received a compensation Helsinki, with written informed consent provided by all of NIS 120 (approximately $37) for participating in the participants. Participants received a compensation of NIS study. 120 (approximately $37) for participating in the study. All participants underwent a series of cognitive tests to Participants underwent a series of cognitive tests evaluate general intelligence as measured by Raven’s SPM (Table  5) to evaluate basic cognitive ability, assessed by tests [75], as well as tests of attentional (ASRS; [52] and the Raven test [75] as well as tests of verbal short-term reading skills [83]. Details of the tests are presented in N issan et al. Behavioral and Brain Functions (2023) 19:6 Page 9 of 14 Table 5 Psychometric Tests Ability Test Description INTELLECTUAL ABILITY Raven This test is designed to assess nonverbal intelligence. Par- (Raven, Court, & Raven, 1992) ticipants are required to choose an item from the bottom of the figure that will complete the pattern at the top of the figure. The maximum raw score for this test is 60. The test reliability coefficient is .9 VERBAL SHORT-TERM MEMORY Digit Span Wechsler Adult Intelligence Scale ( WAIS-III; In this task, participants are required to recall the numbers [104]) presented auditorily in the order they were presented by the examiner. The maximum total raw score is 28. Task administration is discontinued after a failure to recall two trials with a similar length of digits. The test reliability coef- ficient is .9 DECODING One-minute test of words and One-minute test of These tests aim to assess reading skills. The one-minute nonwords [83] test of words contains nonvowelized words of an equiva- lent level of complexity. The one-minute test of nonwords contains increasingly complex vowelized nonwords. Each test requires the participant to read aloud as quickly and accurately as possible within one minute. The maximum raw score for the one-minute test of words is 168. The maximum raw score for the one-minute test of nonwords is 86 PHONOLOGICAL PROCESSING Phoneme Deletion [9] In this test, participants are required to repeat nonwords without a specific phoneme as rapidly as possible. The nonwords are presented auditorily and vary in complexity, with a maximum total raw score of 25 Phoneme segmentation test [9] This measure assesses the participant’s ability to break a word into its component phonemes. For example, the word fo has two phonemes /f/ /o/. The maximum raw score is 16 Spoonerism Task (developed by Peleg & Ben-Dror) Participants are required to switch the first syllables of two word-pairs and then synthesize the segments to provide new words. The maximum raw score is 12 NAMING SKILLS Rapid Automatized Naming (RAN) [9] Participants are required to orally name items presented visually as rapidly as possible. The exemplars are drawn from a constant category (RAN colors, RAN categories, RAN numerals, and RAN letters). This requires retrieval of a famil- iar phonological code for each stimulus and coordination of phonological and visual (color) or orthographic (letter) information quickly on time. The reliability coefficient of these tests ranges from .98 to .99 ATTENTION Adult ADHD Self-Report Scale (ASRS) An 18-item questionnaire based on the DSM-IV criteria for identifying ADHD in adults. The questions refer to the past 6 months. The ASRS rating scale includes 0–5 rating (very often = 5 points, often = 4 points, sometimes = 3 points, rarely = 2 points, never = 1 point). A total score of more than 51 points is used to identify ADHD Table 5, and the results are shown in Table 7. The groups stage, a choice was made between two spaceships. Par- did not differ significantly in age, intelligence, or reading ticipants were told that these spaceships could fly to one skills. Naturally, the ADHD group differed significantly of two different planets. Each spaceship would land more from the control group in the ADHD measures derived often on a specific planet (i.e., common transition; 70% from the ASRS questionnaire. chance, yet could also land on the alternative planet in a minority of trials (i.e., rare transition; 30% chance. In the Two‑step task second stage, participants were asked to decide between The task was similar to that employed in the study con - two aliens. The selection of each alien led probabilisti - ducted by [82]. Each trial was divided into two stages, cally to a reward determined by independently drifting each of which required a decision (see Fig.  2. In the first Gaussian random walks [standard deviation (SD = 0.025] Nissan et al. Behavioral and Brain Functions (2023) 19:6 Page 10 of 14 Table 6 Demographic and psychometric data of the DD and control groups Measurement Control S.D Dyslexia S.D t value p Age (in years) 25 2.828 25.29 3.579 − 0.354 0.724 Decoding Oral words recognition (accuracy) 118.838 15.132 71.967 22.443 9.641 .001 Oral words recognition (speed) 120.193 15.142 75.838 24.992 8.451 .001 Oral non-words recognition (accuracy) 63.903 11.344 25.258 9.774 14.369 .001 Oral non-words recognition (speed) 67.935 11.132 41.387 12.776 8.723 .001 Naming skills Naming letters (time) 21.774 2.883 25.258 3.759 − 4.094 .001 Naming objects (time) 32.548 4.945 41.032 7.259 − 5.378 .001 Naming numbers (time) 17.419 2.566 21.612 2.917 − 6.009 .001 Naming colors (time) 27.387 5.358 32.935 5.703 − 3.948 .001 Phonological processing Phoneme segmentation (time) 72.774 16.206 147.58 66.229 − 6.109 .001 Phoneme segmentation (accuracy) 15.032 0.982 11.935 3.829 4.362 .001 Phoneme deletion (time) 87.29 13.473 183.806 48.387 − 10.699 .001 Phoneme deletion (accuracy) 23.612 1.819 19.322 5.344 4.231 .001 Spoonerism (time) 109.064 22.196 270.193 113.185 − 7.778 .001 Spoonerism (accuracy) 18.741 1.389 15.29 4.54 4.047 .001 Short verbal working memory Digit span 12.677 2.599 9.838 2.222 4.621 .001 Intellectual ability Raven test 70.161 17.817 64.29 24.985 1.065 0.292 Attentional functions ASRS 32.483 6.762 31.903 9.148 0.284 0.777 Table 7 Demographic and psychometric data of the ADHD and control groups Measurement Control Std. Deviation ADHD Std. Deviation t value p Age (in years) 25.2 3.01 24.33 3.844 0.972 0.335 Decoding Oral words recognition (accuracy) 112.966 14.919 107.466 13.415 1.501 0.139 Oral words recognition (speed) 114.966 14.48 109.766 13.317 1.448 0.153 Short verbal working memory Digit span 11.633 2.326 9.833 2.52 2.875 0.006 Intellectual ability Raven test 61.033 18.601 54 29.058 1.117 0.27 Attentional functions ASRS 32.666 6.686 68.766 7.85 − 19.174 .001 Procedure with a lower boundary of 0.25 probability of reward and The experiment consisted of two sessions. Participants an upper boundary of 0.75, such that the probability of completed a background questionnaire at home and reward from any particular second stage option changed were invited to complete the cognitive battery tests. very slowly from trial to trial. Because the transition from In the second session, participants completed the two- the first stage choice to the second stage planet was sto - step task. Sessions were conducted in a sound-attenu- chastic, first stage choices allowed dissociating two learn - ated booth in front of a 14-in laptop monitor. ing strategies, either MF or MB. N issan et al. Behavioral and Brain Functions (2023) 19:6 Page 11 of 14 Fig. 2 Two-step task designed to assess model free and model based learning. A Stage 1 was a choice between two spaceships. This choice determined the transition to the next stage according to a fixed probability scheme: each spaceship was predominantly associated with one or the other Stage 2 states (i.e. planets) and led there 70% of the time. In Stage 2. Participants selected one of two aliens to learn if they would be rewarded. Each alien was associated with a probability rewarded. Each alien was associated with a probability reward (range: 0.25–0.75) that changed gradually over time, based on Gaussian random walk. B Timing of stages within a single trial C Example of one of the sets of Gaussian random walks that determined the probability of reward at each of the four stage 2 options Acknowledgements Declarations Ms. Noyli Nissan conducted the study under the supervision of Dr. Yafit Gabay. Ethics approval and consent to participate Author contributions The Institutional Review Board of the University of Haifa approved the study, YG: Supervision, Conceptualization, Investigation, Writing—original draft, which was conducted in accordance with the Declaration of Helsinki, with Methodology, Software, Validation, Resources, Funding acquisition, Writing— written informed consent provided by all participants. review & editing. NN: Project administration, Investigation, Data curation, Writing—original draft. UH: Conceptualization, Investigation, Methodol- Consent for publication ogy, Software, Validation, Formal analysis, Writing—review & editing. NS: All authors consent for publication of the manuscript in its present form. Conceptualization, Formal analysis, Writing—review & editing. All authors read and approved the final manuscript. All authors read and approved the final Competing interests manuscript. All authors declare that they have no competing interests. Funding This research was supported by grants from the Israel Science Founda- Received: 31 July 2022 Accepted: 26 January 2023 tion (grant No. 734/22) and the National Institute of Psychobiology in Israel awarded to Yafit Gabay and by Joy Ventures (2020 cycle) awarded to Yafit Gabay and Uri Hertz. Availability of data and materials References The datasets used and/or analyzed during the current study are available from 1. Adi-Japha E, Fox O, Karni A. Atypical acquisition and atypical expression the corresponding author on reasonable request. of memory consolidation gains in a motor skill in young female adults with ADHD. Res Dev Disabil. 2011;32(3):1011–20. 2. Akam T, Costa R, Dayan P. Simple plans or sophisticated habits? State, transition and learning interactions in the two-step task. PLoS Comput Biol. 2015;11(12): e1004648. 3. Ashby FG, Paul EJ, Maddox WT. 4 COVIS. In: Formal approaches in categorization. Pothos EM, Wills AJ (eds), Cambridge University Press, 65–88. 2011. Nissan et al. Behavioral and Brain Functions (2023) 19:6 Page 12 of 14 4. Barkley RA. Behavioral inhibition, sustained attention, and execu- inhibitory control in children with ADHD in the antisaccade task. Brain tive functions: constructing a unifying theory of ADHD. Psychol Bull. Imaging Behav. 2020;14(6):2450–63. 1997;121(1):65. 28. Findling C, Skvortsova V, Dromnelle R, Palminteri S, Wyart V. Computa- 5. Barkley RA, Edwards G, Laneri M, Fletcher K, Metevia L. Executive tional noise in reward-guided learning drives behavioral variability in functioning, temporal discounting, and sense of time in adolescents volatile environments. Nat Neurosci. 2019;22(12):2066–77. with attention deficit hyperactivity disorder (ADHD) and oppositional 29. Foerde K. What are habits and do they depend on the striatum? A view defiant disorder (ODD). J Abnorm Child Psychol. 2001;29(6):541–56. from the study of neuropsychological populations. Curr Opin Behav Sci. 6. Barnes KA, Howard JH Jr, Howard DV, Kenealy L, Vaidya CJ. Two 2018;20:17–24. forms of implicit learning in childhood ADHD. Dev Neuropsychol. 30. Foerde K, Braun EK, Shohamy D. A trade-off between feedback-based 2010;35(5):494–505. learning and episodic memory for feedback events: evidence from 7. Baroni A, Castellanos FX. Neuroanatomic and cognitive abnor- Parkinson’s disease. Neurodegener Dis. 2013;11(2):93–101. malities in attention-deficit/hyperactivity disorder in the era of ‘high 31. Foerde K, Daw ND, Rufin T, Walsh BT, Shohamy D, Steinglass JE. Deficient definition’neuroimaging. Curr Opin Neurobiol. 2015;30:1–8. goal-directed control in a population characterized by extreme goal 8. Bates D, Mächler M, Bolker B, Walker S. Fitting linear mixed-effects pursuit. J Cogn Neurosci. 2021;33(3):463–81. models using lme4. arXiv preprint arXiv: 1406. 5823. 2014. 32. Foerde K, Knowlton BJ, Poldrack RA. Modulation of competing memory 9. Breznitz, Z., & Misra, M. (2003). Speed of processing of the visual–ortho- systems by distraction. Proc Natl Acad Sci. 2006;103(31):11778–83. graphic and auditory–phonological systems in adult dyslexics: The 33. Foerde K, Shohamy D. Feedback timing modulates brain systems for contribution of “asynchrony” to word recognition deficits. Brain and learning in humans. J Neurosci. 2011;31(37):13157–67. language, 85(3), 486-502. 34. Fox O, Karni A, Adi-Japha E. The consolidation of a motor skill in young 10. Brunswick, N., McCrory, E., Price, C. J., Frith, C. D., & Frith, U. (1999). adults with ADHD: shorter practice can be better. Res Dev Disabil. Explicit and implicit processing of words and pseudowords by adult 2016;51:135–44. developmental dyslexics: A search for Wernicke’s Wortschatz?. Brain, 35. Frank MJ, Santamaria A, O’Reilly RC, Willcutt E. Testing computational 122(10), 1901-1917. models of dopamine and noradrenaline dysfunction in atten- 11. Cubillo A, Halari R, Smith A, Taylor E, Rubia K. A review of fronto-striatal tion deficit/hyperactivity disorder. Neuropsychopharmacology. and fronto-cortical brain abnormalities in children and adults with 2007;32(7):1583–99. attention deficit hyperactivity disorder (ADHD) and new evidence for 36. Furukawa E, Bado P, da Costa RQM, Melo B, Erthal P, de Oliveira IP, dysfunction in adults with ADHD during motivation and attention. Mattos P. Reward modality modulates striatal responses to reward Cortex. 2012;48(2):194–215. anticipation in ADHD: effects of affiliative and food stimuli. Psychiatry 12. Culbreth AJ, Westbrook A, Daw ND, Botvinick M, Barch DM. Reduced Res: Neuroimaging. 2022. https:// doi. org/ 10. 1016/j. pscyc hresns. 2022. model-based decision-making in schizophrenia. J Abnorm Psychol. 111561. 2016;125(6):777. 37. Furukawa E, Bado P, Tripp G, Mattos P, Wickens JR, Bramati IE, Tovar-Moll 13. Daw ND, Gershman SJ, Seymour B, Dayan P, Dolan RJ. Model-based F. Abnormal striatal BOLD responses to reward anticipation and reward influences on humans’ choices and striatal prediction errors. Neuron. delivery in ADHD. PloS ONE. 2014;9(2):e89129. 2011;69(6):1204–15. 38. Gabay Y. Delaying feedback compensates for impaired reinforcement 14. De Meyer H, Beckers T, Tripp G, Van der Oord S. Reinforcement contin- learning in developmental dyslexia. Neurobiol Learn Mem. 2021;185: gency learning in children with ADHD: back to the basics of behavior 107518. therapy. J Abnorm Child Psychol. 2019;47(12):1889–902. 39. Gabay Y, Goldfarb L. Feedback-based probabilistic category learning is 15. Decker JH, Otto AR, Daw ND, Hartley CA. From creatures of habit to selectively impaired in attention/hyperactivity deficit disorder. Neuro - goal-directed learners: tracking the developmental emergence of biol Learn Mem. 2017;142:200–8. model-based reinforcement learning. Psychol Sci. 2016;27(6):848–58. 40. Gabay Y, Holt LL. Incidental learning of sound categories is impaired in 16. Demurie E, Roeyers H, Baeyens D, Sonuga-Barke E. Temporal discount- developmental dyslexia. Cortex. 2015;73:131–43. ing of monetary rewards in children and adolescents with ADHD and 41. Gabay Y, Shahbari-Khateb E, Mendelsohn A. Feedback timing autism spectrum disorders. Dev Sci. 2012;15(6):791–800. modulates probabilistic learning in adults with ADHD. Sci Rep. 17. Deserno L, Huys QJ, Boehme R, Buchert R, Heinze H-J, Grace AA, 2018;8(1):15524. Schlagenhauf F. Ventral striatal dopamine reflects behavioral and neural 42. Gabay Y, Vakil E, Schiff R, Holt LL. Probabilistic category learning in signatures of model-based control during sequential decision making. developmental dyslexia: evidence from feedback and paired-associate Proc Natl Acad Sci. 2015;112(5):1595–600. weather prediction tasks. Neuropsychology. 2015;29(6):844. 18. Dezfouli A, Balleine BW. Actions, action sequences and habits: evidence 43. Gillan CM, Otto AR, Phelps EA, Daw ND. Model-based learning protects that goal-directed and habitual action control are hierarchically organ- against forming habits. Cogn Aec ff t Behav Neurosci. 2015;15(3):523–36. ized. PLoS Comput Biol. 2013;9(12): e1003364. 44. Gläscher J, Daw N, Dayan P, O’Doherty JP. States versus rewards: dis- 19. Doll BB, Shohamy D, Daw ND. Multiple memory systems as substrates sociable neural prediction error signals underlying model-based and for multiple decision systems. Neurobiol Learn Mem. 2015;117:4–13. model-free reinforcement learning. Neuron. 2010;66(4):585–95. 20. Drummond N, Niv Y. Model-based decision making and model-free 45. Goodman J, Marsh R, Peterson BS, Packard MG. Annual research review: learning. Curr Biol. 2020;30(15):R860–5. the neurobehavioral development of multiple memory systems–impli- 21. Earle FS, Del Tufo SN, Evans TM, Lum JA, Cutting LE, Ullman MT. cations for childhood and adolescent psychiatric disorders. J Child Domain-general learning and memory substrates of reading acquisi- Psychol Psychiatry. 2014;55(6):582–610. tion. Mind Brain Educ. 2020;14(2):176–86. 46. Goswami U. A temporal sampling framework for developmental dys- 22. Farmer ME, Klein RM. The evidence for a temporal processing deficit lexia. Trends Cogn Sci. 2011;15(1):3–10. linked to dyslexia: a review. Psychon Bull Rev. 1995;2(4):460–93. 47. Groen Y, Wijers AA, Mulder LJ, Waggeveld B, Minderaa RB, Althaus M. 23. Faul F, Erdfelder E, Buchner A, Lang A-G. Statistical power analyses using Error and feedback processing in children with ADHD and children with G* power 3.1: tests for correlation and regression analyses. Behav Res autistic spectrum disorder: an EEG event-related potential study. Clin Methods. 2009;41(4):1149–60. Neurophysiol. 2008;119(11):2476–93. 24. Fawcett AJ, Nicolson RI. Development of dyslexia: the delayed neural 48. Hauser TU, Iannaccone R, Ball J, Mathys C, Brandeis D, Walitza S, Brem commitment framework. Front Behav Neurosci. 2019;13:112. S. Role of the medial prefrontal cortex in impaired decision making 25. Feher da Silva C, Hare TA. Humans primarily use model-based inference in juvenile attention-deficit/hyperactivity disorder. JAMA Psychiat. in the two-stage task. Nat Hum Behav. 2020;4(10):1053–66. 2014;71(10):1165–73. 26. Feng G, Yi HG, Chandrasekaran B. The role of the human audi- 49. Huang-Pollock CL, Maddox WT, Tam H. Rule-based and information- tory corticostriatal network in speech learning. Cereb Cortex. integration perceptual category learning in children with attention- 2019;29(10):4077–89. deficit/hyperactivity disorder. Neuropsychology. 2014;28(4):594. 27. Fernandez-Ruiz J, Hakvoort Schwerdtfeger RM, Alahyane N, Brien DC, Coe BC, Munoz DP. Dorsolateral prefrontal cortex hyperactivity during N issan et al. Behavioral and Brain Functions (2023) 19:6 Page 13 of 14 50. Humphries MD, Khamassi M, Gurney K. Dopaminergic control of the population: a meta-analytic review of the fMRI literature. Neurosci exploration-exploitation trade-off via the basal ganglia. Front Neurosci. Biobehav Rev. 2014;38:125–34. 2012. https:// doi. org/ 10. 3389/ fnins. 2012. 00009. 74. Portengen CM, Sprooten E, Zwiers MP, Hoekstra PJ, Dietrich A, Holz 51. Kita Y, Yamamoto H, Oba K, Terasawa Y, Moriguchi Y, Uchiyama H, Ina- NE, Saam MC. Reward and punishment sensitivity are associated with gaki M. Altered brain activity for phonological manipulation in dyslexic cross-disorder traits. Psychiatry Res. 2021;298:113795. Japanese children. Brain. 2013;136(12):3696–708. 75. Raven JC, Court JH. Raven’s progressive matrices and vocabulary scales. 52. Konfortes H. Diagnosing ADHD in Israeli adults: the psychometric New Delhi: Oxford pyschologists Press; 1998. properties of the adult ADHD self report scale (ASRS) in Hebrew. Isr J 76. Richlan F, Kronbichler M, Wimmer H. Meta-analyzing brain dysfunctions Psychiatry. 2010;47(4):308. in dyslexic children and adults. Neuroimage. 2011;56(3):1735–42. 53. Krishnan S, Watkins KE, Bishop DV. Neurobiological basis of language 77. Sagvolden T, Aase H, Zeiner P, Berger D. Altered reinforcement mecha- learning difficulties. Trends Cogn Sci. 2016;20(9):701–14. nisms in attention-deficit/hyperactivity disorder. Behav Brain Res. 54. Laasonen M, Väre J, Oksanen-Hennah H, Leppämäki S, Tani P, Harno H, 1998;94(1):61–71. Cleeremans A. Project DyAdd: Implicit learning in adult dyslexia and 78. Sagvolden T, Johansen EB, Aase H, Russell VA. A dynamic developmen- ADHD. Ann Dyslexia. 2014;64(1):1–33. tal theory of attention-deficit/hyperactivity disorder (ADHD) predomi- 55. Lim S-J, Fiez JA, Holt LL. Role of the striatum in incidental learning of nantly hyperactive/impulsive and combined subtypes. Behav Brain Sci. sound categories. Proc Natl Acad Sci. 2019;116(10):4671–80. 2005;28(3):397–418. 56. Lonergan A, Doyle C, Cassidy C, MacSweeney Mahon S, Roche RA, 79. Scheres A, Dijkstra M, Ainslie E, Balkan J, Reynolds B, Sonuga-Barke E, Boran L, Bramham J. A meta-analysis of executive functioning in Castellanos FX. Temporal and probabilistic discounting of rewards in dyslexia with consideration of the impact of comorbid ADHD. J Cogn children and adolescents: effects of age and ADHD symptoms. Neu- Psychol. 2019;31(7):725–49. ropsychologia. 2006;44(11):2092–103. 57. Lum JA, Ullman MT, Conti-Ramsden G. Procedural learning is 80. Scheres A, Milham MP, Knutson B, Castellanos FX. Ventral striatal impaired in dyslexia: evidence from a meta-analysis of serial reaction hyporesponsiveness during reward anticipation in attention-deficit/ time studies. Res Dev Disabil. 2013;34(10):3460–76. hyperactivity disorder. Biol Psychiat. 2007;61(5):720–4. 58. Luman M, Goos V, Oosterlaan J. Instrumental learning in ADHD 81. Shahar N, Hauser TU, Moutoussis M, Moran R, Keramati M, Consortium in a context of reward: intact learning curves and performance N, Dolan RJ. Improving the reliability of model-based decision-making improvement with methylphenidate. J Abnorm Child Psychol. estimates in the two-stage decision task with reaction-times and drift- 2015;43(4):681–91. diffusion modeling. PLoS Comput Biol. 2019;15(2):e1006803. 59. Luman M, Oosterlaan J, Sergeant JA. The impact of reinforcement 82. Sharp ME, Foerde K, Daw ND, Shohamy D. Dopamine selectively reme- contingencies on AD/HD: a review and theoretical appraisal. Clin diates ‘model-based’reward learning: a computational approach. Brain. Psychol Rev. 2005;25(2):183–213. 2016;139(2):355–64. 60. Luman M, Tripp G, Scheres A. Identifying the neurobiology of altered 83. Shatil E. One-minute test for pseudowords. Unpublished test. Haifa: reinforcement sensitivity in ADHD: a review and research agenda. University of Haifa; 1995. Neurosci Biobehav Rev. 2010;34(5):744–54. 84. Shatil E. One-minute test for words-unpublished test. Haifa: University 61. Luman M, Van Meel CS, Oosterlaan J, Sergeant JA, Geurts HM. Does of Haifa; 1997. reward frequency or magnitude drive reinforcement-learning in atten- 85. Shiels K, Hawk LW Jr, Reynolds B, Mazzullo RJ, Rhodes JD, Pelham WE tion-deficit/hyperactivity disorder? Psychiatry Res. 2009;168(3):222–9. Jr, Gangloff BP. Eec ff ts of methylphenidate on discounting of delayed 62. Mandali A, Sethi A, Cercignani M, Harrison NA, Voon V. Shifting uncer- rewards in attention deficit/hyperactivity disorder. Expe Clin Psychop - tainty intolerance: methylphenidate and attention-deficit hyperactivity harmacol. 2009;17(5):291. disorder. Transl Psychiatry. 2021;11(1):1–9. 86. Smittenaar P, FitzGerald TH, Romei V, Wright ND, Dolan RJ. Disruption 63. Massarwe AO, Nissan N, Gabay Y. Atypical reinforcement learning in of dorsolateral prefrontal cortex decreases model-based in favor of developmental dyslexia. J Int Neuropsychol Soc. 2021. https:// doi. org/ model-free control in humans. Neuron. 2013;80(4):914–9. 10. 1017/ S1355 61772 10002 66. 87. Snowling MJ. From language to reading and dyslexia. Dyslexia. 64. McGrath LM, Stoodley CJ. Are there shared neural correlates between 2001;7(1):37–46. dyslexia and ADHD? A meta-analysis of voxel-based morphometry 88. Sperling AJ, Lu Z-L, Manis FR. Slower implicit categorical learning in studies. J Neurodev Disord. 2019;11(1):1–20. adult poor readers. Ann Dyslexia. 2004;54(2):281–303. 65. Nicolson RI, Fawcett AJ. Dyslexia, dysgraphia, procedural learning and 89. Ströhle A, Stoy M, Wrase J, Schwarzer S, Schlagenhauf F, Huss M, Gregor the cerebellum. Cortex: A Journal Devoted to the Study of the Nervous A. Reward anticipation and outcomes in adult males with attention- System and Behavior. 2011. https:// doi. org/ 10. 1016/j. cortex. 2009. 08. deficit/hyperactivity disorder. NeuroImage. 2008;39(3):966–72. 016. 90. Sutton, R. S., & Barto, A. G. Introduction to reinforcement learning. 1998. 66. Otto AR, Gershman SJ, Markman AB, Daw ND. The curse of planning: 91. Sutton, R. & Barto, A. Reinforcement Learning: An Introduction (MIT dissecting multiple reinforcement-learning systems by taxing the Press, 1998). central executive. Psychol Sci. 2013;24(5):751–61. 92. Taylor, H.; Vestergaard, M.D. Developmental Dyslexia: Disorder or Spe- 67. Otto AR, Raio CM, Chiang A, Phelps EA, Daw ND. Working-memory cialization in Exploration? Front. Psychol. 2022, 13, 889245. capacity protects model-based learning from stress. Proc Natl Acad Sci. 93. Team, R. C. R: the R project for statistical computing. 2019. Accessed 2013;110(52):20941–6. Feb, 28. 2020 68. Otto AR, Skatova A, Madlon-Kay S, Daw ND. Cognitive control pre- 94. Toplak, M. E., Dockstader, C., & Tannock, R. (2006). Temporal information dicts use of model-based reinforcement learning. J Cogn Neurosci. processing in ADHD: findings to date and new methods. Journal of 2014;27(2):319–33. neuroscience methods, 151(1), 15-29. 69. Paloyelis Y, Mehta MA, Faraone SV, Asherson P, Kuntsi J. Striatal sensitiv- 95. Tripp G, Alsop B. Sensitivity to reward frequency in boys with attention ity during reward processing in attention-deficit/hyperactivity disorder. deficit hyperactivity disorder. J Clin Child Psychol. 1999;28(3):366–75. J Am Acad Child Adolesc Psychiatry. 2012;51(7):722–32. 96. Tripp G, Wickens JR. Research review: dopamine transfer deficit: a 70. Pennington BF. From single to multiple deficit models of developmen- neurobiological theory of altered reinforcement mechanisms in ADHD. tal disorders. Cognition. 2006;101(2):385–413. J Child Psychol Psychiatry. 2008;49(7):691–704. 71. Pennington BF, Bishop DV. Relations among speech, language, and 97. Ullman MT. Contributions of memory circuits to language: the declara- reading disorders. Annu Rev Psychol. 2009;60:283–306. tive/procedural model. Cognition. 2004;92(1–2):231–70. 72. Pereira CLW, Zhou R, Pitt MA, Myung JI, Rossi PJ, Caverzasi E, Meyer M. 98. Ullman MT, Earle FS, Walenski M, Janacsek K. The neurocogni- Probabilistic decision-making in children with dyslexia. Front Neurosci. tion of developmental disorders of language. Annu Rev Psychol. 2022. https:// doi. org/ 10. 3389/ fnins. 2022. 782306. 2020;71:389–417. 73. Plichta MM, Scheres A. Ventral–striatal responsiveness during reward 99. Ullman MT, Pullman MY. A compensatory role for declarative anticipation in ADHD and its relation to trait impulsivity in the healthy memory in neurodevelopmental disorders. Neurosci Biobehav Rev. 2015;51:205–22. Nissan et al. Behavioral and Brain Functions (2023) 19:6 Page 14 of 14 100. Vanseijen, H., & Sutton, R. A deeper look at planning as learning from replay. International conference on machine learning. 2015. 101. Vikbladh OM, Meager MR, King J, Blackmon K, Devinsky O, Shohamy D, Daw ND. Hippocampal contributions to model-based planning and spatial memory. Neuron. 2019;102(3):683–93. 102. Voon V, Derbyshire K, Rück C, Irvine MA, Worbe Y, Enander J, Sahakian BJ. Disorders of compulsivity: a common bias towards learning habits. Mol Psychiatry. 2015;20(3):345–52. 103. Wang Z, Yan X, Liu Y, Spray GJ, Deng Y, Cao F. Structural and functional abnormality of the putamen in children with developmental dyslexia. Neuropsychologia. 2019;130:26–37. 104. Wechsler, D. (1997). Wechsler adult intelligence scale-( WAIS-3) San Antonio. TX: Harcourt Assessment. 105. Willcutt EG, Pennington BF. Comorbidity of reading disability and attention-deficit/hyperactivity disorder: differences by gender and subtype. J Learn Disabil. 2000;33(2):179–91. 106. Worbe Y, Palminteri S, Savulich G, Daw N, Fernandez-Egea E, Robbins T, Voon V. Valence-dependent influence of serotonin depletion on model- based choice strategy. Mol Psychiatry. 2016;21(5):624–9. 107. Wunderlich K, Smittenaar P, Dolan RJ. Dopamine enhances model- based over model-free choice behavior. Neuron. 2012;75(3):418–24. 108. Yael W, Tami K, Tali B. The effects of orthographic transparency and familiarity on reading Hebrew words in adults with and without dys- lexia. Ann Dyslexia. 2015;65(2):84–102. 109. Zeithamova D, Maddox WT. Dual-task interference in perceptual cat- egory learning. Mem Cognit. 2006;34(2):387–98. 110. Ballan R, Durrant SJ, Stickgold R, Morgan A, Manoach DS, Gabay Y. A failure of sleep-dependent consolidation of visuoperceptual proce- dural learning in young adults with ADHD. Translational Psychiatry. 2022;12(1):499. 111. Mas-Herrero E, Sescousse G, Cools R, Marco-Pallares J. The contribution of striatal pseudo-reward prediction errors to value-based decision- making. NeuroImage. 2019;193:67–74. 112. Gabay Y, Roark CL, Holt LL. Impaired and Spared Auditory Cat- egory Learning in Developmental Dyslexia. Psychological Science. 2022:09567976231151581. Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in pub- lished maps and institutional affiliations. Re Read ady y to to submit y submit your our re researc search h ? Choose BMC and benefit fr ? Choose BMC and benefit from om: : fast, convenient online submission thorough peer review by experienced researchers in your field rapid publication on acceptance support for research data, including large and complex data types • gold Open Access which fosters wider collaboration and increased citations maximum visibility for your research: over 100M website views per year At BMC, research is always in progress. Learn more biomedcentral.com/submissions http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Behavioral and Brain Functions Springer Journals

Distinct reinforcement learning profiles distinguish between language and attentional neurodevelopmental disorders

Download PDF
14 pages

Loading next page...
 
/lp/springer-journals/distinct-reinforcement-learning-profiles-distinguish-between-language-dv1akLDtxX

References (127)

Publisher
Springer Journals
Copyright
Copyright © The Author(s) 2023
eISSN
1744-9081
DOI
10.1186/s12993-023-00207-w
Publisher site
See Article on Publisher Site

Abstract

Background Theoretical models posit abnormalities in cortico-striatal pathways in two of the most common neu- rodevelopmental disorders (Developmental dyslexia, DD, and Attention deficit hyperactive disorder, ADHD), but it is still unclear what distinct cortico-striatal dysfunction might distinguish language disorders from others that exhibit very different symptomatology. Although impairments in tasks that depend on the cortico-striatal network, includ- ing reinforcement learning (RL), have been implicated in both disorders, there has been little attempt to dissociate between different types of RL or to compare learning processes in these two types of disorders. The present study builds upon prior research indicating the existence of two learning manifestations of RL and evaluates whether these processes can be differentiated in language and attention deficit disorders. We used a two-step RL task shown to dis- sociate model-based from model-free learning in human learners. Results Our results show that, relative to neurotypicals, DD individuals showed an impairment in model-free but not in model-based learning, whereas in ADHD the ability to use both model-free and model-based learning strategies was significantly compromised. Conclusions Thus, learning impairments in DD may be linked to a selective deficit in the ability to form action-out - come associations based on previous history, whereas in ADHD some learning deficits may be related to an incapacity to pursue rewards based on the tasks’ structure. Our results indicate how different patterns of learning deficits may underlie different disorders, and how computation-minded experimental approaches can differentiate between them. Keywords Attention-deficit/hyperactivity disorder, Developmental dyslexia, Two-step task, Model-based vs. Model- free reinforcement learning Background Developmental dyslexia (DD) and Attention-deficit/ hyperactivity disorder (ADHD) are two of the most com- mon neurodevelopmental disorders. Dyslexia is char- *Correspondence: Yafit Gabay acterized by difficulties in acquiring reading, writing, ygabay@edu.haifa.ac.il and spelling skills, whereas ADHD is characterized by Department of Special Education, University of Haifa, Haifa, Israel inattention, impulsivity, and hyperactivity symptoms. Edmond J. Safra Brain Research Center for the Study of Learning Disabilities, University of Haifa, 199 Abba Khoushy Ave, Haifa, Israel Traditionally, DD has been suggested to arise from Department of Cognitive Sciences, University of Haifa, Haifa, Israel phonological impairments [87] but domain-general The School of Psychological Sciences, Tel Aviv University, Tel Aviv, Israel accounts postulate sensory [46] or procedural learning Sagol School of Neuroscience, Tel Aviv University, Tel Aviv, Israel impairments [65, 98, 99] in its etiology, thus providing © The Author(s) 2023. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/. The Creative Commons Public Domain Dedication waiver (http:// creat iveco mmons. org/ publi cdoma in/ zero/1. 0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data. Nissan et al. Behavioral and Brain Functions (2023) 19:6 Page 2 of 14 a mechanistic account for the diverse range of linguistic both DD and ADHD individuals are impaired in learn- and nonlinguistic symptoms observed in this disorder. ing information integration categories [49, 88] which are ADHD has been associated with an executive function believed to be acquired via striatal-based RL mechanisms deficit [4], but a growing body of evidence points to key [3]. Finally, both DD [38] and ADHD individuals [41] deficits in motivational/reward-related processes as well are impaired in probabilistic RL tasks when task condi- [7, 36, 37, 59, 69, 73, 77, 80, 89]. There is a high comor - tions favor striatal-based memory engagement rather bidity between these two childhood neurodevelopmental than hippocampal-based memory engagement, similar disorders [105], including shared symptoms such as tem- to a pattern observed among patients with striatal dys- poral processing impairments [22, 94], executive function function [30, 33]. Notably some studies revealed intact deficits [56], and procedural learning deficiencies [1, 110, RL in ADHD, but such findings are mostly found is tasks 34, 54, 57]. in which feedback is deterministic [47, 61] or in studies Despite decades of research, the neurocognitive basis using relatively simple tasks with low number of stimuli of these two disorders is still highly debated and the rea- [14, 48, 58]. son for the overlap is not yet fully understood. Recent advances in the research of comorbidity prompt a change Model‑free vs. model‑based RL from single deficit models to multiple models of develop - Nevertheless, we still do not have a clear understanding mental neuropsychology. According to the multiple defi - of RL phenomena in both DD and ADHD or whether cit model [70], there are multiple probabilistic predictors they are characterized by distinct/shared RL mecha- of neurodevelopmental disorders across levels of analyses nisms. Recent advances in the field of neurocompu - and comorbidity arises due to shared risk factors. tational models of cognition suggest that RL cannot Interestingly theoretical and empirical findings in the be considered a unitary phenomenon. Rather, people research  of DD and ADHD implicate abnormalities in employ different computational strategies when solving cortico-striatal pathways in both disorders [64, 99]. In RL problems. One of these involves learning stimulus– DD, cortico-striatal  disruption [10, 51, 76, 103]  is pre- response contingencies which, after formation, are less sumed to affect the ability to acquire skills, procedures sensitive to outcome and reward (Yin & Knowlton, 2006). and stimulus–response associations acquired incremen- A more prevalent account of learning describes goal-ori- tally [24, 65, 97, 98]. Since language learning critically ented learning by focusing on learning outcome-action depends upon these domain general abilities [21, 97], contingencies. Here, outcome-action contingencies can impaired striatal-based learning is presumed to disrupt be based solely on recent history and presumed to arise the typical course of reading, writing, and spelling skills computationally from model-free (MF) learning. The in those with DD. In ADHD anatomical and functional MF system learns the expected value of actions through abnormalities within the striatum [11] have been sug- prediction errors, which quantify the difference between gested to give rise to impulsive behaviors [45] and neu- the worth of actual and expected outcomes. In addition, robiological models of ADHD posit that the deficit in action-outcome contingencies can be updated through striatal-based learning and memory is likely to arise model-based (MB) RL, which operates by learning a from dopamine dysfunction within the neostriatum [78]. predictive model of multiple world states and action- Recent evidence points to the right caudate as a shared outcome probabilities, and updating action-outcome neural substrate that is likely to be affected in both disor - contingencies by incorporating this information and ders [64]. planning an action course by using this model to evaluate the different outcomes prospectively over multiple future Reinforcement learning world states [13, 15, 20, 107]. Here MB is likely to involve The cortico-striatal network is responsible for reinforce - learning state values based on planning processes [100]. ment learning (RL), the process in which individuals It has been shown that animals and humans use a learn by trial and error to make choices that exploit the mixture of RL processes [13, 15, 20, 107, 111]. Limiting likelihood of rewards and minimize the occurrence of computational resources by concurrent task [66, 67] or penalties [91]. Therefore, based on the notion of cortico- inducing stress [66, 67] hinders MB but not MF learning, striatal abnormalities in both disorders, RL is likely to somewhat in line with observations that learning based be affected as well. Consistent with this assumption, RL on stimulus–response associations is resistant to distrac- deficits have been documented in DD [38, 42, 63, 72, 88] tion [32, 109]. The ability to use MB strategies follows a as well as in ADHD [35, 39, 49, 61, 95]. Impairments have developmental trajectory, as in children MF learning is been observed across RL tasks involving probabilistic more dominant than MB learning [15]. Furthermore, MF feedback such as the Probabilistic Selection Task [35, 63] learning has been shown to be sensitive to core compo- and the Weather Prediction Task [39, 42]. Furthermore, nents of executive functions, such as working memory N issan et al. Behavioral and Brain Functions (2023) 19:6 Page 3 of 14 and cognitive control [66–68]. Finally, in psychiatric dis- state to the next into their planning and decisions in the orders there is an imbalance between the ability to use first step. Such computations, MF association and MB MF vs. MB learning, such that those who have disorders planning, may be uniquely disturbed in DD and ADHD. associated with compulsivity and impulsivity tend to be Krishnan et  al. [53] argued that cortico-striatal dys- impaired in their ability to use MB learning strategies functions have been noted in both language and [43, 102]. Neurobiologically, these two types of learning psychiatric disorders (such as ADHD) and raised the pos- strategies are presumed to rely upon partially distinct sibility that different computational models may explain neural substrates within the basal ganglia. It has been the behavioral learning profile in each disorder. They suggested that the dorsal lateral striatum subserves MF specifically speculated that in developmental language learning whereas the dorsal medial striatum underlies disorders compared to psychiatric disorders (includ- MB learning [44]. Despite this evidence, however, hip- ing ADHD) learning impairments will be less apparent pocampal damage in humans hampers MB learning but when learning state values (the overall reward that one not MF learning [101]. Furthermore, although basal gan- expects when choosing the state as the starting point). glia dopamine levels affect stimulus-response learning However as learning state values is common in MF and and hence are likely to affect MF learning [29], recent MB learning [90], learning state values based on planning evidence points to the possibility that basal ganglia dopa- processes may distinguish between language and atten- mine levels influence the ability to use MB but not MF tional disorders. This notion is consistent with ample learning strategies [82]. Notably, however, computational evidence showing that those with ADHD, but not those stimulations reveal that tonic dopamine levels influence with DD, exhibit planning deficits and prefer immediate the exploitation-exploration behavior trade-off when small rewards to delayed larger rewards [5, 16, 79, 85, learning values is based on previous reinforcement his- 95]. Therefore, one could predict that MB learning will tory [50]. be selectively disrupted in ADHD. On the other hand, deficits in the MF association are likely to be impacted The present study in both disorders, as shown by evidence pointing to an The purpose of the present study was to examine RL impaired ability to learn reinforcement contingencies in behavior in two of the most common yet very differ - DD based on recent history [42, 63, 88] and ADHD [35, ent neurodevelopmental disorders. The theoretical and 39, 49, 61, 95]. empirical body of research points to cortico-striatal abnormalities in both disorders (for a review see [99], Results which may lead to RL difficulties. RL has been studied Data analysis in both ADHD and DD, but there has been no attempt Power analysis to dissociate between different types of RL processes. To determine whether the current study was adequately Although a previous study revealed that methylpheni- powered, we performed an a priori power analysis. Based date increased risk taking in people with ADHD [62], we on prior research, we computed an effect size of  d = 0.65 are aware of no studies that directly examined MB vs. MF for the key group difference in model-based learning [82]. RL learning in ADHD or DD. Likewise, there has been Using the software package G*Power [23] with power little attempt to compare RL in these neurodevelopmen- (1 − β) set at 0.80 and α = 0.05, one-tailed, we determined tal disorders. The two-step task (TST; [13]) represents a that a sample size of 30 per group was required. u Th s, the recently popular approach to creating a task that differ - current study was adequately powered. entiates between MF learning and MB processes and has been tested in a substantial number of studies in humans Screening (e.g., [18, 66, 67, 82, 102, 106, 107]). In this task, a partici- We excluded individuals who stayed with the same pant is required to make two decisions, each taking him response-key for more than 95% of the trials (0 were closer to the outcome stage where a reward is revealed. excluded) or had more than 25% implausible quick reac- TST allows a differentiation between two types of com - tion-times in either the first or second stage (< 150  ms; putations that may lead to impairments in reward-ori- 1dys, 4 ADHD were omitted). For the remaining ented behavior. The first is the MF effect of outcome on respondents we omitted from analysis trials with implau- decisions, by which actions that were rewarded may not sible reaction times (< 150  ms), and the first trial in the be sufficiently enhanced or associated with reward, lead - task (2.48%). ing to a weak association between actions and rewards. The second is the MB effect in which the likelihood that Modal based vs. model free learning a path will lead to a reward is learned. Here, participants Each clinical population group (DD/ADHD) was tested may not incorporate the probabilities of moving from one against its own control group (neurotypcials matched to Nissan et al. Behavioral and Brain Functions (2023) 19:6 Page 4 of 14 the DD group and neurotypicals matched to the ADHD regression, where transition (rare vs. common) and group group, respectively) and each clinical and control group (DD/ADHD vs. control) were entered as fixed effects pre - were matched by age, gender, and non-verbal intelli- dicting second-stage RTs. The regression included an gence. Analyses were performed using  R  (Team, 2020). additional random effect of participants on the intercept Mixed-effect logistic regression models were conducted parameter. using the lme4 package [8]. For both experiments we used the following analyses: Experiment 1: DD vs. controls To assess whether the groups differed in their ability First stage MF vs. MB effects to use MF vs. MB strategies, we evaluated the effect of Table 1 shows the results of this model and Fig. 1A illus- events on each trial (trial n) on the first-step decision in trates the effects. We observed a significant main effect of the subsequent trial (trial n + 1). The two key predictors previous outcome [χ2 (1) = 114.61, p < 0.001] on partici- in trial n were whether or not a reward was received and pants’ choices, showing that participants were more likely whether this occurred after a common or rare transition to stay with their first-stage choice when the previous to the second stage. We evaluated the impact of these trial was rewarded vs. unrewarded, across groups. This events on the chance of repeating the same first-stage effect is indicative of model-free learning across groups. choice in trial n + 1. A pure model free agent is likely to We further found that group modulated this effect, as repeat a first-stage choice that results in reward regard - evident by a significant previous outcome × group inter- less of the previous transition type, predicting a positive action [χ2 (1) = 8.08, p = 0.004], such that the DD group main effect of reward on first-stage stay probabilities. showed a smaller influence of previous outcome on A pure model-based agent, on the other hand, evalu- first-choice stay probability. We also observed a signifi - ates first-stage actions in terms of second-stage alterna - cant previous outcome × previous transition interaction, tives they tend to lead to. To examine the contribution [χ2 (1) = 13.424, p < 0.001], indicative of model-based of these two systems (i.e., MF vs. MB) we calculated a learning. The three-way interaction of reward × transi- mixed effect logistic regression, where previous out - tion × group was not significant [χ2 (1) = 1.52, p = 0.21], come (rewarded vs. unrewarded), previous transition suggesting that people with DD tended to evaluate first- (rare vs. common), group (DD/ADHD vs. control), and stage actions in terms of the second-stage alternatives all related interactions were entered as fixed effects pre - associated with them, similar to how neurotypicals evalu- dicting the probability that the participant would repeat ated them. the same choice (stay probability). We further included in this analysis (and in all further mixed-effects regression analyses), a random effect of participants on the intercept Second‑stage MB effect parameter [31]. Table 2 shows the results of this model and Fig. 1C illus- As an additional measure of model-based abilities, we trates the effects. We found a significant main effect of analyzed second-stage reaction times (RTs) as a function transition [χ2 (1) = 611.35, p < 0.001], where choices fol- of transition (rare vs. common). A previous study showed lowing a rare transition were slower than those follow- that greater deployment of model-based strategies in the ing common transitions. None of the remaining effects first stage led to shorter RTs after common vs. rare tran - with group were significant. This observation is consist - sitions [81]. Thus, the effect of transition on second-stage ent with the finding that those with DD did not differ RTs can serve as an additional estimate for model-based from matched neurotypicals in their ability to use MB involvement [12, 17]. We calculated a mixed effect linear strategies. Table 1 Results of the mixed-effects model of first-stage MF and MB effects Chisq Df Pr (> Chisq) CI (95%) Reward_oneback 114.62 1.00 < 2.2e-16 − 1.12 − 0.30 Transition_oneback 1.69 1.00 0.19 − 0.41 0.21 Group 1.63 1.00 0.20 − 1.04 0.49 Reward_oneback:transition_oneback 13.42 1.00 0.00 − 0.21 0.71 Reward_oneback:group 8.08 1.00 0.00 − 0.69 0.43 Transition_oneback:group 0.99 1.00 0.32 − 0.55 0.21 Reward_oneback:transition_oneback:group 1.52 1.00 0.22 − 0.42 0.77 N issan et al. Behavioral and Brain Functions (2023) 19:6 Page 5 of 14 Fig. 1 Performance of DD/ADHD and controls on the two-step task. A, B Y-axis represents the probability of repeating the same first-stage choice as a function of the transition in the previous trial (common versus rare) and of the outcome (rewarded versus unrewarded). C, D Y-axis represents second-stage reaction times (RTs) as a function of transition (rare vs. common) and group (DD/ADHD vs. controls) Table 2 Results of the mixed-effects model of RT Experiment 2: ADHD vs. controls First‑stage MF vs. MB effects Chisq Df Pr (> Chisq) CI (95%) Table 3 shows the results of this model and Fig. 1B illus- Transition 611.35 1.00 < 2e-16 126.10 232.26 trates the effects. We observed a significant main effect Group 2.67 1.00 0.10 − 142.31 9.89 of previous outcome [χ2 (1) = 92.603, p < 0.001], indica- Transition:group 0.09 1.00 0.76 − 76.84 73.47 tive of model-free learning across groups. However, group modulated this effect, as evident by a significant Table 3 Results of the mixed-effects model of first-stage MF and MB effects Chisq Df Pr (> Chisq) CI (95%) Reward_oneback 92.60 1.00 < 2.2e-16 − 0.39 − 0.12 Transition_oneback 4.17 1.00 0.04 − 0.29 0.05 Group 0.07 1.00 0.80 − 0.20 0.67 Reward_oneback:transition_oneback 15.97 1.00 0.00 − 0.15 0.35 Reward_oneback:group 8.08 1.00 0.00 − 0.53 − 0.15 Transition_oneback:group 0.10 1.00 0.75 − 0.47 0.02 Reward_oneback:transition_oneback:group 4.75 1.00 0.03 0.07 0.77 Nissan et al. Behavioral and Brain Functions (2023) 19:6 Page 6 of 14 disorders. Consistent with previous studies, neurotypical previous outcome × group interaction [χ2 (1) = 8.077, participants in both Study 1 and 2 exhibited a typical use p = 0.01], such that the ADHD group showed a smaller mixture of MF and MB strategies in the two-step task. influence of previous outcome on first-choice stay prob - However, the performance of young adults with DD and ability. We also observed a significant previous out - ADHD differed relative to matched neurotypicals. come × previous transition interaction [χ2 (1) = 15.967, Our results show that compared to matched controls, p < 0.001], indicative of model-based learning. The triple individuals with DD and individuals with ADHD were interaction of reward*transition*group was significant less likely to repeat a choice that was rewarded com- [χ2 (1) = 4.755, p = 0.029], such that ADHD participants pared to neurotypicals. However, those with ADHD but exhibited a reduced MB behavior (i.e., smaller previous not those with DD were less affected by MB considera - outcome × previous transition interaction) compared to tions in their decisions compared to neurotypicals. Sup- neurotypicals. porting this observation, those with ADHD but not those with DD exhibited reduced expectation violation effects, Second‑stage MB effect as reflected by a reduced RT difference between common Table 4 shows the results of this model and Fig. 1D illus- and rare transitions as another indication of lower MB trates the effects. We found a significant main effect learning. of transition [χ2 (1) = 340.94, p < 0.001], where choices The observation of impaired model-based RL in ADHD following a rare transition were slower than those fol- is consistent with previous findings showing that the lowing common transitions. Importantly, there was a sig- ability to use MB strategies is disrupted in disorders nificant transition by group interaction [χ2 (1) = 29.551, characterized by striatal dopamine dysfunction, such as p < 0.001], such that the transition effect (slower Parkinson’s disease [82] and broadens it to populations responses in rare compared to common states) was that are also associated with striatal dopamine alterations higher in the control group compared with the ADHD and impulsive tendencies, such as ADHD. The results are group, consistent with lower deployment of model-based especially consistent with previous findings showing tem - strategies in the first stage for the ADHD compared to poral discounting in those with ADHD [5, 16, 79, 85, 95]. the control group. To test whether both groups exhib- The impaired ability of people with ADHD to use MB ited a transition  effect despite the differences in magni - strategies could arise from several reasons: First, ADHD tude of the effect as indicated by the interaction, pairwise participants can have difficulties/are slower at generating contrasts were calculated using the emmeans function complex internal models of task environments. Another from the emmeans package [60]. Two pairwise contrasts possibility is that they are able to generate internal mod- for the levels of Transition (rare vs. common) were cal- els but fail to exert the cognitive effort required to follow culated for each group using the output of emmeans as these mental models. Finally, it can be the case that MB input for the function contrast together with the Bonfer- learning is overwhelmed by the absence of automatic roni correction for multiple comparisons. The effect of control routines that are normally provided by the MF transition (slower responses in rare cases compared to system, rendering MB learning less effective in ADHD. common states) was significant for both groups (ADHD: The latter possibility, however, is inconsistent with the estimate = 88.8, SE = 27.9, z. ratio = 3.18, p = 0.0015; TD: results of the DD group that demonstrated preserved estimate = 173, SE = 27, z. ratio = 6.410 p < 0.001). MB learning despite impaired reward effect relative to neurotypicals. Future studies are undoubtedly needed in General discussion order to understand the reduced model-based behavior RL impairments have been implicated in both DD and we observed in those with ADHD. The observation of ADHD [35, 39, 42, 49, 61, 63, 88, 95]. Here, we aimed impaired MF and MB learning in ADHD is consistent to determine how different RL types (MF vs. MB) are with neurobiological models of ADHD positing impaired affected in these two most common yet different neu - RL mechanisms [35, 78, 96]. Although these models dif- rodevelopmental disorders, and whether shared and dis- fer in their level of explanation [60] all assume that RL tinct learning profiles could be observed across the two processes are likely to be impaired in ADHD. The pre - sent findings add to this theoretical body of research by pointing to the possibility that RL deficits in ADHD can - Table 4 Results of the mixed-effects model of RT not be conceived as a unitary phenomenon but that two Chisq Df Pr (> Chisq) CI (95%) distinct types of RL processes are likely to be affected in Transition 340.95 1.00 < 2.2e-16 35.26 142.36 this disorder. Despite differences in the ability to use MB Group 2.67 1.00 0.10 − 203.55 − 6.63 strategies in the ADHD and DD groups, a similar previ- Transition:group 29.55 1.00 0.00 6.45 157.96 ous-outcome main effect impairment was observed in N issan et al. Behavioral and Brain Functions (2023) 19:6 Page 7 of 14 both groups compared to neurotypicals. There are several DD consistent with recent observations (Gabay, Roark & possibilities for explaining the reduced previous-outcome Holt, [112]). Procedural learning plays an important role main effect we observed in the two groups. First, such an in language acquisition [97] including the ability to form effect could be explained by noise or an increased ten - sound categories [26, 55]. Impaired category learning via dency to explore the environment [92], which could rea- procedural learning mechanisms could therefore influ - sonably be associated with decreased use of MF strategies ence the ability of people with DD to form precise pho- [28]. This possibility is consistent with recent findings nological representations with negative effects on reading showing that ADHD symptoms are negatively correlated and phonological skills [40]. with win-stay scores [74]. Indeed, computational stimu- Taken together, the present findings reveal an interest - lations reveal an effect of altered dopamine levels on the ing dissociation between attentional and language devel- exploration-exploitation trade-off. As such, altered dopa - opmental disorders. A common deficit in MF association mine levels in ADHD could give rise to such trade-off, may lead to learning impairments in both disorders. Such consistent with neurobiological models of ADHD [35, impairments may be related to attenuated effect or detec - 78, 96]. Notably, increased exploration in DD is less con- tion of outcome valance, or to problems in associating sistent with recent findings showing similar win-stay and the reward with its preceding actions, especially linking lose-shift scores in DD compared to neurotypicals in a it to actions that are twice removed from the outcome probabilistic reinforcement learning task [63]. Another (first-stage decisions). However, the two disorders show possibility is that the ability to learn reinforcement con- different effects of MB mechanisms. While the DD group tingencies based on the recent outcome history is more showed an intact MB representation of the path lead- disrupted in neurodevelopmental disorders compared ing to outcome and the ability to dynamically use this to typical populations [35, 39, 42, 49, 61, 63, 88, 95]. In information when making planning decisions, i.e., think- this regard, some have speculated that MF learning has ing ahead, ADHD participants did not incorporate this notable parallels with procedural learning and that hip- information. This may be because of inappropriate rep - pocampal-based learning is more equivalent with model- resentation of transition probability (i.e., of the path) or based behavior [19]. Considering this, the present results by failing to incorporate this information in decisions. resonate with theoretical models positing a procedural This distinction between planning ahead and updating learning dysfunction in DD alongside intact hippocam- backwards may be a characteristic of other deficiencies pal-based learning abilities [65, 98, 99]. Furthermore, between these two disorders, to be explored in future at first glance the observation of impaired MF and MB studies, and may call for different interventions. Such learning in ADHD is inconsistent with theoretical and findings could be interpreted in light of the multiple defi - empirical research positing impaired striatal-based learn- cit model of developmental disorders, according to which ing in ADHD alongside spared hippocampal-based learn- every developmental disorder involves multiple cognitive ing [6, 41, 45, 99]. However, MB learning is also likely to risk factors [70]. Based on this notion, it may be the case involve additional neural substrates and in particular the that impairments in model-free RL may be one of the dorsolateral prefrontal cortex [86], which has been shown key risk factors for DD and ADHD [71] but that the MB to be affected in ADHD [27]. Therefore, it can be the case learning deficit is related to the defining neuropsycholog - that RL that rely on the dorsolateral prefrontal cortex as ical features of ADHD but not of DD. well are more likely to be affected in ADHD [49], rather The two-step task is one of the most common than RL that are mostly associated with greater activation paradigms that has been suggested to differentiate in hippocampal-based structures [41]. Further studies are between  MF learning and MB processes  and has been required to explore this possibility. tested in a substantial number of typical and impaired A further major contribution of the present study to populations. Nevertheless, caution is warranted in inter- previous literature is the examination of types of strate- preting behavioral performance in this task, as several gies employed by participants with DD during learning. modifications to this paradigm could affect the relative The results of the present study suggest that learning contribution of each system to behavior. For example, it deficits observed in DD might arise from impaired effi - has been shown that MF RL can produce behavioral pat- ciency in using MF-based strategies. Our study therefore terns in the two-step task that could be interpreted as highlights the importance of studying not only learning MB RL [2]. Furthermore, providing explicit instructions deficits in DD but also use of strategies that might have led participants to make primarily model-based choices a role in them. Since rule-based learning may be analo- with little model-free influence [25]. However, in the cur - gous to MB RL and procedural-based strategy may be rent study, we found that ADHD and DD showed distinc- analogous to model-free RL [68], the ability to use proce- tive deviation from the behavior of control participants dural-based strategies should be selectively disrupted in in the same task. This suggests that, to some extent, the Nissan et al. Behavioral and Brain Functions (2023) 19:6 Page 8 of 14 two-step task used here can differentiate between learn - memory  (Digit span test; Wechsler, 1997 [104]), rapid ing processes and provide an informative insight into automatized naming skills (RAN tests;[9], phonological how such learning processes are impaired in different processing (phoneme segmentation, phoneme deletion, neurodevelopmental disorders. It will be important to and Spoonerism), reading skills  [83, 84], and attentional direct future investigations to examining variants of the functions (ASRS; [52]. two-step task in ADHD/DD in order to more precisely These tests were used to assert group differences in understand the nature of MF/MB processes in these neu- reading and phonological abilities. The results, shown rodevelopmental conditions. in Table  6, indicate that the groups did not differ in age, To conclude, in the present study we compared differ - cognitive abilities, or attentional skills, but compared to ent types of RL across DD and ADHD participants and the control group the DD group displayed a profile of their matched controls. Our results show a shared cog- reading disability compatible with the symptomatology nitive deficit in MF learning across participants with DD of developmental dyslexia. This group differed signifi - and ADHD relative to neurotypicals, alongside a deficit cantly from the control group on both rate and accuracy in MB learning that was selectively disrupted only in the measures of word reading and decoding skills. The DD ADHD group. These results suggest that distinct RL pro - group demonstrated deficits also in the three key pho - files can distinguish between language and attentional nological domains: phonological awareness (Spooner- disorders. ism, phoneme segmentation, phoneme deletion), verbal short-term memory (digit span), and rapid naming (rapid automatized naming). Methods Experiment 1: Participants with DD and neurotypical Experimenet 2: Participants with ADHD and neurotypical participants participants Sixty-six university students (35 with DD, 15F and 31 Sixty-five university students (35 with ADHD; 23F and controls, 18F) took part in the study. All participants 30 controls; 22F) took part in the study. All participants were university students in Israel, from families with were university students in Israel, from families with middle to high socioeconomic status. All participants middle to high socioeconomic status.  All participants were screened for being native Hebrew speakers, had no were screened for being native Hebrew speakers, had no history of neurological disorders and/or psychiatric dis- history of neurological disorders and/or psychiatric dis- orders, had normal or corrected-to-normal vision and orders, had normal or corrected-to-normal vision and normal hearing. The inclusion criteria for the DD group normal hearing. The inclusion criteria for the ADHD was (1) a formal diagnosis by a licensed clinician; (2) the group included (1) a formal diagnosis of ADHD by an absence of a formal diagnosis of attention deficit hyper - authorized clinician; (2) positive screening for ADHD activity disorder (ADHD) or a specific language impair - based on the adult ADHD self-report scale (ASRS; [52], ment; (3) a score below the clinical cutoff on  the adult namely a score > = 51; (3) the lack of a formal diagnosis ADHD self-report scale (ASRS); (4) a score below a 1SD of a comorbid developmental disorder such as develop- local norm cut-off for  phonological decoding [108]; (5) mental dyslexia; (4) a cognitive ability score within the a cognitive ability score within the normal range > 10th normal range > 10th percentile  Raven score. The con - percentile  Raven score [75]. Based on these criteria, trol group was composed of individuals with no history three participants with DD were excluded from the final of learning disabilities who exhibited no difficulties in sample. The control group was composed of individuals attentional skills (e.g., did not receive a positive score with no history of learning disabilities who exhibited no of ADHD based on the ASRS) and was matched in age, difficulties in reading (e.g., were above the reading cut - gender, and nonverbal intelligence (assessed by the Raven off  (non-word reading) and was matched in age, gender, test) to the DD group. The Institutional Review Board of and nonverbal intelligence (assessed by the Raven test) to the University of Haifa approved the study (no. 18/099), the DD group. The Institutional Review Board of the Uni - which was conducted in accordance with the Declara- versity of Haifa approved the study (no. 18/099), which tion of Helsinki, with written informed consent provided was conducted in accordance with the Declaration of by all participants. Participants received a compensation Helsinki, with written informed consent provided by all of NIS 120 (approximately $37) for participating in the participants. Participants received a compensation of NIS study. 120 (approximately $37) for participating in the study. All participants underwent a series of cognitive tests to Participants underwent a series of cognitive tests evaluate general intelligence as measured by Raven’s SPM (Table  5) to evaluate basic cognitive ability, assessed by tests [75], as well as tests of attentional (ASRS; [52] and the Raven test [75] as well as tests of verbal short-term reading skills [83]. Details of the tests are presented in N issan et al. Behavioral and Brain Functions (2023) 19:6 Page 9 of 14 Table 5 Psychometric Tests Ability Test Description INTELLECTUAL ABILITY Raven This test is designed to assess nonverbal intelligence. Par- (Raven, Court, & Raven, 1992) ticipants are required to choose an item from the bottom of the figure that will complete the pattern at the top of the figure. The maximum raw score for this test is 60. The test reliability coefficient is .9 VERBAL SHORT-TERM MEMORY Digit Span Wechsler Adult Intelligence Scale ( WAIS-III; In this task, participants are required to recall the numbers [104]) presented auditorily in the order they were presented by the examiner. The maximum total raw score is 28. Task administration is discontinued after a failure to recall two trials with a similar length of digits. The test reliability coef- ficient is .9 DECODING One-minute test of words and One-minute test of These tests aim to assess reading skills. The one-minute nonwords [83] test of words contains nonvowelized words of an equiva- lent level of complexity. The one-minute test of nonwords contains increasingly complex vowelized nonwords. Each test requires the participant to read aloud as quickly and accurately as possible within one minute. The maximum raw score for the one-minute test of words is 168. The maximum raw score for the one-minute test of nonwords is 86 PHONOLOGICAL PROCESSING Phoneme Deletion [9] In this test, participants are required to repeat nonwords without a specific phoneme as rapidly as possible. The nonwords are presented auditorily and vary in complexity, with a maximum total raw score of 25 Phoneme segmentation test [9] This measure assesses the participant’s ability to break a word into its component phonemes. For example, the word fo has two phonemes /f/ /o/. The maximum raw score is 16 Spoonerism Task (developed by Peleg & Ben-Dror) Participants are required to switch the first syllables of two word-pairs and then synthesize the segments to provide new words. The maximum raw score is 12 NAMING SKILLS Rapid Automatized Naming (RAN) [9] Participants are required to orally name items presented visually as rapidly as possible. The exemplars are drawn from a constant category (RAN colors, RAN categories, RAN numerals, and RAN letters). This requires retrieval of a famil- iar phonological code for each stimulus and coordination of phonological and visual (color) or orthographic (letter) information quickly on time. The reliability coefficient of these tests ranges from .98 to .99 ATTENTION Adult ADHD Self-Report Scale (ASRS) An 18-item questionnaire based on the DSM-IV criteria for identifying ADHD in adults. The questions refer to the past 6 months. The ASRS rating scale includes 0–5 rating (very often = 5 points, often = 4 points, sometimes = 3 points, rarely = 2 points, never = 1 point). A total score of more than 51 points is used to identify ADHD Table 5, and the results are shown in Table 7. The groups stage, a choice was made between two spaceships. Par- did not differ significantly in age, intelligence, or reading ticipants were told that these spaceships could fly to one skills. Naturally, the ADHD group differed significantly of two different planets. Each spaceship would land more from the control group in the ADHD measures derived often on a specific planet (i.e., common transition; 70% from the ASRS questionnaire. chance, yet could also land on the alternative planet in a minority of trials (i.e., rare transition; 30% chance. In the Two‑step task second stage, participants were asked to decide between The task was similar to that employed in the study con - two aliens. The selection of each alien led probabilisti - ducted by [82]. Each trial was divided into two stages, cally to a reward determined by independently drifting each of which required a decision (see Fig.  2. In the first Gaussian random walks [standard deviation (SD = 0.025] Nissan et al. Behavioral and Brain Functions (2023) 19:6 Page 10 of 14 Table 6 Demographic and psychometric data of the DD and control groups Measurement Control S.D Dyslexia S.D t value p Age (in years) 25 2.828 25.29 3.579 − 0.354 0.724 Decoding Oral words recognition (accuracy) 118.838 15.132 71.967 22.443 9.641 .001 Oral words recognition (speed) 120.193 15.142 75.838 24.992 8.451 .001 Oral non-words recognition (accuracy) 63.903 11.344 25.258 9.774 14.369 .001 Oral non-words recognition (speed) 67.935 11.132 41.387 12.776 8.723 .001 Naming skills Naming letters (time) 21.774 2.883 25.258 3.759 − 4.094 .001 Naming objects (time) 32.548 4.945 41.032 7.259 − 5.378 .001 Naming numbers (time) 17.419 2.566 21.612 2.917 − 6.009 .001 Naming colors (time) 27.387 5.358 32.935 5.703 − 3.948 .001 Phonological processing Phoneme segmentation (time) 72.774 16.206 147.58 66.229 − 6.109 .001 Phoneme segmentation (accuracy) 15.032 0.982 11.935 3.829 4.362 .001 Phoneme deletion (time) 87.29 13.473 183.806 48.387 − 10.699 .001 Phoneme deletion (accuracy) 23.612 1.819 19.322 5.344 4.231 .001 Spoonerism (time) 109.064 22.196 270.193 113.185 − 7.778 .001 Spoonerism (accuracy) 18.741 1.389 15.29 4.54 4.047 .001 Short verbal working memory Digit span 12.677 2.599 9.838 2.222 4.621 .001 Intellectual ability Raven test 70.161 17.817 64.29 24.985 1.065 0.292 Attentional functions ASRS 32.483 6.762 31.903 9.148 0.284 0.777 Table 7 Demographic and psychometric data of the ADHD and control groups Measurement Control Std. Deviation ADHD Std. Deviation t value p Age (in years) 25.2 3.01 24.33 3.844 0.972 0.335 Decoding Oral words recognition (accuracy) 112.966 14.919 107.466 13.415 1.501 0.139 Oral words recognition (speed) 114.966 14.48 109.766 13.317 1.448 0.153 Short verbal working memory Digit span 11.633 2.326 9.833 2.52 2.875 0.006 Intellectual ability Raven test 61.033 18.601 54 29.058 1.117 0.27 Attentional functions ASRS 32.666 6.686 68.766 7.85 − 19.174 .001 Procedure with a lower boundary of 0.25 probability of reward and The experiment consisted of two sessions. Participants an upper boundary of 0.75, such that the probability of completed a background questionnaire at home and reward from any particular second stage option changed were invited to complete the cognitive battery tests. very slowly from trial to trial. Because the transition from In the second session, participants completed the two- the first stage choice to the second stage planet was sto - step task. Sessions were conducted in a sound-attenu- chastic, first stage choices allowed dissociating two learn - ated booth in front of a 14-in laptop monitor. ing strategies, either MF or MB. N issan et al. Behavioral and Brain Functions (2023) 19:6 Page 11 of 14 Fig. 2 Two-step task designed to assess model free and model based learning. A Stage 1 was a choice between two spaceships. This choice determined the transition to the next stage according to a fixed probability scheme: each spaceship was predominantly associated with one or the other Stage 2 states (i.e. planets) and led there 70% of the time. In Stage 2. Participants selected one of two aliens to learn if they would be rewarded. Each alien was associated with a probability rewarded. Each alien was associated with a probability reward (range: 0.25–0.75) that changed gradually over time, based on Gaussian random walk. B Timing of stages within a single trial C Example of one of the sets of Gaussian random walks that determined the probability of reward at each of the four stage 2 options Acknowledgements Declarations Ms. Noyli Nissan conducted the study under the supervision of Dr. Yafit Gabay. Ethics approval and consent to participate Author contributions The Institutional Review Board of the University of Haifa approved the study, YG: Supervision, Conceptualization, Investigation, Writing—original draft, which was conducted in accordance with the Declaration of Helsinki, with Methodology, Software, Validation, Resources, Funding acquisition, Writing— written informed consent provided by all participants. review & editing. NN: Project administration, Investigation, Data curation, Writing—original draft. UH: Conceptualization, Investigation, Methodol- Consent for publication ogy, Software, Validation, Formal analysis, Writing—review & editing. NS: All authors consent for publication of the manuscript in its present form. Conceptualization, Formal analysis, Writing—review & editing. All authors read and approved the final manuscript. All authors read and approved the final Competing interests manuscript. All authors declare that they have no competing interests. Funding This research was supported by grants from the Israel Science Founda- Received: 31 July 2022 Accepted: 26 January 2023 tion (grant No. 734/22) and the National Institute of Psychobiology in Israel awarded to Yafit Gabay and by Joy Ventures (2020 cycle) awarded to Yafit Gabay and Uri Hertz. Availability of data and materials References The datasets used and/or analyzed during the current study are available from 1. Adi-Japha E, Fox O, Karni A. Atypical acquisition and atypical expression the corresponding author on reasonable request. of memory consolidation gains in a motor skill in young female adults with ADHD. Res Dev Disabil. 2011;32(3):1011–20. 2. Akam T, Costa R, Dayan P. Simple plans or sophisticated habits? State, transition and learning interactions in the two-step task. PLoS Comput Biol. 2015;11(12): e1004648. 3. Ashby FG, Paul EJ, Maddox WT. 4 COVIS. In: Formal approaches in categorization. Pothos EM, Wills AJ (eds), Cambridge University Press, 65–88. 2011. Nissan et al. Behavioral and Brain Functions (2023) 19:6 Page 12 of 14 4. Barkley RA. Behavioral inhibition, sustained attention, and execu- inhibitory control in children with ADHD in the antisaccade task. Brain tive functions: constructing a unifying theory of ADHD. Psychol Bull. Imaging Behav. 2020;14(6):2450–63. 1997;121(1):65. 28. Findling C, Skvortsova V, Dromnelle R, Palminteri S, Wyart V. Computa- 5. Barkley RA, Edwards G, Laneri M, Fletcher K, Metevia L. Executive tional noise in reward-guided learning drives behavioral variability in functioning, temporal discounting, and sense of time in adolescents volatile environments. Nat Neurosci. 2019;22(12):2066–77. with attention deficit hyperactivity disorder (ADHD) and oppositional 29. Foerde K. What are habits and do they depend on the striatum? A view defiant disorder (ODD). J Abnorm Child Psychol. 2001;29(6):541–56. from the study of neuropsychological populations. Curr Opin Behav Sci. 6. Barnes KA, Howard JH Jr, Howard DV, Kenealy L, Vaidya CJ. Two 2018;20:17–24. forms of implicit learning in childhood ADHD. Dev Neuropsychol. 30. Foerde K, Braun EK, Shohamy D. A trade-off between feedback-based 2010;35(5):494–505. learning and episodic memory for feedback events: evidence from 7. Baroni A, Castellanos FX. Neuroanatomic and cognitive abnor- Parkinson’s disease. Neurodegener Dis. 2013;11(2):93–101. malities in attention-deficit/hyperactivity disorder in the era of ‘high 31. Foerde K, Daw ND, Rufin T, Walsh BT, Shohamy D, Steinglass JE. Deficient definition’neuroimaging. Curr Opin Neurobiol. 2015;30:1–8. goal-directed control in a population characterized by extreme goal 8. Bates D, Mächler M, Bolker B, Walker S. Fitting linear mixed-effects pursuit. J Cogn Neurosci. 2021;33(3):463–81. models using lme4. arXiv preprint arXiv: 1406. 5823. 2014. 32. Foerde K, Knowlton BJ, Poldrack RA. Modulation of competing memory 9. Breznitz, Z., & Misra, M. (2003). Speed of processing of the visual–ortho- systems by distraction. Proc Natl Acad Sci. 2006;103(31):11778–83. graphic and auditory–phonological systems in adult dyslexics: The 33. Foerde K, Shohamy D. Feedback timing modulates brain systems for contribution of “asynchrony” to word recognition deficits. Brain and learning in humans. J Neurosci. 2011;31(37):13157–67. language, 85(3), 486-502. 34. Fox O, Karni A, Adi-Japha E. The consolidation of a motor skill in young 10. Brunswick, N., McCrory, E., Price, C. J., Frith, C. D., & Frith, U. (1999). adults with ADHD: shorter practice can be better. Res Dev Disabil. Explicit and implicit processing of words and pseudowords by adult 2016;51:135–44. developmental dyslexics: A search for Wernicke’s Wortschatz?. Brain, 35. Frank MJ, Santamaria A, O’Reilly RC, Willcutt E. Testing computational 122(10), 1901-1917. models of dopamine and noradrenaline dysfunction in atten- 11. Cubillo A, Halari R, Smith A, Taylor E, Rubia K. A review of fronto-striatal tion deficit/hyperactivity disorder. Neuropsychopharmacology. and fronto-cortical brain abnormalities in children and adults with 2007;32(7):1583–99. attention deficit hyperactivity disorder (ADHD) and new evidence for 36. Furukawa E, Bado P, da Costa RQM, Melo B, Erthal P, de Oliveira IP, dysfunction in adults with ADHD during motivation and attention. Mattos P. Reward modality modulates striatal responses to reward Cortex. 2012;48(2):194–215. anticipation in ADHD: effects of affiliative and food stimuli. Psychiatry 12. Culbreth AJ, Westbrook A, Daw ND, Botvinick M, Barch DM. Reduced Res: Neuroimaging. 2022. https:// doi. org/ 10. 1016/j. pscyc hresns. 2022. model-based decision-making in schizophrenia. J Abnorm Psychol. 111561. 2016;125(6):777. 37. Furukawa E, Bado P, Tripp G, Mattos P, Wickens JR, Bramati IE, Tovar-Moll 13. Daw ND, Gershman SJ, Seymour B, Dayan P, Dolan RJ. Model-based F. Abnormal striatal BOLD responses to reward anticipation and reward influences on humans’ choices and striatal prediction errors. Neuron. delivery in ADHD. PloS ONE. 2014;9(2):e89129. 2011;69(6):1204–15. 38. Gabay Y. Delaying feedback compensates for impaired reinforcement 14. De Meyer H, Beckers T, Tripp G, Van der Oord S. Reinforcement contin- learning in developmental dyslexia. Neurobiol Learn Mem. 2021;185: gency learning in children with ADHD: back to the basics of behavior 107518. therapy. J Abnorm Child Psychol. 2019;47(12):1889–902. 39. Gabay Y, Goldfarb L. Feedback-based probabilistic category learning is 15. Decker JH, Otto AR, Daw ND, Hartley CA. From creatures of habit to selectively impaired in attention/hyperactivity deficit disorder. Neuro - goal-directed learners: tracking the developmental emergence of biol Learn Mem. 2017;142:200–8. model-based reinforcement learning. Psychol Sci. 2016;27(6):848–58. 40. Gabay Y, Holt LL. Incidental learning of sound categories is impaired in 16. Demurie E, Roeyers H, Baeyens D, Sonuga-Barke E. Temporal discount- developmental dyslexia. Cortex. 2015;73:131–43. ing of monetary rewards in children and adolescents with ADHD and 41. Gabay Y, Shahbari-Khateb E, Mendelsohn A. Feedback timing autism spectrum disorders. Dev Sci. 2012;15(6):791–800. modulates probabilistic learning in adults with ADHD. Sci Rep. 17. Deserno L, Huys QJ, Boehme R, Buchert R, Heinze H-J, Grace AA, 2018;8(1):15524. Schlagenhauf F. Ventral striatal dopamine reflects behavioral and neural 42. Gabay Y, Vakil E, Schiff R, Holt LL. Probabilistic category learning in signatures of model-based control during sequential decision making. developmental dyslexia: evidence from feedback and paired-associate Proc Natl Acad Sci. 2015;112(5):1595–600. weather prediction tasks. Neuropsychology. 2015;29(6):844. 18. Dezfouli A, Balleine BW. Actions, action sequences and habits: evidence 43. Gillan CM, Otto AR, Phelps EA, Daw ND. Model-based learning protects that goal-directed and habitual action control are hierarchically organ- against forming habits. Cogn Aec ff t Behav Neurosci. 2015;15(3):523–36. ized. PLoS Comput Biol. 2013;9(12): e1003364. 44. Gläscher J, Daw N, Dayan P, O’Doherty JP. States versus rewards: dis- 19. Doll BB, Shohamy D, Daw ND. Multiple memory systems as substrates sociable neural prediction error signals underlying model-based and for multiple decision systems. Neurobiol Learn Mem. 2015;117:4–13. model-free reinforcement learning. Neuron. 2010;66(4):585–95. 20. Drummond N, Niv Y. Model-based decision making and model-free 45. Goodman J, Marsh R, Peterson BS, Packard MG. Annual research review: learning. Curr Biol. 2020;30(15):R860–5. the neurobehavioral development of multiple memory systems–impli- 21. Earle FS, Del Tufo SN, Evans TM, Lum JA, Cutting LE, Ullman MT. cations for childhood and adolescent psychiatric disorders. J Child Domain-general learning and memory substrates of reading acquisi- Psychol Psychiatry. 2014;55(6):582–610. tion. Mind Brain Educ. 2020;14(2):176–86. 46. Goswami U. A temporal sampling framework for developmental dys- 22. Farmer ME, Klein RM. The evidence for a temporal processing deficit lexia. Trends Cogn Sci. 2011;15(1):3–10. linked to dyslexia: a review. Psychon Bull Rev. 1995;2(4):460–93. 47. Groen Y, Wijers AA, Mulder LJ, Waggeveld B, Minderaa RB, Althaus M. 23. Faul F, Erdfelder E, Buchner A, Lang A-G. Statistical power analyses using Error and feedback processing in children with ADHD and children with G* power 3.1: tests for correlation and regression analyses. Behav Res autistic spectrum disorder: an EEG event-related potential study. Clin Methods. 2009;41(4):1149–60. Neurophysiol. 2008;119(11):2476–93. 24. Fawcett AJ, Nicolson RI. Development of dyslexia: the delayed neural 48. Hauser TU, Iannaccone R, Ball J, Mathys C, Brandeis D, Walitza S, Brem commitment framework. Front Behav Neurosci. 2019;13:112. S. Role of the medial prefrontal cortex in impaired decision making 25. Feher da Silva C, Hare TA. Humans primarily use model-based inference in juvenile attention-deficit/hyperactivity disorder. JAMA Psychiat. in the two-stage task. Nat Hum Behav. 2020;4(10):1053–66. 2014;71(10):1165–73. 26. Feng G, Yi HG, Chandrasekaran B. The role of the human audi- 49. Huang-Pollock CL, Maddox WT, Tam H. Rule-based and information- tory corticostriatal network in speech learning. Cereb Cortex. integration perceptual category learning in children with attention- 2019;29(10):4077–89. deficit/hyperactivity disorder. Neuropsychology. 2014;28(4):594. 27. Fernandez-Ruiz J, Hakvoort Schwerdtfeger RM, Alahyane N, Brien DC, Coe BC, Munoz DP. Dorsolateral prefrontal cortex hyperactivity during N issan et al. Behavioral and Brain Functions (2023) 19:6 Page 13 of 14 50. Humphries MD, Khamassi M, Gurney K. Dopaminergic control of the population: a meta-analytic review of the fMRI literature. Neurosci exploration-exploitation trade-off via the basal ganglia. Front Neurosci. Biobehav Rev. 2014;38:125–34. 2012. https:// doi. org/ 10. 3389/ fnins. 2012. 00009. 74. Portengen CM, Sprooten E, Zwiers MP, Hoekstra PJ, Dietrich A, Holz 51. Kita Y, Yamamoto H, Oba K, Terasawa Y, Moriguchi Y, Uchiyama H, Ina- NE, Saam MC. Reward and punishment sensitivity are associated with gaki M. Altered brain activity for phonological manipulation in dyslexic cross-disorder traits. Psychiatry Res. 2021;298:113795. Japanese children. Brain. 2013;136(12):3696–708. 75. Raven JC, Court JH. Raven’s progressive matrices and vocabulary scales. 52. Konfortes H. Diagnosing ADHD in Israeli adults: the psychometric New Delhi: Oxford pyschologists Press; 1998. properties of the adult ADHD self report scale (ASRS) in Hebrew. Isr J 76. Richlan F, Kronbichler M, Wimmer H. Meta-analyzing brain dysfunctions Psychiatry. 2010;47(4):308. in dyslexic children and adults. Neuroimage. 2011;56(3):1735–42. 53. Krishnan S, Watkins KE, Bishop DV. Neurobiological basis of language 77. Sagvolden T, Aase H, Zeiner P, Berger D. Altered reinforcement mecha- learning difficulties. Trends Cogn Sci. 2016;20(9):701–14. nisms in attention-deficit/hyperactivity disorder. Behav Brain Res. 54. Laasonen M, Väre J, Oksanen-Hennah H, Leppämäki S, Tani P, Harno H, 1998;94(1):61–71. Cleeremans A. Project DyAdd: Implicit learning in adult dyslexia and 78. Sagvolden T, Johansen EB, Aase H, Russell VA. A dynamic developmen- ADHD. Ann Dyslexia. 2014;64(1):1–33. tal theory of attention-deficit/hyperactivity disorder (ADHD) predomi- 55. Lim S-J, Fiez JA, Holt LL. Role of the striatum in incidental learning of nantly hyperactive/impulsive and combined subtypes. Behav Brain Sci. sound categories. Proc Natl Acad Sci. 2019;116(10):4671–80. 2005;28(3):397–418. 56. Lonergan A, Doyle C, Cassidy C, MacSweeney Mahon S, Roche RA, 79. Scheres A, Dijkstra M, Ainslie E, Balkan J, Reynolds B, Sonuga-Barke E, Boran L, Bramham J. A meta-analysis of executive functioning in Castellanos FX. Temporal and probabilistic discounting of rewards in dyslexia with consideration of the impact of comorbid ADHD. J Cogn children and adolescents: effects of age and ADHD symptoms. Neu- Psychol. 2019;31(7):725–49. ropsychologia. 2006;44(11):2092–103. 57. Lum JA, Ullman MT, Conti-Ramsden G. Procedural learning is 80. Scheres A, Milham MP, Knutson B, Castellanos FX. Ventral striatal impaired in dyslexia: evidence from a meta-analysis of serial reaction hyporesponsiveness during reward anticipation in attention-deficit/ time studies. Res Dev Disabil. 2013;34(10):3460–76. hyperactivity disorder. Biol Psychiat. 2007;61(5):720–4. 58. Luman M, Goos V, Oosterlaan J. Instrumental learning in ADHD 81. Shahar N, Hauser TU, Moutoussis M, Moran R, Keramati M, Consortium in a context of reward: intact learning curves and performance N, Dolan RJ. Improving the reliability of model-based decision-making improvement with methylphenidate. J Abnorm Child Psychol. estimates in the two-stage decision task with reaction-times and drift- 2015;43(4):681–91. diffusion modeling. PLoS Comput Biol. 2019;15(2):e1006803. 59. Luman M, Oosterlaan J, Sergeant JA. The impact of reinforcement 82. Sharp ME, Foerde K, Daw ND, Shohamy D. Dopamine selectively reme- contingencies on AD/HD: a review and theoretical appraisal. Clin diates ‘model-based’reward learning: a computational approach. Brain. Psychol Rev. 2005;25(2):183–213. 2016;139(2):355–64. 60. Luman M, Tripp G, Scheres A. Identifying the neurobiology of altered 83. Shatil E. One-minute test for pseudowords. Unpublished test. Haifa: reinforcement sensitivity in ADHD: a review and research agenda. University of Haifa; 1995. Neurosci Biobehav Rev. 2010;34(5):744–54. 84. Shatil E. One-minute test for words-unpublished test. Haifa: University 61. Luman M, Van Meel CS, Oosterlaan J, Sergeant JA, Geurts HM. Does of Haifa; 1997. reward frequency or magnitude drive reinforcement-learning in atten- 85. Shiels K, Hawk LW Jr, Reynolds B, Mazzullo RJ, Rhodes JD, Pelham WE tion-deficit/hyperactivity disorder? Psychiatry Res. 2009;168(3):222–9. Jr, Gangloff BP. Eec ff ts of methylphenidate on discounting of delayed 62. Mandali A, Sethi A, Cercignani M, Harrison NA, Voon V. Shifting uncer- rewards in attention deficit/hyperactivity disorder. Expe Clin Psychop - tainty intolerance: methylphenidate and attention-deficit hyperactivity harmacol. 2009;17(5):291. disorder. Transl Psychiatry. 2021;11(1):1–9. 86. Smittenaar P, FitzGerald TH, Romei V, Wright ND, Dolan RJ. Disruption 63. Massarwe AO, Nissan N, Gabay Y. Atypical reinforcement learning in of dorsolateral prefrontal cortex decreases model-based in favor of developmental dyslexia. J Int Neuropsychol Soc. 2021. https:// doi. org/ model-free control in humans. Neuron. 2013;80(4):914–9. 10. 1017/ S1355 61772 10002 66. 87. Snowling MJ. From language to reading and dyslexia. Dyslexia. 64. McGrath LM, Stoodley CJ. Are there shared neural correlates between 2001;7(1):37–46. dyslexia and ADHD? A meta-analysis of voxel-based morphometry 88. Sperling AJ, Lu Z-L, Manis FR. Slower implicit categorical learning in studies. J Neurodev Disord. 2019;11(1):1–20. adult poor readers. Ann Dyslexia. 2004;54(2):281–303. 65. Nicolson RI, Fawcett AJ. Dyslexia, dysgraphia, procedural learning and 89. Ströhle A, Stoy M, Wrase J, Schwarzer S, Schlagenhauf F, Huss M, Gregor the cerebellum. Cortex: A Journal Devoted to the Study of the Nervous A. Reward anticipation and outcomes in adult males with attention- System and Behavior. 2011. https:// doi. org/ 10. 1016/j. cortex. 2009. 08. deficit/hyperactivity disorder. NeuroImage. 2008;39(3):966–72. 016. 90. Sutton, R. S., & Barto, A. G. Introduction to reinforcement learning. 1998. 66. Otto AR, Gershman SJ, Markman AB, Daw ND. The curse of planning: 91. Sutton, R. & Barto, A. Reinforcement Learning: An Introduction (MIT dissecting multiple reinforcement-learning systems by taxing the Press, 1998). central executive. Psychol Sci. 2013;24(5):751–61. 92. Taylor, H.; Vestergaard, M.D. Developmental Dyslexia: Disorder or Spe- 67. Otto AR, Raio CM, Chiang A, Phelps EA, Daw ND. Working-memory cialization in Exploration? Front. Psychol. 2022, 13, 889245. capacity protects model-based learning from stress. Proc Natl Acad Sci. 93. Team, R. C. R: the R project for statistical computing. 2019. Accessed 2013;110(52):20941–6. Feb, 28. 2020 68. Otto AR, Skatova A, Madlon-Kay S, Daw ND. Cognitive control pre- 94. Toplak, M. E., Dockstader, C., & Tannock, R. (2006). Temporal information dicts use of model-based reinforcement learning. J Cogn Neurosci. processing in ADHD: findings to date and new methods. Journal of 2014;27(2):319–33. neuroscience methods, 151(1), 15-29. 69. Paloyelis Y, Mehta MA, Faraone SV, Asherson P, Kuntsi J. Striatal sensitiv- 95. Tripp G, Alsop B. Sensitivity to reward frequency in boys with attention ity during reward processing in attention-deficit/hyperactivity disorder. deficit hyperactivity disorder. J Clin Child Psychol. 1999;28(3):366–75. J Am Acad Child Adolesc Psychiatry. 2012;51(7):722–32. 96. Tripp G, Wickens JR. Research review: dopamine transfer deficit: a 70. Pennington BF. From single to multiple deficit models of developmen- neurobiological theory of altered reinforcement mechanisms in ADHD. tal disorders. Cognition. 2006;101(2):385–413. J Child Psychol Psychiatry. 2008;49(7):691–704. 71. Pennington BF, Bishop DV. Relations among speech, language, and 97. Ullman MT. Contributions of memory circuits to language: the declara- reading disorders. Annu Rev Psychol. 2009;60:283–306. tive/procedural model. Cognition. 2004;92(1–2):231–70. 72. Pereira CLW, Zhou R, Pitt MA, Myung JI, Rossi PJ, Caverzasi E, Meyer M. 98. Ullman MT, Earle FS, Walenski M, Janacsek K. The neurocogni- Probabilistic decision-making in children with dyslexia. Front Neurosci. tion of developmental disorders of language. Annu Rev Psychol. 2022. https:// doi. org/ 10. 3389/ fnins. 2022. 782306. 2020;71:389–417. 73. Plichta MM, Scheres A. Ventral–striatal responsiveness during reward 99. Ullman MT, Pullman MY. A compensatory role for declarative anticipation in ADHD and its relation to trait impulsivity in the healthy memory in neurodevelopmental disorders. Neurosci Biobehav Rev. 2015;51:205–22. Nissan et al. Behavioral and Brain Functions (2023) 19:6 Page 14 of 14 100. Vanseijen, H., & Sutton, R. A deeper look at planning as learning from replay. International conference on machine learning. 2015. 101. Vikbladh OM, Meager MR, King J, Blackmon K, Devinsky O, Shohamy D, Daw ND. Hippocampal contributions to model-based planning and spatial memory. Neuron. 2019;102(3):683–93. 102. Voon V, Derbyshire K, Rück C, Irvine MA, Worbe Y, Enander J, Sahakian BJ. Disorders of compulsivity: a common bias towards learning habits. Mol Psychiatry. 2015;20(3):345–52. 103. Wang Z, Yan X, Liu Y, Spray GJ, Deng Y, Cao F. Structural and functional abnormality of the putamen in children with developmental dyslexia. Neuropsychologia. 2019;130:26–37. 104. Wechsler, D. (1997). Wechsler adult intelligence scale-( WAIS-3) San Antonio. TX: Harcourt Assessment. 105. Willcutt EG, Pennington BF. Comorbidity of reading disability and attention-deficit/hyperactivity disorder: differences by gender and subtype. J Learn Disabil. 2000;33(2):179–91. 106. Worbe Y, Palminteri S, Savulich G, Daw N, Fernandez-Egea E, Robbins T, Voon V. Valence-dependent influence of serotonin depletion on model- based choice strategy. Mol Psychiatry. 2016;21(5):624–9. 107. Wunderlich K, Smittenaar P, Dolan RJ. Dopamine enhances model- based over model-free choice behavior. Neuron. 2012;75(3):418–24. 108. Yael W, Tami K, Tali B. The effects of orthographic transparency and familiarity on reading Hebrew words in adults with and without dys- lexia. Ann Dyslexia. 2015;65(2):84–102. 109. Zeithamova D, Maddox WT. Dual-task interference in perceptual cat- egory learning. Mem Cognit. 2006;34(2):387–98. 110. Ballan R, Durrant SJ, Stickgold R, Morgan A, Manoach DS, Gabay Y. A failure of sleep-dependent consolidation of visuoperceptual proce- dural learning in young adults with ADHD. Translational Psychiatry. 2022;12(1):499. 111. Mas-Herrero E, Sescousse G, Cools R, Marco-Pallares J. The contribution of striatal pseudo-reward prediction errors to value-based decision- making. NeuroImage. 2019;193:67–74. 112. Gabay Y, Roark CL, Holt LL. Impaired and Spared Auditory Cat- egory Learning in Developmental Dyslexia. Psychological Science. 2022:09567976231151581. Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in pub- lished maps and institutional affiliations. Re Read ady y to to submit y submit your our re researc search h ? Choose BMC and benefit fr ? Choose BMC and benefit from om: : fast, convenient online submission thorough peer review by experienced researchers in your field rapid publication on acceptance support for research data, including large and complex data types • gold Open Access which fosters wider collaboration and increased citations maximum visibility for your research: over 100M website views per year At BMC, research is always in progress. Learn more biomedcentral.com/submissions

Journal

Behavioral and Brain Functions – Springer Journals

Published: Mar 21, 2023

Keywords: Attention-deficit/hyperactivity disorder; Developmental dyslexia; Two-step task; Model-based vs. Model-free reinforcement learning

There are no references for this article.