2014-CNS*2014 Workshop on Basal Ganglia: Structure, dynamics and function (31 July, 2014, Quebec, Canada)

 Organizers

• Dr. Arvind Kumar (Faculty of Biology, Bernstein Center Freiburg, University of Freiburg, Germany.)
• Dr. Jeanette Hellgren Kotaleski (Royal Institute of Technology, Stockholm, Sweden)
  

Topic description

The basal ganglia (BG) are involved in a wide range of motor and cognitive processes, and accordingly, their dysfunction can lead to several neurological diseases. To understand the computational role of BG in these various functions and dysfunction several bottom-up and top-down models have been proposed. Bottom-up computational approaches have addressed the dynamical properties and interaction of the neural activity in the BG nuclei, while top-down approaches rather have described BG function inspired by machine learning algorithms.

Recent advances in experimental methods have allowed for the characterization of BG activity and cortico-basal ganglia interactions with great detail both in normal and pathological conditions and have challenged the classical feedforward view of the basal ganglia network.

In this workshop, we bring together both experimentalists and theoreticians to review the recent advances in understanding of BG function. Specifically, we will discuss how computational models of BG have advanced to integrate the new data on BG network structure and neuronal activity and thus, to understand how the relationship between BG dynamics and function/dysfunction of BG. Finally, we will discuss how top-down functional models could be linked to the bottom-up dynamical models and provide new predictions and explanations of the experimental data in terms of neuronal and network properties.
 
 

Schedule

0900 – 0945     Joshua Berke    
Is it Worth It? Dopamine Dynamics and the Value of Work.

0945 – 1030    Aryn Gittis             
Rapid target-specific remodeling of fast-spiking inhibitory circuits after loss of dopamine.

Coffee break

1100 – 1145    Mikael Lindhal            
The role of striatal inhibition in action selection

1145 - 1230    Julian Vitay    
Functions and plasticity of the Basal Ganglia pathways in functional and dysfunctional networks  

Lunch

1400 – 1445    Jesse Goldberg
Lessons from the songbird: A role for efference copy in basal ganglia dependent reinforcement learning

1445 – 1530    Yashar  Zeighami
Impulse control disorders in Parkinson's disease are associated with dysfunction in stimulus but not in action valuation

Coffee break

1600 – 1645    V. Srinivasa Chakravarti
The Basal Ganglia system as an Engine for Exploration

1645 – 1730    Michael Frank    
Frontal control over striatal learning and choice
 

Contact

•   Dr. Arvind Kumar [arvind.kumar@biologie.uni-freiburg.de]
    Faculty of Biology, Bernstein Center Freiburg, University of Freiburg, Germany.
•   Dr. Jeanette Hellgren Kotaleski [jeanette@nada.kth.se]
    Royal Institute of Technology, Stockholm, Sweden

 

 
Joshua Berke            Michigan State University, Ann Arbor, MI, USA

Is it Worth It? Dopamine Dynamics and the Value of Work.

Dopamine cell firing is thought to be a learning signal, encoding reward prediction errors to help guide future adaptive behavior. Yet dopamine also modulates current behavior, helping to invigorate action. Some current theories suggest that subsecond (phasic) dopamine fluctuations support learning, while much slower (tonic) dopamine changes affect motivation. I will present both empirical data and computational modeling to support an alternative, unified view of dopamine function across multiple timescales. We have found that dopamine signals the instantaneous reward rate available for investment of effort, combining the time-discounted sum of experienced past rewards and the time-discounted sum of anticipated future rewards. This reward rate strongly influences ongoing decisions to work, i.e. to engage in instrumental task performance rather than alternative unconditioned activities. Abrupt changes in this dopamine reward rate signal convey reward prediction errors and reinforce preceding behavior. Our results reconcile many prior findings within a single framework, accounting for how mesolimbic dopamine contributes to both learning and motivation



Michael Frank            Brown University, Providence, RI, USA

Frontal control over striatal learning and choice
 

Aryn Gittis                 Carnegie Mellon University, Pittsburg, PA, USA

Rapid target-specific remodeling of fast-spiking inhibitory circuits after loss of dopamine.

In Parkinson's disease (PD), dopamine depletion alters neuronal activity in the direct and indirect pathways and leads to increased synchrony in the basal ganglia network. However, the origins of these changes remain elusive. Because GABAergic interneurons regulate activity of projection neurons and promote neuronal synchrony, we recorded from pairs of striatal fast-spiking (FS) interneurons and direct- or indirect-pathway MSNs after dopamine depletion with 6-OHDA. Synaptic properties of FS-MSN connections remained similar, yet within 3 days of dopamine depletion, individual FS cells doubled their connectivity to indirect-pathway MSNs, whereas connections to direct-pathway MSNs remained unchanged. A model of the striatal microcircuit revealed that such increases in FS innervation were effective at enhancing synchrony within targeted cell populations. These data suggest that after dopamine depletion, rapid target-specific microcircuit organization in the striatum may lead to increased synchrony of indirect-pathway MSNs that contributes to pathological network oscillations and motor symptoms of PD.

Jesse Goldberg            Cornell University, NY, USA

Lessons from the songbird: A role for efference copy in basal ganglia dependent reinforcement learning

The central tenets of reinforcement learning were articulated over a century ago by Edward Thorndike in his Law of Effect: “Responses that produce a satisfying effect in a particular situation become more likely to occur again in that situation.” Implementing this principle requires three pieces of information: (1) the exploratory action (response) that the animal makes, (2) the context (situation) in which an action takes place, (3) and an evaluation of the outcome (effect) of the action. Here, I integrate my recent findings with others from the birdsong field to propose a mechanistic model for how the basal ganglia implement RL. I suggest that the songbird BG receives three main inputs which carry the exact three pieces of information predicted by Thorndike: (1) a topographic efference copy of exploratory action, (2) a non-topographic context signal conveying what ‘time it is’ in the song sequence, and (3) a global evaluation signal. I propose a novel three-factor learning rule where the efference copy and evaluation signals ‘gate’ synaptic plasticity of ‘context’ inputs to the BG. Implementing this rule enables the BG to generate a premotor bias signal that, in essence, instructs ‘what’ the bird should do ‘when’ in its song sequence. Finally, I will generalize this rule to motor sequence learning in mammals and lay out the testable predictions that we are currently testing in my lab.


Mikael Lindhal            Royal Institute of Technology, Stockholm, Sweden

The role of striatal inhibition in action selection

Synaptic inhibition is crucial for controlling and shaping signaling in the basal ganglia circuits. The medium spiny neurons (MSN) are the projection neurons of the striatum, the input nucleus of the basal ganglia, and they receive inhibitory inputs from mainly three different sources: collateral input from neighboring MSNs, feedforward inhibition from fast spiking interneurons and oligosynaptic inhibition via part of the globus pallidus externa (GPe) (Mallet et al., 2012; Tepper et al., 2008), the latter in turn controlled from the subthalamic nucleus (STN). I will present a semi-quantitative spiking neural network model of the striatum, STN, GPe and SNr, with synapse dynamics and connectivity set based on available experimental data. Cortical inputs are emulated to both striatum and STN. The model reproduces network dynamics measured during in vivo experiments and seen in response to both slow wave and active brains states in control and dopamine depleted animals (Mallet et all, 2008). I will illustrate how feedback, feedforward and oligosynaptic mediated inhibition differentially could control the excitability of MSNs in striatum. Furthermore, I will also use the model framework to investigate to what extent the inhibitory connectivity within stratum can enhance the basal ganglia hypothesized role in action selection.  

References:
Mallet, N., Micklem, B. R., Henny, P., Brown, M. T., Williams, C., Bolam, J. P., Nakamura, K. C., and Magill, P. J. (2012). Dichotomous Organization of the External Globus Pallidus. Neuron 74, 1075–1086.
Taverna, S., Ilijic, E., and Surmeier, D. J. (2008). Recurrent collateral connections of striatal medium spiny neurons are disrupted in models of Parkinson’s disease. J. Neurosci. 28, 5504–12.
Mallet, N., Pogosyan, A., Márton, L. F., Bolam, J. P., Brown, P., and Magill, P. J. (2008). Parkinsonian beta oscillations in the external globus pallidus and their relationship with subthalamic nucleus activity. J. Neurosci. 28, 14245–58.


Julian Vitay                Chemnitz University of Technology, Chemnitz, Germany

Functions and plasticity of the Basal Ganglia pathways in functional and dysfunctional networks  

Although the Basal Ganglia (BG) have been extensively studied from the computational point of view, the exact role of the different pathways within the BG is still debated, especially with respect to their plasticity. The functional contribution of the three major feedforward pathways (direct, indirect and hyperdirect) during different cognitive/motor tasks in healthy or pathological patients has become clearer, but several issues are still open, among others: how does synaptic plasticity within the BG influence the acquisition of motor and cognitive behavior; which role do feedback loops within the BG play in the overall function? This talk will present the different modeling works done in the lab of Pr. Fred Hamker on these subjects. A recent rate-coded model (Schroll, Vitay and Hamker, 2014) investigated pathway unbalance resulting from dopamine depletion in Parkinson's disease (PD), not only through neural firing but also on the internal connectivity within the BG. It sheds a new light on the mechanisms leading to motor and cognitive impairments in PD and has been extended to explain compensatory mechanisms in Tourette's syndrome. In order to study in more details some neural correlates of PD, such as oscillatory activities in the GPe-STN loop, a spiking-neuron model of the whole BG structure has been developed (Baladron and Hamker, submitted), introducing spike timing-dependent plasticity in the three pathways and studying the role of the STN-GPe in habit formation.
 

V. Srinivasa Chakravarthy        Indian Institute of Technology, Madras, Chennai, India

The Basal Ganglia system as an Engine for Exploration

The Basal Ganglia (BG) system has wide-ranging brain functions including action selection, reinforcement learning, motor preparation, sequence generation, goal-directed behavior and working memory. There is a growing consensus about explaining BG functions using concepts from Reinforcement Learning (RL). Exploitation and exploration are the yin and yang of RL dynamics. Although cortical substrates for exploration have been found, no subcortical counterparts of the same have been identified. The talk presents and develops the consequences of a hypothesis that the Indirect Pathway – a part of BG – is the subcortical substrate for exploration. This hypothesis seems to supply a missing piece in the application of RL concepts to BG anatomy. It presents a picture of the functional anatomy of BG wherein the Direct Pathway subserves Exploitation, while the Indirect Pathway subserves Exploration. The proposed theory of BG can explain a range of BG functions including action selection, reaching, saccade generation, spatial navigation, gait, precision grip and finally willed action.

Yashar Zeighami        University of Western Sydney, Australia

Impulse control disorders in Parkinson's disease are associated with dysfunction in stimulus but not in action valuation 

A substantial subset of Parkinson’s disease (PD) patients suffers from impulse control disorders (ICDs), which are side effects of dopaminergic medication. Dopamine plays a key role in reinforcement learning processes. One class of reinforcement learning models, known as the actor-critic model, suggests that two components are involved in these reinforcement learning processes: a critic, which estimates values of stimuli and calculates prediction errors, and an actor, which estimates values of potential actions. To understand the information processing mechanism underlying impulsive behavior, we investigated stimulus and action value learning from reward and punishment in four groups of participants: on-medication PD patients with ICD, on-medication PD patients without ICD, off-medication PD patients without ICD, and healthy controls. Analysis of responses suggested that participants used an actor-critic learning strategy and computed prediction errors based on stimulus values rather than action values. Quantitative model fits also revealed that an actor-critic model of the basal ganglia with different learning rates for positive and negative prediction errors best matched the choice data. Moreover, whereas ICDs were associated with model parameters related to stimulus valuation (critic), PD was associated with parameters related to action valuation (actor). Specifically, PD patients with ICD exhibited lower learning from negative prediction errors in the critic, resulting in an underestimation of adverse consequences associated with stimuli. These findings offer a specific neurocomputational account of the nature of compulsive behaviors induced by dopaminergic drugs.

Key words: computational modeling; dopamine; impulse control disorders; Parkinson’s disease; prediction error; reinforcement learning

Conference Website

www.cnsorg.org/cns-2014