Feed on
Posts
Comments

SFN 2008 Poster

Rivest, Bengio & Kalaska
Learning timing in reinforcement learning model of dopamine responses in appetitive fixed-delay classical conditioning.
The temporal difference (TD) learning algorithm is now a common choice when modeling dopaminergic neuron activity in learning. When time is an important component of the experiment however, most models use timing elements, such as delay lines, that are not very biologically plausible for intervals in the range of seconds. The interval timing literature provides several alternative models, but most require explicit start and stop signals, leaving the credit-assignment problem unsolved. In this paper, we present a general learning model that learns a time representation when needed based on long short-term memory (LSTM) networks. Using a naïve network learning its environment in conjunction with TD, we reproduce dopaminergic neuron activity in appetitive fixed-delay classical trace conditioning, including probe trials of unexpected delays. The proposed model learns a natural time representation and also solves the credit assignment problem in an adaptive biologically plausible framework. The response profiles of elements in the LSTM network suggest that time may be learned and represented in a wide-spread cortical network, without need of pre-set delay lines, pacemaker circuits or other explicit timing mechanisms.
SfN2008Poster.pdf

Trackback URI | Comments RSS

Leave a Reply