SFN 2008 Poster
December 7th, 2009 by Francois Rivest
Rivest, Bengio & Kalaska
Learning timing in reinforcement learning model of dopamine responses in appetitive fixed-delay classical conditioning.
The temporal difference (TD) learning algorithm is now a common choice when modeling dopaminergic neuron activity in learning. When time is an important component of the experiment however, most models use timing elements, such as delay lines, that are not very biologically plausible for intervals in the range of seconds. The interval timing literature provides several alternative models, but most require explicit start and stop signals, leaving the credit-assignment problem unsolved. In this paper, we present a general learning model that learns a time representation when needed based on long short-term memory (LSTM) networks. Using a naïve network learning its environment in conjunction with TD, we reproduce dopaminergic neuron activity in appetitive fixed-delay classical trace conditioning, including probe trials of unexpected delays. The proposed model learns a natural time representation and also solves the credit assignment problem in an adaptive biologically plausible framework. The response profiles of elements in the LSTM network suggest that time may be learned and represented in a wide-spread cortical network, without need of pre-set delay lines, pacemaker circuits or other explicit timing mechanisms.
SfN2008Poster.pdf
Computational Neuroscience (and Programming) Blog