Please use this identifier to cite or link to this item:
Appears in Collections:Psychology Journal Articles
Peer Review Status: Refereed
Title: Path-finding in real and simulated rats: assessing the influence of path characteristics on navigation learning
Authors: Tamosiunaite, Minija
Ainge, James A
Kulvicius, Tomas
Porr, Bernd
Dudchenko, Paul
Worgotter, Florentin
Contact Email:
Keywords: Reinforcement learning
Place field system
Function approximation
Weight decay
Issue Date: Dec-2008
Publisher: Springer
Citation: Tamosiunaite M, Ainge JA, Kulvicius T, Porr B, Dudchenko P & Worgotter F (2008) Path-finding in real and simulated rats: assessing the influence of path characteristics on navigation learning, Journal of Computational Neuroscience, 25 (3), pp. 562-582.
Abstract: A large body of experimental evidence suggests that the hippocampal place field system is involved in reward based navigation learning in rodents. Reinforcement learning (RL) mechanisms have been used to model this, associating the state space in an RL-algorithm to the place-field map in a rat. The convergence properties of RL-algorithms are affected by the exploration patterns of the learner. Therefore, we first analyzed the path characteristics of freely exploring rats in a test arena. We found that straight path segments with mean length 23 cm up to a maximal length of 80 cm take up a significant proportion of the total paths. Thus, rat paths are biased as compared to random exploration. Next we designed a RL system that reproduces these specific path characteristics. Our model arena is covered by overlapping, probabilistically firing place fields (PF) of realistic size and coverage. Because convergence of RL-algorithms is also influenced by the state space characteristics, different PF-sizes and densities, leading to a different degree of overlap, were also investigated. The model rat learns finding a reward opposite to its starting point. We observed that the combination of biased straight exploration, overlapping coverage and probabilistic firing will strongly impair the convergence of learning. When the degree of randomness in the exploration is increased, convergence improves, but the distribution of straight path segments becomes unrealistic and paths become ‘wiggly’. To mend this situation without affecting the path characteristic two additional mechanisms are implemented: A gradual drop of the learned weights (weight decay) and path length limitation, which prevents learning if the reward is not found after some expected time. Both mechanisms limit the memory of the system and thereby counteract effects of getting trapped on a wrong path. When using these strategies individually divergent cases get substantially reduced and for some parameter settings no divergence was found anymore at all. Using weight decay and path length limitation at the same time, convergence is not much improved but instead time to convergence increases as the memory limiting effect is getting too strong. The degree of improvement relies also on the size and degree of overlap (coverage density) in the place field system. The used combination of these two parameters leads to a trade-off between convergence and speed to convergence. Thus, this study suggests that the role of the PF-system in navigation learning cannot be considered independently from the animals’ exploration pattern.
Type: Journal Article
DOI Link:
Rights: The publisher does not allow this work to be made publicly available in this Repository. Please use the Request a Copy feature at the foot of the Repository record to request a copy directly from the author; you can only request a copy if you wish to use this work for your own research or private study.
Notes: 10.1007/s10827-008-0094-6
Affiliation: University of Stirling
University of Stirling
Vytautas Magnus University
University of Glasgow
University of Stirling

Files in This Item:
File Description SizeFormat 
dudchenko_2008.pdf873.19 kBAdobe PDFUnder Embargo until 31/12/2999     Request a copy

Note: If any of the files in this item are currently embargoed, you can request a copy directly from the author by clicking the padlock icon above. However, this facility is dependant on the depositor still being contactable at their original email address.

This item is protected by original copyright

Items in the Repository are protected by copyright, with all rights reserved, unless otherwise indicated.

If you believe that any material held in STORRE infringes copyright, please contact providing details and we will remove the Work from public display in STORRE and investigate your claim.