Please use this identifier to cite or link to this item: http://hdl.handle.net/1893/2807
Appears in Collections:Psychology Journal Articles
Peer Review Status: Refereed
Title: Path-finding in real and simulated rats: assessing the influence of path characteristics on navigation learning
Author(s): Tamosiunaite, Minija
Ainge, James A
Kulvicius, Tomas
Porr, Bernd
Dudchenko, Paul
Worgotter, Florentin
Contact Email: p.a.dudchenko@stir.ac.uk
Keywords: Reinforcement learning
SARSA
Place field system
Function approximation
Weight decay
Animal navigation
Hippocampus (Brain)
Issue Date: Dec-2008
Date Deposited: 16-Mar-2011
Citation: Tamosiunaite M, Ainge JA, Kulvicius T, Porr B, Dudchenko P & Worgotter F (2008) Path-finding in real and simulated rats: assessing the influence of path characteristics on navigation learning. Journal of Computational Neuroscience, 25 (3), pp. 562-582. https://doi.org/10.1007/s10827-008-0094-6
Abstract: A large body of experimental evidence suggests that the hippocampal place field system is involved in reward based navigation learning in rodents. Reinforcement learning (RL) mechanisms have been used to model this, associating the state space in an RL-algorithm to the place-field map in a rat. The convergence properties of RL-algorithms are affected by the exploration patterns of the learner. Therefore, we first analyzed the path characteristics of freely exploring rats in a test arena. We found that straight path segments with mean length 23 cm up to a maximal length of 80 cm take up a significant proportion of the total paths. Thus, rat paths are biased as compared to random exploration. Next we designed a RL system that reproduces these specific path characteristics. Our model arena is covered by overlapping, probabilistically firing place fields (PF) of realistic size and coverage. Because convergence of RL-algorithms is also influenced by the state space characteristics, different PF-sizes and densities, leading to a different degree of overlap, were also investigated. The model rat learns finding a reward opposite to its starting point. We observed that the combination of biased straight exploration, overlapping coverage and probabilistic firing will strongly impair the convergence of learning. When the degree of randomness in the exploration is increased, convergence improves, but the distribution of straight path segments becomes unrealistic and paths become ‘wiggly’. To mend this situation without affecting the path characteristic two additional mechanisms are implemented: A gradual drop of the learned weights (weight decay) and path length limitation, which prevents learning if the reward is not found after some expected time. Both mechanisms limit the memory of the system and thereby counteract effects of getting trapped on a wrong path. When using these strategies individually divergent cases get substantially reduced and for some parameter settings no divergence was found anymore at all. Using weight decay and path length limitation at the same time, convergence is not much improved but instead time to convergence increases as the memory limiting effect is getting too strong. The degree of improvement relies also on the size and degree of overlap (coverage density) in the place field system. The used combination of these two parameters leads to a trade-off between convergence and speed to convergence. Thus, this study suggests that the role of the PF-system in navigation learning cannot be considered independently from the animals’ exploration pattern.
DOI Link: 10.1007/s10827-008-0094-6
Rights: The publisher does not allow this work to be made publicly available in this Repository. Please use the Request a Copy feature at the foot of the Repository record to request a copy directly from the author; you can only request a copy if you wish to use this work for your own research or private study.
Licence URL(s): http://www.rioxx.net/licenses/under-embargo-all-rights-reserved

Files in This Item:
File Description SizeFormat 
dudchenko_2008.pdfFulltext - Published Version873.19 kBAdobe PDFUnder Embargo until 3000-01-01    Request a copy

Note: If any of the files in this item are currently embargoed, you can request a copy directly from the author by clicking the padlock icon above. However, this facility is dependent on the depositor still being contactable at their original email address.



This item is protected by original copyright



Items in the Repository are protected by copyright, with all rights reserved, unless otherwise indicated.

The metadata of the records in the Repository are available under the CC0 public domain dedication: No Rights Reserved https://creativecommons.org/publicdomain/zero/1.0/

If you believe that any material held in STORRE infringes copyright, please contact library@stir.ac.uk providing details and we will remove the Work from public display in STORRE and investigate your claim.