Cerebral Cortex Advance Access published online on June 4, 2007
Cerebral Cortex, doi:10.1093/cercor/bhm064
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Dynamic Signals Related to Choices and Outcomes in the Dorsolateral Prefrontal Cortex
1 Department of Neurobiology, Yale University School of Medicine, New Haven, CT 06510, USA, 2 Department of Neurobiology and Anatomy, Center for Visual Science, University of Rochester, Rochester, NY 14627, USA
Address correspondence to Department of Neurobiology, Yale University School of Medicine, 333 Cedar Street, SHM C303, New Haven, CT 06510, USA. Email: daeyeol.lee{at}yale.edu.
Although economic theories based on utility maximization account for a range of choice behaviors, utilities must be estimated through experience. Dynamics of this learning process may account for certain discrepancies between the predictions of economic theories and real choice behaviors of humans and other animals. To understand the neural mechanisms responsible for such adaptive decision making, we trained rhesus monkeys to play a simulated matching pennies game. Small but systematic deviations of the animal's behavior from the optimal strategy were consistent with the predictions of reinforcement learning theory. In addition, individual neurons in the dorsolateral prefrontal cortex (DLPFC) encoded 3 different types of signals that can potentially influence the animal's future choices. First, activity modulated by the animal's previous choices might provide the eligibility trace that can be used to attribute a particular outcome to its causative action. Second, activity related to the animal's rewards in the previous trials might be used to compute an average reward rate. Finally, activity of some neurons was modulated by the computer's choices in the previous trials and may reflect the process of updating the value functions. These results suggest that the DLPFC might be an important node in the cortical network of decision making.
Key Words: eligibility trace • neuroeconomics • reinforcement learning • reward • state-space model
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  J. L. Pardo-Vazquez, V. Leboran, and C. Acuna A role for the ventral premotor cortex beyond performance monitoring PNAS, November 3, 2009; 106(44): 18815 - 18819. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
K. Wunderlich, A. Rangel, and J. P. O'Doherty Neural computations underlying action-based decision making in the human brain PNAS, October 6, 2009; 106(40): 17199 - 17204. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 C.-H. Luk and J. D. Wallis Dynamic Encoding of Responses and Outcomes by Neurons in Medial Prefrontal Cortex J. Neurosci., June 10, 2009; 29(23): 7526 - 7539. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
H. Seo, D. J. Barraclough, and D. Lee Lateral Intraparietal Cortex and Reinforcement Learning during a Mixed-Strategy Game J. Neurosci., June 3, 2009; 29(22): 7278 - 7289. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
H. Seo and D. Lee Behavioral and Neural Changes after Gains and Losses of Conditioned Reinforcers J. Neurosci., March 18, 2009; 29(11): 3627 - 3641. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. Tsujimoto, A. Genovesio, and S. P. Wise Monkey Orbitofrontal Cortex Encodes Response Choices Near Feedback Time J. Neurosci., February 25, 2009; 29(8): 2569 - 2574. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 H. Seo and D. Lee Cortical mechanisms for reinforcement learning in competitive games Phil Trans R Soc B, December 12, 2008; 363(1511): 3845 - 3857. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J.-W. Sohn and D. Lee Order-Dependent Modulation of Directional Signals in the Supplementary and Presupplementary Motor Areas J. Neurosci., December 12, 2007; 27(50): 13655 - 13666. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 Y. B. Kim, N. Huh, H. Lee, E. H. Baeg, D. Lee, and M. W. Jung Encoding of Action History in the Rat Ventral Striatum J Neurophysiol, December 1, 2007; 98(6): 3548 - 3556. [Abstract] [Full Text] [PDF]
|
|