Copyright © 2006 The Institute of Electronics, Information and Communication Engineers
Regular Section -- Papers -- Nonlinear Problems |
Reinforcement Learning for Continuous Stochastic Actions—An Approximation of Probability Density Function by Orthogonal Wave Function Expansion—
1 The author is with the Future University-Hakodate, Hakodate-shi, 041-8655 Japan. E-mail: jamisato{at}m.ieice.org
A function approximation based on an orthonormal wave function expansion in a complex space is derived. Although a probability density function (PDF) cannot always be expanded in an orthogonal series in a real space because a PDF is a positive real function, the function approximation can approximate an arbitrary PDF with high accuracy. It is applied to an actor-critic method of reinforcement learning to derive an optimal policy expressed by an arbitrary PDF in a continuous-action continuous-state environment. A chaos control problem and a PDF approximation problem are solved using the actor-critic method with the function approximation, and it is shown that the function approximation can approximate a PDF well and that the actor-critic method with the function approximation exhibits high performance.
Key Words: actor-critic, continuous, probability density, orthogonal expansion, approximation
Manuscript received July 22, 2005. Manuscript revised March 6, 2006. Final manuscript received April 17, 2006.