Processus de décision markovien

Anglais

Markov decision process

Markov decision processes (MDPs) provide a mathematical framework for modeling decision making in situations where outcomes are partly random and partly under the control of a decision maker. MDPs are useful for studying a wide range of optimization problems solved via dynamic programming and reinforcement learning. MDPs were known at least as early as the 1950s (cf. Bellman 1957); a core body of research on Markov decision processes resulted from Ronald A. Howard's book published in 1960, Dynamic Programming and Markov Processes.[1] They are used in a wide area of disciplines, including robotics, automatic control, economics, and manufacturing.

More precisely, a Markov decision process is a discrete time stochastic control process. At each time step, the process is in some state s {\displaystyle s} s, and the decision maker may choose any action a {\displaystyle a} a that is available in state s {\displaystyle s} s. The process responds at the next time step by randomly moving into a new state s ′ {\displaystyle s'} s', and giving the decision maker a corresponding reward R a ( s , s ′ ) {\displaystyle R_{a}(s,s')} R_a(s,s').