Markovian control problems. Functional equations and algorithms. Reprint.

*(English)*Zbl 0541.90068
Mathematical Centre Tracts, 97. Amsterdam: Mathematisch Centrum. VII, 212 p. Dfl. 31.00 (1983).

This is a reprint of the author’s doctoral dissertation which was published in 1978. It consist of two parts: I. Markov decision theory and II. Stochastic games.

Part I is divided into five chapters: 1. Value iteration in finite Markov decision problems; 2. Contraction mappings underlying undiscounted Markov decision problems; 3. Nonstationary Markov decision problems with converging parameters; 4. Successive approximation methods for solving nested functional equations in Markov decision theory; 5. The optimality equation in average cost denumerable state semi-Markov decision problems, recurrence conditions and algorithms.

Part II is divided into three chapters: 6. On N-person stochastic games with denumerable state space; 7. On the functional equations in undiscounted and sensitive discounted stochastic games; 8. Successive approximation methods in two-person zero-sum stochastic games.

The list of references is up-to-dated apart from some papers (the most by the author or co-authored by himself). The treatment is self-contained and presents many important contributions due to the author. The book is very clearly written and makes genuine pleasant reading.

Part I is divided into five chapters: 1. Value iteration in finite Markov decision problems; 2. Contraction mappings underlying undiscounted Markov decision problems; 3. Nonstationary Markov decision problems with converging parameters; 4. Successive approximation methods for solving nested functional equations in Markov decision theory; 5. The optimality equation in average cost denumerable state semi-Markov decision problems, recurrence conditions and algorithms.

Part II is divided into three chapters: 6. On N-person stochastic games with denumerable state space; 7. On the functional equations in undiscounted and sensitive discounted stochastic games; 8. Successive approximation methods in two-person zero-sum stochastic games.

The list of references is up-to-dated apart from some papers (the most by the author or co-authored by himself). The treatment is self-contained and presents many important contributions due to the author. The book is very clearly written and makes genuine pleasant reading.

Reviewer: M.Iosifescu

##### MSC:

90C40 | Markov and semi-Markov decision processes |

91A60 | Probabilistic games; gambling |

91A15 | Stochastic games, stochastic differential games |

91A05 | 2-person games |

91A10 | Noncooperative games |

90-02 | Research exposition (monographs, survey articles) pertaining to operations research and mathematical programming |