#5766. Efficient algorithms for Risk-Sensitive Markov Decision Processes with limited budget

July 2026publication date
Proposal available till 12-05-2025
4 total number of authors per manuscript0 $

The title of the journal is available only for the authors who have already paid for
Journal’s subject area:
Applied Mathematics;
Theoretical Computer Science;
Software;
Artificial Intelligence;
Places in the authors’ list:
place 1place 2place 3place 4
FreeFreeFreeFree
2350 $1200 $1050 $900 $
Contract №5766.1 Contract №5766.2 Contract №5766.3 Contract №5766.4
1 place - free (for sale)
2 place - free (for sale)
3 place - free (for sale)
4 place - free (for sale)

More details about the manuscript: Science Citation Index Expanded or/and Social Sciences Citation Index
Abstract:
We tackle the problem of finding optimal policies for Markov Decision Processes, that minimize the probability of the cumulative cost exceeding a given budget. Such task falls under the umbrella of Risk-Sensitive Markov Decision Processes, which optimize a non-additive, non-linear function of cumulative cost that incorporates the users attitude towards risk. Current algorithms for solving that task, for any budget equal or smaller than an user-defined budget, scale poorly when the support of the cost function is large, since they operate in an augmented state space which enumerates all possible remaining budgets. To circumvent this issue, we develop (i) an improved version of the Topological Value Iteration with Dynamic Programming algorithm (TVI-DP), and (ii) the first symbolic dynamic programming algorithm for this class of problems, called RS-SPUDD, that exploits conditional independence in the transition function in the augmented state space. The proposed algorithms improve efficiency by pruning irrelevant states and terminating early, without sacrificing optimality.
Keywords:
Markov decision processes; Probabilistic planning; Risk-Sensitive Markov decision processes

Contacts :
0