Publication Details

Fuel in Markov Decision Processes (FiMDP): A Practical Approach to Consumption

NOVOTNÝ Petr, BLAHOUDEK František, CUBUKTEPE Murat, ORNIK Melkior, THANGEDA Pranay and TOPCU Ufuk. Fuel in Markov Decision Processes (FiMDP): A Practical Approach to Consumption. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Pittsburgh, 2021, pp. 640-656. ISBN 978-3-030-90869-0. Available from: https://link.springer.com/chapter/10.1007/978-3-030-90870-6_34?getft_integrator=scopus
Czech title
Palivo v rozhodovacích procesech Markov (FiMDP): Praktický přístup ke spotřebě
Type
conference paper
Language
english
Authors
Novotný Petr, doc. RNDr., Ph.D. (FI MUNI)
Blahoudek František, RNDr., Ph.D. (DITS FIT BUT)
Cubuktepe Murat (UTAUSTIN)
Ornik Melkior (UILLINOIS)
Thangeda Pranay (UILLINOIS)
Topcu Ufuk (UTAUSTIN)
URL
Keywords

Behavioral research, Decision making, Model checking, Multi agent systems, Polynomial approximation

Abstract

Consumption Markov Decision Processes (CMDPs) are probabilistic decision-making models of resource-constrained systems. We introduce FiMDP, a tool for controller synthesis in CMDPs with LTL objectives expressible by deterministic Büchi automata. The tool implements the recent algorithm for polynomial-time controller synthesis in CMDPs, but extends it with many additional features. On the conceptual level, the tool implements heuristics for improving the expected reachability times of accepting states, and a support for multi-agent task allocation. On the practical level, the tool offers (among other features) a new strategy simulation framework, integration with the Storm model checker, and FiMDPEnv - a new set of CMDPs that model real-world resource-constrained systems. We also present an evaluation of FiMDP on these real-world scenarios.

Published
2021
Pages
640-656
Proceedings
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Conference
24rd International Symposium on Formal Methods, Porto, PT
ISBN
978-3-030-90869-0
Place
Pittsburgh, US
DOI
10.1007/978-3-030-90870-6_34
UT WoS
000758218600034
EID Scopus
2-s2.0-85119837676
BibTeX 
@INPROCEEDINGS{FITPUB13361,
   author = "Petr Novotn\'{y} and Franti\v{s}ek Blahoudek and Murat Cubuktepe and Melkior Ornik and Pranay Thangeda and Ufuk Topcu",
   title = "Fuel in Markov Decision Processes (FiMDP): A Practical Approach to Consumption",
   pages = "640--656",
   booktitle = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
   year = 2021,
   location = "Pittsburgh, US",
   ISBN = "978-3-030-90869-0",
   doi = "10.1007/978-3-030-90870-6\_34",
   language = "english",
   url = "https://www.fit.vut.cz/research/publication/13361"
}
Back to top