Reward Machines for Deep RL in Noisy and Uncertain Environments

Simple item page
dc.catalogadorgjm
dc.contributor.authorLi, Andrew C.
dc.contributor.authorChen, Zizhao
dc.contributor.authorKlassen, Toryn Q.
dc.contributor.authorVaezipoor, Pashootan
dc.contributor.authorToro Icarte, Rodrigo Andrés
dc.contributor.authorMcIlraith, Sheila A.
dc.date.accessioned2025-06-12T16:08:18Z
dc.date.available2025-06-12T16:08:18Z
dc.date.issued2024
dc.description.abstractReward Machines provide an automaton-inspired structure for specifying instructions, safety constraints, and other temporally extended reward-worthy behaviour. By exposing the underlying structure of a reward function, they enable the decomposition of an RL task, leading to impressive gains in sample efficiency. Although Reward Machines and similar formal specifications have a rich history of application towards sequential decision-making problems, they critically rely on a ground-truth interpretation of the domain-specific vocabulary that forms the building blocks of the reward function—such ground-truth interpretations are elusive in the real world due in part to partial observability and noisy sensing. In this work, we explore the use of Reward Machines for Deep RL in noisy and uncertain environments. We characterize this problem as a POMDP and propose a suite of RL algorithms that exploit task structure under uncertain interpretation of the domain- specific vocabulary. Through theory and experiments, we expose pitfalls in naive approaches to this problem while simultaneously demonstrating how task structure can be successfully leveraged under noisy interpretations of the vocabulary.
dc.fechaingreso.objetodigital2025-06-12
dc.format.extent28 páginas
dc.fuente.origenWOS
dc.identifier.doi10.48550/arXiv.2406.00120
dc.identifier.urihttps://doi.org/10.48550/arXiv.2406.00120
dc.identifier.urihttps://repositorio.uc.cl/handle/11534/104656
dc.identifier.wosidPPRN:89164299
dc.information.autorucEscuela de Ingeniería; Toro Icarte, Rodrigo Andrés; 0000-0002-7734-099X; 170373
dc.language.isoen
dc.nota.accesocontenido completo
dc.revistaArxiv
dc.rightsacceso abierto
dc.rights.licenseCC BY 4.0 Attribution 4.0 International
dc.rights.urihttps://creativecommons.org/licenses/by/4.0/
dc.subjectMachine Learning
dc.subjectArtificial Intelligence
dc.subjectFormal Languages and Automata Theory
dc.subject.ddc600
dc.subject.deweyTecnologíaes_ES
dc.titleReward Machines for Deep RL in Noisy and Uncertain Environments
dc.typepreprint
sipa.codpersvinculados170373
sipa.trazabilidadWOS;2024-11-30
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
2406.00120v4.pdf
Size:
1.56 MB
Format:
Adobe Portable Document Format
Description:
Download
Collections
Artículos de revistas