| 1 |
On Reinforcement Learning, Effect Handlers, and the State Monad ...
|
|
|
|
Abstract:
We study the algebraic effects and handlers as a way to support decision-making abstractions in functional programs, whereas a user can ask a learning algorithm to resolve choices without implementing the underlying selection mechanism, and give a feedback by way of rewards. Differently from some recently proposed approach to the problem based on the selection monad [Abadi and Plotkin, LICS 2021], we express the underlying intelligence as a reinforcement learning algorithm implemented as a set of handlers for some of these algebraic operations, including those for choices and rewards. We show how we can in practice use algebraic operations and handlers -- as available in the programming language EFF -- to clearly separate the learning algorithm from its environment, thus allowing for a good level of modularity. We then show how the host language can be taken as a lambda-calculus with handlers, this way showing what the essential linguistic features are. We conclude by hinting at how type and effect systems ...
|
|
Keyword:
FOS Computer and information sciences; Logic in Computer Science cs.LO; Machine Learning cs.LG; Programming Languages cs.PL
|
|
URL: https://arxiv.org/abs/2203.15426
https://dx.doi.org/10.48550/arxiv.2203.15426
|
|
BASE
|
|
Hide details
|
|
|
|
|
