Dear all,
If you are interested in meeting with Dr. Enrique Mallada, or joining the lunch at Ikes, please feel free to claim a slot in this link<https://gmuedu-my.sharepoint.com/:x:/g/personal/jchangn_gmu_edu/EQVXR4LYMTlNnbqkcoc-jWUBW6rp_DpLk5q0fldhAeflbQ?e=Uk1Ucl>.
Best regards,
Ningshi
--------------------------------------------------------------
Ningshi (Zoe) Yao
Assistant Professor
Rm 3241, Engineering Building,
Electrical and Computer Engineering, affiliated with Bio-engineering
George Mason University, Fairfax, VA, 22030
Lab Websites: https://mason.gmu.edu/~nyao4/ and https://secure-web.cisco.com/1WoWNIkQ9WO4ZH1x1hYRgKl79VEfFaWGHUiit95giotBxJw8AlC0DA34jF5ZAyO8_FQS6ydUeKImObKs1GOjYRQQVlG7ciSfLjmk_qYdcZis8t27jrUeaMCP7j_-vS9KuSHyZgEHRSWzWAwtNsNWwpFzVhoRf0wYul8wwhO-GcRi4IAuttru1kN3CNSawUwctF0wahG2CbYkQ87jZe0ZEpBBt7prexgaMdbRkgS8el_S-LMlW5KhzkFtbdrbzJl_5xrQ6oEchAkrlhRHvLGKHTed9wEtbeiQD6OIGtW0xK0EX6BzuG1jRrIRAw18Nddl38xsPfwb0NaYuaXHGXV5dZZatAaxXPZtpTwJ3rp410eTvYEUShRwtx2lvCtoZsVxg1UqP6XuEqCnx5kyLZ-LCpTrjMRtmunonI_-Ua9rPZ9EdizB-VUnYQeGJQKi5etAM/https%3A%2F%2Fciaolab.org<https://secure-web.cisco.com/1ImK6q7yE0qBYZEx8Am3oSNmEsk-wVxNPdzqkbNtQ6EoTWi_Xfog4cP87-QtwRp2ddXbo3HNN3o8eK10upkj5hJ0H970PxcFV_wkWu9V_1J3xYRzDTNM1JIZEwT0jqP9ftb1rNgFl7e69qwd8-pDru1-yRs-03FaQJghGiihmHvl4oGA_Oq9890jHqsx4JafbVHvSoaSY2RybH4PSzCTPjLcvKOjL_Yw5CsgAGS91p5iJCY-ai9h_2V1Dn5Il-fVwfl0JXIfzB3j01vfFwSF_93yRApz-9pRZ09Sqpqc49FgMmdR276P9Omv1457ZIEsqDlIz4EtMGK_Rd72wlBk9_L2MgwgeK8JfkoMQKvF210-X9MosY54RrffDlzvPYxLNzeEoY6YVH3PfvezhlybnRjiC52iZ4FuJJYZDU9UWIItc_QB8_i0uEXmlPSAPS8GU/https%3A%2F%2Fciaolab.org%2F>
Personal Website: https://secure-web.cisco.com/1t7hPYVueV-mwBQmtWgpI9P-_bo5HP-2Lz9jHTUPfxHl7dF19yYLvM7pCbhZJ77tnzJangLTCdroJM3DeikiyK_wS6Pv6d23VyZ1Zb63b1libiFYMxyT9BntsVxIEapDPYG5pV0jj6Vx8j23yzsOdgNTmJxcLLT_py4iS9roZqcXJ2vi41EciIdX6yWKCvvYvPvU8JRsUfeBnPjGKhcccRi-r0yKhlkgdIFdPUY1EY4PYHMJYjuaA-X238pg8q5_1QuwNdTdZqa0FPXwWf7gvWTEIOJDzAEIYJuD3qqtAmrnwOX6h6Bu1ylzf7NWIB-LZBIsdxbNZGJ0mtMS4A-JPy-Nu4Yws2wYw5bh67w7wsvaeYuHyzTaphdbs2V92xT6g4TsGe6oBR4kd255Osdw8guGkYMpMklbygcmdLYETMfMNwvuoGsT2GuoEYiXR-7MM/https%3A%2F%2Fningshiyao.com%2F
From: Jammie Chang <[log in to unmask]>
Date: Monday, February 5, 2024 at 12:44 PM
To: [log in to unmask] <[log in to unmask]>
Subject: Spring 2024 ECE Distinguished Seminar: Reinforcement Learning for Safety-Critical Systems, Feb 16, 11:00 am at ENGR 4201
Spring 2024 ECE Distinguished Seminar Series
Reinforcement Learning for Safety-Critical Systems
Dr. Enrique Mallada
Associate Professor, Department of Electrical and Computer Engineering
Johns Hopkins University
February 16, 2024, 11:00 am
ENGR 4201
Abstract
Integrating Reinforcement Learning (RL) in safety-critical applications, such as autonomous vehicles, healthcare, and industrial automation, necessitates an increased focus on safety and reliability. In this talk, we consider two complementary mechanisms to augment RL's suitability for safety-critical systems.
Firstly, we consider a constrained reinforcement learning (C-RL) setting, wherein agents aim to maximize rewards while adhering to required constraints on secondary specifications. Several algorithms rooted in sampled-based primal-dual methods have been recently proposed to solve this problem in policy space. However, such methods exhibit a discrepancy between the behavioral and optimal policies due to their reliance on stochastic gradient descent-ascent algorithms. We propose a novel algorithm for constrained RL that does not suffer from these limitations. Leveraging recent results on regularized saddle-flow dynamics, we develop a novel stochastic gradient descent-ascent algorithm whose trajectories almost surely converge to the optimal policy.
Secondly, we study the problem of incorporating safety-critical constraints to RL that allow an agent to avoid (unsafe) regions of the state space. Though such a safety goal can be captured by an action-value-like function, a.k.a. safety critics, the associated operator lacks the desired contraction and uniqueness properties that the classical Bellman operator enjoys. In this work, we overcome the non-contractiveness of safety critic operators by leveraging that safety is a binary property. To that end, we study the properties of the binary safety critic associated with a deterministic dynamical system that seeks to avoid reaching an unsafe region. We formulate the corresponding binary Bellman equation (B2E) for safety and study its properties. While the resulting operator is still non-contractive, we fully characterize its fixed points representing--except for a spurious solution--maximal persistently safe regions of the state space that can always avoid failure. We provide an algorithm that, by design, leverages axiomatic knowledge of safe data to avoid spurious fixed points.
Bio: Enrique Mallada has been an associate professor of electrical and computer engineering at Johns Hopkins University since 2022. Before joining Hopkins in 2016 as an assistant professor, he was a post-doctoral fellow at the Center for the Mathematics of Information at the California Institute of Technology from 2014 to 2016. He received his telecommunications engineering degree from Universidad ORT, Uruguay, in 2005 and his Ph.D. degree in electrical and computer engineering with a minor in applied mathematics from Cornell University in 2014. Dr. Mallada was awarded the Johns Hopkins Alumni Association Excellence in Teaching award in 2021, the NSF CAREER award in 2018, the ECE Director's Ph.D. Thesis Research Award for his dissertation in 2014, the Cornell University's Jacobs Fellowship in 2011 and the Organization of American States scholarship from 2008 to 2010. His research interests lie in the areas of control, dynamical systems, optimization, and machine learning, with applications to infrastructure networks and autonomous systems.
|
