PhD Proposal by Marcus A. Pereira

Title: Scalable and Safe Deep Learning Architectures and Algorithms for Stochastic Optimal Control using Forward-Backward Stochastic Differential Equations 

 

Date: Monday, November 1, 2021

Time: 10:00 am – 12:00 pm EST

Location (Virtual)Teams Meeting Link

Location (In person): Montgomery Knight Building, MK 317 (conference room)

 

Marcus A. Pereira 

Robotics Ph.D. Student 

Daniel Guggenheim School of Aerospace Engineering, Georgia Institute of Technology

Georgia Institute of Technology

 

Committee:

Dr. Evangelos A. Theodorou (Advisor) – Daniel Guggenheim School of Aerospace Engineering, Georgia Institute of Technology

Dr. Enlu Zhou – H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology

Dr. Samuel Coogan – School of Electrical and Computer Engineering, Georgia Institute of Technology

Dr. Yongxin Chen – Daniel Guggenheim School of Aerospace Engineering, Georgia Institute of Technology

Dr. Kyriakos G. Vamvoudakis – Daniel Guggenheim School of Aerospace Engineering, Georgia Institute of Technology

Dr. Ioannis Exarchos - Microsoft 

 

Abstract:

Stochastic Optimal Control in continuous-time requires solving the Hamilton-Jacobi-Bellman equation which suffers from the well-known curse-of-dimensionality. Instead of directly attempting to solve the HJB, one can obtain probabilistic representations of the solution via the Nonlinear Feynman-Kac lemma that relates the unique solution of the HJB to that of a system of Foward-Backward Stochastic Differential Equations or FBSDEs. This work develops novel algorithms that leverage the function approximation capabilities of deep neural networks to solve FBSDEs and empirically demonstrates that the framework is immune to compounding errors in comparison to past FBSDE works and is therefore, scalable. The resulting deep FBSDE framework is memory efficient, provides temporally smoother controls and can be employed for long time-horizons owing to the underlying Long-Short Term Memory network architecture. Starting from a basic Stochastic Optimal Control problem, the framework is extended to problem formulations such as systems with control-multiplicative noise, systems with dynamics non-affine with respect to the control and for non-quadratic control cost functions, for safety critical tasks which employs stochastic control barrier functions and for L1 stochastic optimal control in minimum-fuel aerospace applications. Each problem formulation is accompanied with necessary structural modifications to the deep learning architecture to enable end-to-end learning. Two directions of future research are then proposed which, (i.) improve scalability and (ii.) improve applicability of the framework. The first employs the Alternating Direction Method of Multipliers to offer a scalable solution for large-scale safe multi-agent stochastic optimal control and the second direction aims to extend the framework to systems with unknown dynamics using recent techniques borrowed from model-based reinforcement learning. 

 

Event Details

Date/Time:

  • Monday, November 1, 2021
    10:00 am - 12:00 pm
Location: Montgomery Knight Building, MK 317 (conference room)

Accessibility Information

Per accessibility compliance standards, this page may have links to files that would require the downloading of additional software:

  • Click here to download Microsoft Products.
  • Click here to download Adobe Reader.