Reinforcement Learning

Markov Decision Processes (MDPs) and Reinforcement Learning (RL) provide a mathematical and computational framework for optimizing complex, time-dependent systems for example in mechanics. By modeling systems as agents interacting with stochastic environments, these methods allow for real-time adaptation in scenarios where traditional control laws struggle with uncertainty or high-dimensional dynamics. Application examples in mechanics:

• Fluid Mechanics: RL is applied to active flow control, enabling systems to manipulate actuators to reduce drag, suppress vortex shedding, or optimize mixing by learning from the fluid’s instantaneous pressure or velocity fields.

• Structural Mechanics: MDPs are used in Structural Health Monitoring (SHM) and maintenance planning, where the model determines the optimal sequence of inspections and repairs to maximize a structure’s lifespan while minimizing costs.

• Robotic Kinematics: RL algorithms solve complex motion planning tasks, allowing autonomous systems to navigate irregular terrains or handle flexible materials by adapting to friction and load variations that are difficult to model analytically.

This section of the course is very much inspired by the relevant section of the Youtube course [LS] (Stanford online CS221 2019). To go further, a very comprehensive book [SB18] is also freely available online (Reinforcement Learning).

• Markov Decision Process
  • Definitions
  • Policy value
  • Optimum policy
  • Algorithms
    • Policy evaluation
    • Value iteration
• Examples
  • Game
  • Magic tramway
  • Swamp isle
• Learning by acting
  • Definitions
  • Model based
  • Model free
    • Monte-Carlo
    • Sarsa
    • Q-learning
  • Exploration
• Examples
  • Environment
  • Game
  • Magic tram
  • Swamp isle