In this lecture, we will examine the emerging paradigm of foundation models for physics.
\nRapid advances on scientific problems such as jet tagging in particle physics, gravitational-wave detection in astrophysics, or weather prediction in meteorology suggest that the next breakthroughs in the physical sciences will be driven by machine learning. Recent work has increasingly adopted the foundation model paradigm for science, in which models are trained on broad and heterogeneous datasets spanning multiple physical regimes, and later adapted to a wide range of downstream tasks, including prediction, parameter inference, and other inverse problems. We will show how fundamental concepts such as multiscale dynamics, partial observability, and generalization, guide the design of these foundation models, through examples and hands-on exercises in fluid dynamics and solar physics.
\n\u00a0
\nLink to google colab:\u00a0
\nhttps://colab.research.google.com/drive/1v0cNw7TveOTGUDj71AcW908NwGX_-pQ7?usp=sharing
\nRudy Morel is an Associate Research Scientist in the Scientific Computing Core at the Flatiron Institute in New York. Previously, he was a Research Fellow in the Center for Computational Mathematics and completed his PhD in Applied Mathematics at the \u00c9cole Normale Sup\u00e9rieure in Paris.
\nPlease note that pictures and videos might be taken during the event. The pictures and videos might be used for communication about the event. By joining the lecture, you are agreeing to being featured in these communication actions.\u00a0
Reinforcement Learning (RL) has emerged as a powerful paradigm in artificial intelligence and has found exciting applications in various fields, including particle accelerators at CERN. This introductory lecture aims to provide an overview of RL and its application in optimizing beam steering in the AWAKE beamline and automating bunch splitting in the Proton Synchrotron (PS) at CERN.
\nThe lecture will begin with an introduction to the fundamentals of RL, where we will explore the concept of agents learning to make decisions through interaction with an environment. Central to RL is the concept of Markov Decision Processes (MDPs), which model sequential decision-making problems. We will discuss the components of an MDP, including states, actions, rewards, and transition probabilities.
\nNext, we will delve into sample-based methods, such as Monte Carlo and Temporal Difference (TD) learning, which are essential for learning in uncertain and dynamic environments. These techniques allow RL agents to estimate value functions and improve their policies through experience.
\nModel-based RL will be introduced as an approach to learn a model of the environment to aid in decision-making. Dyna, a well-known model-based RL algorithm, will be presented as an example of how this integration can be achieved.
\nFunction approximation techniques applied to both parametric value functions and policies will be covered. These methods enable RL agents to handle high-dimensional state and action spaces, making them valuable tools in complex applications.
\nThe lecture will conclude by introducing actor-critic methods, which combine the advantages of both policy gradient and value-based methods. We will briefly discuss how these algorithms facilitate efficient learning and convergence in RL tasks.
\nFinally, we will transition to the exciting RL applications at CERN.\u00a0
\nMatteo Bunino earned a MSc in Data Science and Computer Engineering from both the Polytechnic University of Turin (Italy) and EURECOM (France). He did his thesis at Huawei's Munich Research Center (MRC), where he developed a prototype for analyzing dynamically evasive malware with the aid of reinforcement learning.
\nAfter university, Matteo joined CERN, where he worked on interTwin, a European project aimed at developing a unified digital twin engine (DTE) for science, and contributed to CERN openlab. During this time he focused on the development of \"itwinai\", a framework for advanced MLOps on cloud and HPC.
\nCurrently, Matteo is part of the Kubernetes platform team in the CERN IT department, where he works on the ODISSEE project, whose goal is to create the digital twin of the LHCb datacentre.