Collaboration on Reinforcement Learning for Autonomous Accelerators
From 5 to 7 February 2024, IDA Lab at the Paris Lodron University of Salzburg kindly hosted the RL4AA community for the 2nd workshop on Reinforcement Learning for Autonomous Accelerators (RL4AA'24). With over 50 participants from more than 10 different countries, we are excited to see that our community is growing and that the interest in reinforcement learning (for particle accelerators) is increasing. In a total of 19 talks, we got to hear about the latest developments and impressive results in the field....
The call for abstracts for the RL4AA'24 workshop, taking place 05 - 07 February 2024 in Salzburg, Austria, has been extended. Register for the workshop and submit your abstract until 5 January 2024! We are also excited to announce our keynote speakers: Antonin Raffin (German Aerospace Center) Felix Berkenkamp (Bosch Center for AI) For more information on the workshop, please see the official workshop website: https://rl4aa.github.io/RL4AA24/ To get directly to registration, please visit: https://indico....
Announcing RL4AA'24 - Registration is open! Following up on the very successful RL4AA'23 workshop in Karlsruhe earlier this year, we are excited to announce the 2nd RL4AA workshop RL4AA'24, which will be held in Salzburg, Austria, from 05 - 07 February 2024. The workshop will be hosted at the Paris Lodron University of Salzburg. We are looking forward to an exciting workshop with many interesting talks and discussions on reinforcement learning for autonomous particle accelerators and hope to see you all in Salzburg!...
J. Kaiser1, C. Xu2, A. Eichler1, A. Santamaria Garcia2, O. Stein1, E. Bründermann2, W. Kuropka1, H. Dinter1, F. Mayet1, T. Vinatier1, F. Burkart1, H. Schlarb1 1Deutsches Elektronen-Synchrotron DESY, 2 Karlsruhe Institute of Technology KIT arXiv Abstract Online tuning of real-world plants is a complex optimisation problem that continues to require manual intervention by experienced human operators. Autonomous tuning is a rapidly expanding field of research, where learning-based methods, such as Reinforcement Learning-trained Optimisation (RLO) and Bayesian optimisation (BO), hold great promise for achieving outstanding plant performance and reducing tuning times....
Good News! The RL4AA community is happy to announce its Discord server! If you are interested in discussing reinforcement learning applied to accelerators, please join for announcements (e.g. new publications), forum discussions, an open chat, and meeting rooms. Hope to see you there! https://discord.gg/QtBMqsjWH2
X. Chen, X. Qi, C. Su, Y. He, Z. Wang, K. Sun, C. Jin, W. Chen, S. Liu, X. Zhao, D. Jia, M. Yi Chinese Academy of Sciences arXiv Abstract The superconducting linear accelerator is a highly flexiable facility for modern scientific discoveries, necessitating weekly reconfiguration and tuning. Accordingly, minimizing setup time proves essential in affording users with ample experimental time. We propose a trend-based soft actor-critic(TBSAC) beam control method with strong robustness, allowing the agents to be trained in a simulated environment and applied to the real accelerator directly with zero-shot....
** Simon Hirlaender, Lukas Lamminger, Giovanni Zevi Della Porta, Verena Kain** Abstract Reinforcement learning (RL) is a promising direction in machine learning for the control and optimisation of particle accelerators since it learns directly from experience without needing a model a-priori. However, RL generally suffers from low sample efficiency and thus training from scracth on the machine is often not an option. RL agents are usually trained or pre-tuned on simulators and then transferred to the real environment....
Good News - save the date! Our first workshop, RL4AA 2023, was very successful. Because of this, we are hoping to hold the 2nd RL4AA workshop in spring 2024 in Salzburg, Austria in 05 - 07 February 2024. Further details will follow soon.
F. M. Velotti1, B. Goddard1, V. Kain1, R. Ramjiawan1, G. Z. Della Porta1 and S. Hirlaender2 1CERN, 2University of Salzburg Machine Learning: Science and Technology Abstract To improve the performance-critical stability and brightness of the electron bunch at injection into the proton-driven plasma wakefield at the AWAKE CERN experiment, automation approaches based on unsupervised machine learning (ML) were developed and deployed. Numerical optimisers were tested together with different model-free reinforcement learning (RL) agents....
X. Chen, Y. Jia, X. Qi, Z. Wang, Y. He Chinese Academy of Sciences Physical Review Accelerators and Beams Abstract Recently, reinforcement learning (RL) algorithms have been applied to a wide range of control problems in accelerator commissioning. In order to achieve efficient and fast control, these algorithms need to be highly efficient, so as to minimize the online training time. In this paper, we incorporated the beam position monitor trend into the observation space of the twin delayed deep deterministic policy gradient (TD3) algorithm and trained two different structure agents, one based on physical prior knowledge and the other using the original TD3 network architecture....