RL4AA Collaboration

Collaboration on Reinforcement Learning for Autonomous Accelerators


Find your way around the site

RL4AA'24 workshop group photo

Successful RL4AA'24 workshop in Salzburg: Thanks everyone for joining!

From 5 to 7 February 2024, IDA Lab at the Paris Lodron University of Salzburg kindly hosted the RL4AA community for the 2nd workshop on Reinforcement Learning for Autonomous Accelerators (RL4AA'24). With over 50 participants from more than 10 different countries, we are excited to see that our community is growing and that the interest in reinforcement learning (for particle accelerators) is increasing. In a total of 19 talks, we got to hear about the latest developments and impressive results in the field....

February 16, 2024 · 339 words · RL4AA Collaboration
RL4AA'24 flyer.

RL4AA'24 call for abstracts extended to 5 January. Exciting keynote speakers announced. Register now!

The call for abstracts for the RL4AA'24 workshop, taking place 05 - 07 February 2024 in Salzburg, Austria, has been extended. Register for the workshop and submit your abstract until 5 January 2024! We are also excited to announce our keynote speakers: Antonin Raffin (German Aerospace Center) Felix Berkenkamp (Bosch Center for AI) For more information on the workshop, please see the official workshop website: https://rl4aa.github.io/RL4AA24/ To get directly to registration, please visit: https://indico....

December 22, 2023 · 95 words · RL4AA Collaboration

Registration is now open for RL4AA'24 taking place 05 - 07 February 2024 in Salzburg, Austria

Announcing RL4AA'24 - Registration is open! Following up on the very successful RL4AA'23 workshop in Karlsruhe earlier this year, we are excited to announce the 2nd RL4AA workshop RL4AA'24, which will be held in Salzburg, Austria, from 05 - 07 February 2024. The workshop will be hosted at the Paris Lodron University of Salzburg. We are looking forward to an exciting workshop with many interesting talks and discussions on reinforcement learning for autonomous particle accelerators and hope to see you all in Salzburg!...

August 18, 2023 · 96 words · RL4AA Collaboration
Simplified 3D illustration of the considered section of the ARES particle accelerator.

Learning to Do or Learning While Doing: Reinforcement Learning and Bayesian Optimisation for Online Continuous Tuning

J. Kaiser1, C. Xu2, A. Eichler1, A. Santamaria Garcia2, O. Stein1, E. Bründermann2, W. Kuropka1, H. Dinter1, F. Mayet1, T. Vinatier1, F. Burkart1, H. Schlarb1 1Deutsches Elektronen-Synchrotron DESY, 2 Karlsruhe Institute of Technology KIT arXiv Abstract Online tuning of real-world plants is a complex optimisation problem that continues to require manual intervention by experienced human operators. Autonomous tuning is a rapidly expanding field of research, where learning-based methods, such as Reinforcement Learning-trained Optimisation (RLO) and Bayesian optimisation (BO), hold great promise for achieving outstanding plant performance and reducing tuning times....

June 6, 2023 · 201 words · RL4AA Collaboration
Discord logo.

The RL4AA Discord server is up!

Good News! The RL4AA community is happy to announce its Discord server! If you are interested in discussing reinforcement learning applied to accelerators, please join for announcements (e.g. new publications), forum discussions, an open chat, and meeting rooms. Hope to see you there! https://discord.gg/QtBMqsjWH2

June 2, 2023 · 44 words · RL4AA Collaboration
Overview of the training loop and the structure of simulated environment

Trend-Based SAC Beam Control Method with Zero-Shot in Superconducting Linear Accelerator

X. Chen, X. Qi, C. Su, Y. He, Z. Wang, K. Sun, C. Jin, W. Chen, S. Liu, X. Zhao, D. Jia, M. Yi Chinese Academy of Sciences arXiv Abstract The superconducting linear accelerator is a highly flexiable facility for modern scientific discoveries, necessitating weekly reconfiguration and tuning. Accordingly, minimizing setup time proves essential in affording users with ample experimental time. We propose a trend-based soft actor-critic(TBSAC) beam control method with strong robustness, allowing the agents to be trained in a simulated environment and applied to the real accelerator directly with zero-shot....

May 23, 2023 · 244 words · RL4AA Collaboration
Overview of the steering method.

Ultra fast reinforcement learning demonstrated at CERN AWAKE

** Simon Hirlaender, Lukas Lamminger, Giovanni Zevi Della Porta, Verena Kain** Abstract Reinforcement learning (RL) is a promising direction in machine learning for the control and optimisation of particle accelerators since it learns directly from experience without needing a model a-priori. However, RL generally suffers from low sample efficiency and thus training from scracth on the machine is often not an option. RL agents are usually trained or pre-tuned on simulators and then transferred to the real environment....

May 1, 2023 · 233 words · RL4AA Collaboration

The 2nd RL4AA workshop 2024 will be held 05 - 07 February 2024 in Salzburg

Good News - save the date! Our first workshop, RL4AA 2023, was very successful. Because of this, we are hoping to hold the 2nd RL4AA workshop in spring 2024 in Salzburg, Austria in 05 - 07 February 2024. Further details will follow soon.

April 27, 2023 · 43 words · RL4AA Collaboration
AWAKE beamline showing location of the matching devices (actions) and the observation BTV.

Towards automatic setup of 18 MeV electron beamline using machine learning

F. M. Velotti1, B. Goddard1, V. Kain1, R. Ramjiawan1, G. Z. Della Porta1 and S. Hirlaender2 1CERN, 2University of Salzburg Machine Learning: Science and Technology Abstract To improve the performance-critical stability and brightness of the electron bunch at injection into the proton-driven plasma wakefield at the AWAKE CERN experiment, automation approaches based on unsupervised machine learning (ML) were developed and deployed. Numerical optimisers were tested together with different model-free reinforcement learning (RL) agents....

April 27, 2023 · 189 words · RL4AA Collaboration
Overview of the orbit correction method.

Orbit Correction Based on Improved Reinforcement Learning Algorithm

X. Chen, Y. Jia, X. Qi, Z. Wang, Y. He Chinese Academy of Sciences Physical Review Accelerators and Beams Abstract Recently, reinforcement learning (RL) algorithms have been applied to a wide range of control problems in accelerator commissioning. In order to achieve efficient and fast control, these algorithms need to be highly efficient, so as to minimize the online training time. In this paper, we incorporated the beam position monitor trend into the observation space of the twin delayed deep deterministic policy gradient (TD3) algorithm and trained two different structure agents, one based on physical prior knowledge and the other using the original TD3 network architecture....

April 13, 2023 · 327 words · RL4AA Collaboration