publication

Schematic view of the GMPS control environment.

Real-time artificial intelligence for accelerator control: A study at the Fermilab Booster

J. St. John1, C. Herwig1, D. Kafkes1, J. Mitrevski1, W. A. Pellico1, G. N. Perdue1, A. Quintero-Parra1, B. A. Schupbach1, K. Seiya1, N. Tran1, M. Schram2, J. M. Duarte3, Y. Huang4, R. Keller5 1Fermi National Accelerator Laboratory, 2Thomas Jefferson National Accelerator Laboratory, 3University of California San Diego, 4Pacific Northwest National Laboratory, 5Columbia University Physical Review Accelerators and Beams Abstract We describe a method for precisely regulating the gradient magnet power supply (GMPS) at the Fermilab Booster accelerator complex using a neural network trained via reinforcement learning....

Online training of NAF Agent of AWAKE electronline trajectory steering in the horizontal plane.

Test of Machine Learning at the CERN LINAC4

V. Kain1, N. Bruchon1, S. Hirlander1, N. Madysa1, I. Vojskovic1, P. Skowronski1, G. Valentino2 1CERN, 2University of Malta 61st ICFA ABDW on High-Intensity and High-Brightness Hadron Beams Abstract The CERN H−linear accelerator, LINAC4, served as atest bed for advanced algorithms during the CERN LongShutdown 2 in the years 2019/20. One of the main goals wasto show that reinforcement learning with all its benefits canbe used as a replacement for numerical optimization and asa complement to classical control in the accelerator controlcontext....

Best PPO agent. Action is deterministic.

Renovation of the beam-based feedback systems in the LHC

L. Grech University of Malta PhD thesis Abstract The Large Hadron Collider (LHC) at the European Organization for Nuclear Research (CERN) is the largest synchrotron built to date, having a circumference of approx- imately 27km. The LHC is able to accelerate two counter-rotating proton and/or heavy-ion beams up to 7 TeV per charge. These highly energetic beams are contained inside a vacuum chamber with an inner diameter of 80 mm by means of strong mag- netic fields produced by superconducting magnets....

Accelerated Deep Reinforcement Learning for Fast Feedback of Beam Dynamics at KARA

W. Wang1, M. Caselle1, T. Boltz1, E. Blomley1, M. Brosi1, T. Dritschler1, A. Ebersoldt1, A. Kopmann1, A. Santamaria Garcia1, P. Schreiber1, E. Bründermann1, M. Weber1, A.-S. Müller1, Y. Fang2 1Karlsruhe Insitute of Technology KIT, 2Northwestern Polytechnical University IEEE Transactions on Nuclear Science Abstract Coherent synchrotron radiation (CSR) is generated when the electron bunch length is in the order of the magnitude of the wavelength of the emitted radiation. The self-interaction of short electron bunches with their own electromagnetic fields changes the longitudinal beam dynamics significantly....

RL environment for beam optimisation in theARES EA.

First Steps Toward an Autonomous Accelerator, A Common Project Between DESY and KIT

A. Eichler1, F. Burkart1, J. Kaiser1, W. Kuropka1, O. Stein1, E. Bründermann2, A. Santamaria Garcia2, C. Xu2 1Deutsches Elektronen-Synchrotron DESY, 2Karlsruhe Institute of Technology KIT 12th International Particle Accelerator Conference Abstract Reinforcement learning algorithms have risen in pop-ularity in the accelerator physics community in recentyears, showing potential in beam control and in the opti-mization and automation of tasks in accelerator operation.The Helmholtz AI project “Machine Learning Toward Au-tonomous Accelerators” is a collaboration between DESYand KIT that works on investigating and developing rein-forcement learning applications for the automatic start-upof electron linear accelerators....

Reinforcement learning agent joint with the physics-based polynomial neural network.

Physics-Enhanced Reinforcement Learning for Optimal Control

A. Ivanov, I. Agapov, A. Eichler, S. Tomin Deutsches Elektronen Synchrotron DESY 12th International Particle Accelerator Conference Abstract We propose an approach for incorporating acceleratorphysics models into reinforcement learning agents. The proposed approach is based on the Taylor mapping technique for the simulation of particle dynamics. The resulting computational graph is represented as a polynomial neural network and embedded into the traditional reinforcement learning agents. The application of the model is demonstrated in a nonlinear simulation model of beam transmission....

Simple scheme of the FERMI FEL seed laser alignment set up.

Feasibility Investigation on Several Reinforcement Learning Techniques to Improve the Performance of the FERMI Free-Electron Laser

N. Bruchon University of Trieste PhD thesis Abstract The research carried out in particle accelerator facilities does not concern only particle and condensed matter physics, although these are the main topics covered in the field. Indeed, since a particle accelerator is composed of many different sub-systems, its proper functioning depends both on each of these parts and their interconnection. It follows that the study, implementation, and improvement of the various sub-systems are fundamental points of investigation too....

Plot of the reward received by the agent versus step number.

Policy gradient methods for free-electron laser and terahertz source optimization and stabilization at the FERMI free-electron laser at Elettra

F. H. O’Shea1, N. Bruchon2, G. Gaio1 1Elettra Sincrotrone Trieste, 2University of Trieste Physical Review Accelerators and Beams Abstract In this article we report on the application of a model-free reinforcement learning method to the optimization of accelerator systems. We simplify a policy gradient algorithm to accelerator control from sophisticated algorithms that have recently been demonstrated to solve complex dynamic problems. After outlining a theoretical basis for the functioning of the algorithm, we explore the small hyperparameter space to develop intuition about said parameters using a simple number-guess environment....

A schematic overview of theAE-DYNAapproach used in this paper.

Model-free and Bayesian Ensembling Model-based Deep Reinforcement Learning for Particle Accelerator Control Demonstrated on the FERMI FEL

S. Hirlaender1, N. Bruchon2 1University of Salzburg, 2University of Trieste arXiv Abstract Reinforcement learning holds tremendous promise in accelerator controls. The primary goal of this paper is to show how this approach can be utilised on an operational level on accelerator physics problems. Despite the success of model-free reinforcement learning in several domains, sample-efficiency still is a bottle-neck, which might be encompassed by model-based methods. We compare well-suited purely model-based to model-free reinforcement learning applied to the intensity optimisation on the FERMI FEL system....

Autonomous Control of a Particle Accelerator using Deep Reinforcement Learning

X. Pang1, S. Thulasidasan2, L. Rybarcyk2 1Apple, 2Los Alamos National Laboratory Machine Learning for Engineering Modeling, Simulation, and Design Workshop at Neural Information Processing Systems 2020 Abstract We describe an approach to learning optimal control policies for a large, linear particle accelerator using deep reinforcement learning coupled with a high-fidelity physics engine. The framework consists of an AI controller that uses deep neural networks for state and action-space representation and learns optimal policies using reward signals that are provided by the physics simulator....