Publications

The RL paradigm as applied to particle accelerator control, showing the example of trajectory correction.

Sample-efficient reinforcement learning for CERN accelerator control

V. Kain1, S. Hirlander1, B. Goddard1, F. M. Velotti1, G. Z. Della Porta1, N. Bruchon2, G. Valentino3 1CERN, 2University of Trieste, 3University of Malta Physical Review Accelerators and Beams Abstract Numerical optimization algorithms are already established tools to increase and stabilize the performance of particle accelerators. These algorithms have many advantages, are available out of the box, and can be adapted to a wide range of optimization problems in accelerator operation....

Schematic for neural network control policy updates.

Neural Networks for Modeling and Control of Particle Accelerators

A. L. Edelen Colorado State University PhD thesis Abstract Charged particle accelerators support a wide variety of scientific, industrial, and medical applications. They range in scale and complexity from systems with just a few components for beam acceleration and manipulation, to large scientific user facilities that span many kilometers and have hundreds-to-thousands of individually-controllable components. Specific operational requirements must be met by adjusting the many controllable variables of the accelerator. Meeting these requirements can be challenging, both in terms of the ability to achieve specific beam quality metrics in a reliable fashion and in terms of the time needed to set up and maintain the optimal operating conditions....

Simple scheme of the FERMI FEL seed laser alignment set up.

Basic Reinforcement Learning Techniques to Control the Intensity of a Seeded Free-Electron Laser

N. Bruchon1, G. Fenu1, G. Gaio2, M. Lonza2, F. H. O’Shea2, F. A. Pellegrino1, E. Salvato1 1University of Trieste, 2Elettra Sincrotrone Trieste Electronics Abstract Optimal tuning of particle accelerators is a challenging task. Many different approaches have been proposed in the past to solve two main problems—attainment of an optimal working point and performance recovery after machine drifts. The most classical model-free techniques (e.g., Gradient Ascent or Extremum Seeking algorithms) have some intrinsic limitations....

Simple scheme of the EOS laser alignment set up.

Toward the Application of Reinforcement Learning to the Intensity Control of a Seeded Free-Electron Laser

N. Bruchon, G. Fenu, G. Gaio, M. Lonza, F. A. Pellegrino, E. Salvato University of Trieste 23rd International Conference on Mechatronics Technology Abstract The optimization of particle accelerators is a challenging task, and many different approaches have been proposed in years, to obtain an optimal tuning of the plant and to keep it optimally tuned despite drifts or disturbances. Indeed, the classical model-free approaches (such as Gradient Ascent or Extremum Seeking algorithms) have intrinsic limitations....

Scheme of the FERMI FEL seed laser alignmentset up

Free-electron Laser Optimization with Reinforcement Learning

N. Bruchon1, G. Gaio2, G. Fenu1, M. Lonza2, F. A. Pellegrino1, E. Salvato1 1University of Trieste, 2Elettra Sincrotrone Trieste 17th International Conference on Accelerator and Large Experimental Physics Control Systems Abstract Reinforcement Learning (RL) is one of the most promis-ing techniques in Machine Learning because of its modestcomputational requirements with respect to other algorithms.RL uses an agent that takes actions within its environmentto maximize a reward related to the goal it is designed toachieve....

General feedback scheme using the CSR powersignal to construct both, the state and reward signal of the Markov decision process (MDP).

Feedback Design for Control of the Micro-Bunching Instability Based on Reinforcement Learning

T. Boltz, M. Brosi, E. Bründermann, B. Haerer, P. Kaiser, C. Pohl, P. Schreiber, M. Yan,T. Asfour, A.-S. Müller Karlsruhe Insitute of Technology KIT 10th International Particle Accelerator Conference Abstract The operation of ring-based synchrotron light sourceswith short electron bunches increases the emission of co-herent synchrotron radiation (CSR) in the THz frequencyrange. However, the micro-bunching instability resultingfrom self-interaction of the bunch with its own radiationfield limits stable operation with constant intensity of CSRemission to a particular threshold current....

Using a neural network control policy for rapid switching between beam parameters in an FEL

A. L. Edelen1, S. G. Biedron2, J. P. Edelen3, S. V. Milton4, P. J. M. van der Slot5 1Colorado State University, 2Element Aero, 3Fermi National Accelerator Laboratory, 4Los Alamos National Laboratory, 5University of Twente 38th International Free Electron Laser Conference Abstract FEL user facilities often must accommodate requests for a variety of beam parameters. This usually requires skilled operators to tune the machine, reducing the amount of available time for users....

General setup for the neural network model.

Using Neural Network Control Policies For Rapid Switching Between Beam Parameters in a Free Electron Laser

A. L. Edelen1, S. G. Biedron2, J. P. Edelen3, S. V. Milton4, P. J. M. van der Slot5 1Colorado State University, 2Element Aero, 3Fermi National Accelerator Laboratory, 4Los Alamos National Laboratory, 5University of Twente Workshop on Deep Learning for Physical Sciences at the Conference on Neural Information Processing Systems 2017 Abstract Free Electron Laser (FEL) facilities often must accommodate requests for a varietyof electron beam parameters in order to supply scientific users with appropriatephoton beam characteristics....

Orbit Correction Studies Using Neural Networks

E. Meier, Y.-R. E. Tan, G. S. LeBlanc Australian Synchrotron 3rd International Particle Accelerator Conference Abstract This paper reports the use of neural networks for orbitcorrection at the Australian Synchrotron Storage Ring. Theproposed system uses two neural networks in an actor-criticscheme to model a long term cost function and computeappropriate corrections. The system is entirely based onthe history of the beam position and the actuators, i.e. thecorrector magnets, in the storage ring....