A team comprising engineers, physicists, and data scientists from Princeton University and the US Department of Energy’s (DOE’s) Princeton Plasma Physics Laboratory (PPPL) have harnessed the power of artificial intelligence (AI) to predict and avoid the formation of a specific plasma problem in real time.
During experiments at the DIII-D National Fusion Facility in San Diego, the researchers demonstrated their model, trained only on past experimental data, could forecast potential plasma instabilities known as tearing mode instabilities up to 300 milliseconds in advance. This was enough time for the AI controller to change certain operating parameters to avoid what would have developed into a tear within the plasma’s magnetic field lines, upsetting its equilibrium and ending the reaction.
“By learning from past experiments, rather than incorporating information from physics-based models, the AI could develop a final control policy that supported a stable, high-powered plasma regime in real time, at a real reactor,” said research leader Egemen Kolemen, Associate Professor of Mechanical & Aerospace Engineering at the Andlinger Center for Energy & the Environment, as well as staff research physicist at PPPL.
The research paves the way for more dynamic control of a fusion reaction than current approaches and provides a foundation for using AI to solve a broad range of plasma instabilities, which have long been obstacles to achieving a sustained fusion reaction.
“Previous studies have generally focused on either suppressing or mitigating the effects of these tearing instabilities after they occur in the plasma,” said first author Jaemin Seo, Assistant Professor of Physics at South Korea’s Chung-Ang University, who performed much of the work as a postdoctoral researcher in Kolemen’s group. “But our approach allows us to predict and avoid those instabilities before they ever appear.”
In tokamak fusion reactors, magnetic fields are used to contain plasmas that reach above 100m degrees Celsius. While there are many types of plasma instabilities that can terminate the reaction in a tokamak, the Princeton team concentrated on solving tearing mode instabilities, a disturbance in which the magnetic field lines within a plasma break and create an opportunity for the plasma to escape.
“Tearing mode instabilities are one of the major causes of plasma disruption, and they will become even more prominent as we try to run fusion reactions at the high powers required to produce enough energy,” said Seo. “They are an important challenge for us to solve.” Because tearing mode instabilities can form and derail a fusion reaction in milliseconds, the researchers looked to artificial intelligence for its ability to quickly process and act in response to new data.
However, the process to develop an effective AI controller proved to be more difficult than trying out a few things on a tokamak, where time is limited, and the stakes are high. Co-author Azarakhsh Jalalvand, a research scholar in Kolemen’s group, compared teaching an algorithm to run a fusion reaction in a tokamak to teaching someone how to fly a plane. “You wouldn’t teach someone by handing them a set of keys and telling them to try their best,” he said. “Instead, you’d have them practice on a very intricate flight simulator until they’ve learned enough to try out the real thing.”
Like developing a flight simulator, the Princeton team used data from past experiments at the DIII-D tokamak to construct a deep neural network capable of predicting the likelihood of a future tearing instability based on real-time plasma characteristics. They then used that neural network to train a reinforcement learning algorithm. Like a pilot trainee, the reinforcement learning algorithm could try out different strategies for controlling plasma, learning through trial and error which strategies worked within the safety of a simulated environment.
“We don’t teach the reinforcement learning model all of the complex physics of a fusion reaction,” Jalalvand said. “We tell it what the goal is – to maintain a high-powered reaction – what to avoid – a tearing mode instability – and the knobs it can turn to achieve those outcomes. Over time, it learns the optimal pathway for achieving the goal of high power while avoiding the punishment of an instability.”
As the model went through countless simulated fusion experiments, trying to find ways to maintain high power levels while avoiding instabilities, co-author SangKyeun Kim could observe and refine its actions. “In the background, we can see the intentions of the model,” said Kim, a staff research scientist at PPPL and former postdoctoral researcher in Kolemen’s group. “Some of the changes that the model wants are too rapid, so we work to smooth and calm the model. As humans, we arbitrate between what the AI wants to do and what the tokamak can accommodate.”
Once they were confident in the AI controller’s abilities, they tested it during a fusion experiment at the D-III D tokamak, seeing how the controller made real-time changes to certain tokamak parameters to avoid the onset of an instability. These parameters included changing the shape of the plasma and the strength of the beams inputting power into the reaction.
“Being able to predict instabilities ahead of time can make it easier to run these reactions than current approaches, which are more passive,” said Kim. “We no longer have to wait for the instabilities to occur and then take quick corrective action before the plasma becomes disrupted.”
This is just the first of many steps already ongoing in Kolemen’s group. There is a need to get more evidence of the AI controller in action at the DIII-D tokamak, and then expand the controller to function at other tokamaks. “We have strong evidence that the controller works quite well at DIII-D, but we need more data to show that it can work in a number of different situations,” said Seo. “We want to work toward something more universal.”
A second line of research involves expanding the algorithm to handle many different control problems at the same time. While the current model uses a limited number of diagnostics to avoid one specific type of instability, the researchers could provide data on other types of instabilities and give access to more knobs for the AI controller to tune.
“You could imagine one large reward function that turns many different knobs to simultaneously control for several types of instabilities,” said co-author Ricardo Shousha, a postdoc at PPPL and former graduate student in Kolemen’s group who provided support for the experiments at DIII-D. And in the course of developing better AI controllers for fusion reactions, researchers may also gain more understanding of the underlying physics. “Eventually, it may be more than just a one-way interaction of scientists developing and deploying these AI models,” said Kolemen. “By studying them in more detail, they may have certain things that they can teach us too.”
The paper, “Avoiding fusion plasma tearing instability with deep reinforcement learning,” was published in Nature. In addition to Kolemen, Seo, Jalalvand, Kim, and Shousha, co-authors include Rory Conlin, Joseph Abbate and Josiah Wai of Princeton University, as well as Keith Erickson of PPPL.
The work was supported by the DOE’s Office of Fusion Energy Sciences, as well as the National Research Foundation of Korea (NRF). The authors also acknowledge the use of the DIII-D National Fusion Facility, a Department of Energy Office of Science user facility.
Image (L-R): Study authors Azarakhsh Jalalvand, Egemen Kolemen, and Ricardo Shousha stand in the Andlinger Center (courtesy of Princeton University)
