Designing efficient new catalysts is no easy task. In catalysts that contain more than one element, for example, researchers not only need to take into account all the possible elemental combinations, they must also add a number of other variables, such as particle size, shape and surface structure, as well as the degree of alloying or phase segregation. This ultimately leads to an overwhelmingly large number of potential candidates.
To address this challenge, scientists employ computational design techniques that focus on screening material components and alloy composition to optimize a catalyst’s activity for a given reaction and so reduce the overall number of prospective structures that would need to be tested and then developed. Such techniques require combinatorial approaches coupled with theory calculations, which can both be time-consuming and complex.
The best surface atom configurations
A team led by Zachary Ulissi of Carnegie Mellon University has now taken a different approach by developing a deep reinforcement learning (DRL) programme, dubbed CatGym, that iteratively changes the positions of atoms on the surface of a catalyst to find the best configurations from a given starting configuration.
The researchers showcased their technique by predicting the surface reconstruction pathways of a ternary Ni3Pd3Au2(111) alloy catalyst. Their results show that the DRL programme can not only be used to explore more diverse surface compositions than conventional methods, but that it can also generate new pathways based on how energetically favourable they are.
The team also demonstrated that the kinetic pathways that lead to a stable surface composition (with a low minimum energy surface composition) and the associated transition state predicted by the DRL programme agree well with the minimum energy path predicted by traditional “nudged elastic band” calculations done “by hand”.
A lot of human input
There has been much excitement in recent years for when it comes to using machine learning methods to accelerate catalysis simulations, says Ulissi. Such an approach reduces the computational cost of each step in the simulation, but the downside is that it requires a lot of human input to run the calculation. This is because scientists need to define what structure is used from the outset, what mechanisms should be investigated and if there is a better path to take to go from reaction A to reaction B. All these questions require a trained expert many days or weeks to answer.
“The new work is very exciting for us because it proposes using DRL methods to tackle these strategic questions,” Ulissi tells Physics World. “With our system, we can let the computer autonomously explore a number of different possible pathways.”
Representation and action space
DRL requires three things, he explains. “The first is a representation – that is, how do we show an atomic structure of a catalyst to the computer in a way that it understands? In our system we use a common representation from the literature. The second is an “action space”: what are we going to let the computer do? In our approach, it can move an atom, find an energy minimum, find a transition state or run a short dynamic simulation. Finally, how do we decide what action to take next? In our case, we tried many DRL schemes to answer this question.”
“One aspect that made this project really interesting was that the final goal was not clear,” explains Ulissi. “In a video game, for example, it is obvious what you want you want the DRL to do – maximize the final score. We thus spent a lot of time defining and identifying the goal the DRL would work well with.”
Double-checking results
Ulissi says he previously studied catalyst surface reconstruction mechanisms by hand, which can be very tedious. “A tool to automate and accelerate this process not only allows us to ask much more interesting questions, it can also be used to double-check the results obtained by human experts.”
Computational methods lead researchers to more precise chemistry
The researchers, who report their work in Machine Learning: Science and Technology, are now using the method they have developed to predict how stable hypothetical catalyst surfaces are. “We also hope to apply our approach to better understand the mechanisms at play on these surfaces,” adds Ulissi. “Doing this will help us think creatively about what might happen to a catalyst during real-world reactions.”
It will not all be plain sailing, however, he admits. One of the major limitations of the current technique is that, like most DRL applications, it requires a lot of data input and training episodes. “Accurate simulations are extremely demanding computationally,” he explains, “and the simulations we performed in our work are fast but rather coarse approximations.” The researchers are trying to solve this problem by also using machine learning models to make them not only faster but also more accurate.
