Electricity transmission networks are changing fast – not least because as a society, we’re setting ambitious sustainability goals to transition away from fossil fuels and towards renewable energy. “This is causing a fundamental change in approach,” explains Alessandro Zocca. “In the past, power grids were easier to manage – with predictable electricity flows and fully controllable power plants based on conventional fossil fuels. Now, with renewable energy sources, they’re less controllable because of the weather. When the wind doesn’t blow, wind turbines produce less energy; meanwhile, solar power depends on the availability of sunlight, with strong geographical correlations. Adding to the complexity, many consumers now contribute energy back to the grid from their solar panels. All of this is happening while electricity demand continues to rise, driven by electric vehicles, data centres, and other emerging technologies. As a result, the energy infrastructure is becoming more intricate and complex to manage.”
In the Netherlands, TenneT is the transmission system operator that manages the long-range transmission network, at the interface between power generators and energy users. Done right, TenneT’s work ensures that everyone in the Netherlands gets the electricity they need, avoids congestion on the power lines, improves safety and makes it possible to bring new, planned power plants online.
Topological actions to optimise power flow
The research that Zocca and his colleagues, Erica van der Sar and Sandjai Bhulai, are doing helps network operators like TenneT to take so-called “topological actions”, like switching lines to optimise the way the power flows. And this is where multi-agent reinforcement learning comes in. “We have lots of historical data on power generation and consumption. We can use this data to train an agent to find the right actions to take – including in potentially harmful test scenarios that are generated artificially. This approach allows the agent to learn from both the previous and potential situations and then, at a later stage, to leverage its insights to identify the best actions in real-time to maintain network stability and minimise congestion.”
With any electricity network, however, there are exponentially many possible topological actions that could be taken. The decision making required is too much for a single agent to manage, so the network is split into clusters – each with its own local agent. “It’s essentially a collection of brains,” explains Alessandro. “But as you might imagine, having hundreds of brains making decisions simultaneously would get pretty chaotic pretty quickly. So instead, we’ve created a hierarchy: there’s effectively one brain at a higher level coordinating all the other brains about when it’s their turn. This has the dual benefits of making it easier for the other agents to adjust their own decisions, and mirroring how human engineers currently manage the grid.”
Control Room of the Future
But how does this all work in reality? Are reinforcement learning agents really making all these decisions? Jan Viebahn, lead data scientist at TenneT, explains: “In practice, we’re developing what we call the Control Room of the Future, which combines the best of human and computer intelligence. It has an innovative visualisation dashboard, which will give operators a comprehensive oversight of the whole power system, down to individual network components – which will enable the grid to be used to its maximum capacity, safely and securely.”
Zocca concludes: “At the end of the day, it’s still the humans making the decision – but they have AI-driven tools at their fingertips every day. In this rapidly evolving energy landscape where there are more renewable energy sources being deployed as we speak, having a dashboard to help humans make important decisions based on these highly-trained AI agents could just be the difference between safe energy provision and a blackout.”
