EnergyVille is building a strong international reputation in the field of demand response in a smart grids context, both at an experimental [1] and an algorithmic level [2,3]. Near-optimal control of flexibility however, relies on the availability of an accurate model describing the dynamics of the cluster. Defining and calibrating an accurate model often requires dedicated expert knowledge. This strongly reduces the practicality of such an approach, certainly given that the model and its parameters need repeated updates. Whilst this is less a bottleneck for large consumers, this can be a show-stopper for managing flexibility related to small consumers. To this end EnergyVille leverages on recent developments in machine learning and reinforcement learning [4,5,6] to obtain general-purpose self-learning control algorithms tailored to energy applications for small to large groups of consumers harbouring flexibility.

Multi-agent reinforcement learning (MARL) is a promising technique for automatically managing complex and highly-distributed networked systems. Reinforcement learning possesses the inherit ability to cope with uncertainty and allows agents to automatically learn how to properly behave in face of unexpected situations. The introduction of multi-agent concepts supports effective inter-agent collaborations, allowing them to jointly achieve their, possibly competing, goals. Additionally, human operators will still be needed to effectively control and guide the automated agents in line with the high-level business goals and processes of the managed system. Intuitive and novel programming abstractions are needed to facilitate the straight-forward configuration of such multi-agent systems.

The overall project’s goals is to develop a MARL framework, and to apply it to different application domains (telecom, smart grids, traffic, … – see figure online). The goal of this PhD focuses on the application to smart grids. Specific research questions include dealing with uncertainty, learning over large periods of time, self-healing and robustness.

Specifically,the researcher will drive the evaluation of the developed MARL framework from a smart grids point of view, by setting requirements, developing optimization strategies, increasing robustness through self-healing mechanism, implementing learning techniques, and driving valorisation for the smart grids case.