Control of Hybrid Microgrids Using Deep Reinforcement Learning and Digital Twin

Document Type : Original Article

Authors

Faculty of Electrical and Computer Engineering, University of Tabriz, Tabriz, Iran

Abstract

The increasing integration of renewable energy sources (RES) in hybrid microgrids has introduced new challenges in maintaining stability, reliability, and optimal performance. This paper proposes a novel control framework that combines deep reinforcement learning (DRL) with digital twin (DT) technology to address these challenges. The DRL agent is trained in a virtual DT environment, enabling rapid learning and optimization of control strategies under dynamic conditions without risking real-world operations. The proposed method is tested on a hybrid microgrid comprising photovoltaic (PV), wind, and battery storage systems. Simulation results demonstrate that the DRL-DT framework achieves a 28.5% improvement in energy efficiency compared to conventional model predictive control (MPC). Additionally, the proposed approach enhances system stability by reducing voltage fluctuations by 21.3% and achieves a 32.7% reduction in load shedding during peak demand scenarios. The training time for the DRL agent is reduced by 40% due to the efficient simulation capabilities of the DT. These results highlight the robustness and adaptability of the DRL-DT framework, making it a promising solution for next-generation hybrid microgrid management. This study provides a significant step forward in leveraging artificial intelligence and digital twins to optimize hybrid microgrid operations, ensuring sustainable and resilient energy systems.

Keywords

Main Subjects

References

[1] H. Joorabli, G. B. Gharehpetian, S. Ghasemzadeh, and V. Ghods, "Identification and Determination of Contribution of Current Harmonics and Unbalanced in Microgrids Equipped with Advanced Metering Infrastructure " TABRIZ JOURNAL OF ELECTRICAL ENGINEERING, vol. 51, no. 1, pp. 71-81, 2021.
[2] V. K. Saini, A. S. Al-Sumaiti, and R. Kumar, "Data driven net load uncertainty quantification for cloud energy storage management in residential microgrid," Electric Power Systems Research, vol. 226, p. 109920, 2024.
[3] M. F. P. Salehpour, "Optimizing Data Transfer and Convergence Time for Federated Learning based on NSGA II," TABRIZ JOURNAL OF ELECTRICAL ENGINEERING, vol. 53, no. 1, pp. 61-67, 2023.
[4] J. G. Ordoñez, J. Barco-Jiménez, A. Pantoja, J. Revelo-Fuelagán, and J. E. Candelo-Becerra, "Comprehensive analysis of MPC-based energy management strategies for isolated microgrids empowered by storage units and renewable energy sources," Journal of Energy Storage, vol. 94, p. 112127, 2024.
[5] A. Sharma and N. Singh, "Load frequency control of connected multi-area multi-source power systems using energy storage and lyrebird optimization algorithm tuned PID controller," Journal of Energy Storage, vol. 100, p. 113609, 2024.
[6] M. VASOUJOUYBARI, E. Ataie, and M. Bastam, "An MLP-based Deep Learning Approach for Detecting DDoS Attacks " TABRIZ JOURNAL OF ELECTRICAL ENGINEERING, vol. 52, no. 3, pp. 195-204, 2022.
[7] H. Delavari and S. Naderian, "Reinforcement learning robust nonlinear control of a microgrid with hybrid energy storage systems," Journal of Energy Storage, vol. 81, p. 110407, 2024.
[8] K. Kumar, S. Kwon, and S. Bae, "Deep reinforcement learning-based control strategy for integration of a hybrid energy storage system in microgrids," Journal of Energy Storage, vol. 108, p. 114936, 2025.
[9] M. S. Abid, H. J. Apon, S. Hossain, A. Ahmed, R. Ahshan, and M. H. Lipu, "A novel multi-objective optimization based multi-agent deep reinforcement learning approach for microgrid resources planning," Applied Energy, vol. 353, p. 122029, 2024.
[10] W. Cao and L. Zhou, "Resilient microgrid modeling in Digital Twin considering demand response and landscape design of renewable energy," Sustainable Energy Technologies Assessments, vol. 64, p. 103628, 2024.
[11] O. A. Talab and I. Avci, "Energy Management in Microgrids Using Model-Free Deep Reinforcement Learning Approach," J IEEE Access, 2025.
[12] N. Cheng, X. Wang, Z. Li, Z. Yin, T. Luan, and X. S. Shen, "Toward enhanced reinforcement learning-based resource management via digital twin: Opportunities, applications, and challenges," J IEEE Network, 2024.
[13] C. Wang, M. Wang, A. Wang, X. Zhang, J. Zhang, H. Ma, N. Yang, Z. Zhao, C. S. Lai, and L. L. Lai, "Multiagent deep reinforcement learning-based cooperative optimal operation with strong scalability for residential microgrid clusters," J Energy, vol. 314, p. 134165, 2025.
[14] S. I. Kaitouni, I. Ait Abdelmoula, N. Es-sakali, M. O. Mghazli, H. Er-retby, Z. Zoubir, F. El Mansouri, M. Ahachad, and J. Brigui, "Implementing a Digital Twin-based fault detection and diagnosis approach for optimal operation and maintenance of urban distributed solar photovoltaics," J Renewable Energy Focus, vol. 48, p. 100530, 2024.
[15] "IEEE Guide for Using IEEE Std 1547 for Interconnection of Energy Storage Distributed Energy Resources with Electric Power Systems," IEEE Std 1547.9-2022, vol. 36, no. 7, pp. 1-87, 2022.
[16] M. Sengupta, Y. Xie, A. Habte, G. Buster, G. Maclaurin, P. Edwards, H. Sky, M. Bannister, and E. Rosenlieb, "The National Solar Radiation Database (NSRDB) Fiscal Years 2019-2021," National Renewable Energy Lab.(NREL), Golden, CO (United States)2022.
[17] T. Basso and R. DeBlasio, "IEEE smart grid series of standards IEEE 2030 (interoperability) and IEEE 1547 (interconnection) status," National Renewable Energy Lab.(NREL), Golden, CO (United States)2012.
[18] C. Edison, "Bronzeville Community Microgrid," ComEd Project Report 2020.
Tabriz Journal of Electrical Engineering
Volume 55, Issue 4 - Serial Number 114
December 2025
Pages 707-718

Files

Share

How to cite

Statistics