Traffic Signal Timing Optimization at Intersections Using an Improved Q-Learning Algorithm

Yuanshuai LAN, Chuan LI, Min LIAO, Ting GUO

Abstract

In order to address the limitations of the conventional traffic signal control methods with regard to adapting to dynamic traffic scenarios and the challenges associated with multi-objective collaborative optimization, this study proposes a Firefly-guided Hybrid Deep Q-Network (FH-DQN) algorithm for optimizing the intersection signal timing. The proposed framework synergistically integrates firefly algorithm-based swarm intelligence with deep reinforcement learning, employing a brightness-driven hierarchical exploration strategy for enhancing the action selection efficiency. A multi-objective dynamic reward function incorporating vehicle delay, queue length, carbon emissions, and traffic throughput is developed in order to achieve a balanced optimization of traffic efficiency and environmental sustainability. The algorithm architecture includes the following innovative components: a dual-network collaborative structure that enhances the vehicle responsiveness to sudden congestion at individual intersections, an adaptive brightness update rule derived from the firefly algorithm-based optimization principles and the multi-objective dynamic reward function which achieves the dynamic adjustment of the intersection traffic signals. The extensive experiments conducted on the Simulation of Urban Mobility (SUMO) platform demonstrate that the FH-DQN algorithm achieves a superior performance in comparison with the fixed-time and conventional DQN approaches in typical cross-intersection scenarios. Specifically, the average queue waiting time for vehicles when applying the proposed method is reduced by 26.09% in comparison with the DQN-based approach. The ablation experiment confirms the individual contributions of the dynamic reward function and global network components to enhancing the overall performance of the FH-DQN algorithm. To sum up, this study provides a novel framework for collaborative traffic signal optimization in complex urban road networks, significantly improving both the adaptive capability and multi-objective optimization performance of intelligent transportation systems. Future work could focus on adapting the algorithm to metropolitan-scale networks and on integrating multimodal traffic data for enhancing the operational robustness of the proposed model.

Keywords

Intersection Traffic Lights Timing, Q-Learning Algorithm, Reinforcement Learning, Dynamic Reward Function, Firefly Algorithm, Regional Cooperative Control.

View full article