A Multi-Scale Vehicle-Based Traffic Control Architecture via Deep Reinforcement Learning and CAVs Platooning

Connected and Automated Vehicles (CAVs) represent the key enabling technology for reshaping the mobility paradigm and contributing to more efficient, safe, and comfortable ground transportation systems. In this direction, CAVs can be leveraged to develop the vehicle-based control strategies for traffic dynamics regulation and congestion mitigation purposes. To this aim, by leveraging an enriched version of the Cell Transmission Model (CTM), this work proposes a Deep Reinforcement Learning (DRL) multi-scale control architecture, where: i) a Deep Q-Network (DQN) controller, based on the effective traffic conditions, suggests the optimal speed profile and the desired spacing policy to be imposed on CAVs for moving in platoon formation; ii) a distributed PID controller, acting at the microscopic level, aims at properly driving the platoon motion according to the DQN-based traffic control policy. Numerical validations, performed on the real stretch of the A20 freeway in the Netherlands, demonstrate the effectiveness of the proposed multi-scale control architecture in significant reducing the Total Travel Time with improvement of about 18% w.r.t. the uncontrolled traffic flow case.