Impacts of connected and autonomous vehicles on deep reinforcement learning controlled intersection systems

Connected and autonomous vehicle (CAV) technologies could significantly change the car-following behaviors and affect the performance of the intersection systems. As it is expected to have a long transition time during which human driven vehicles (HDVs) and CAVs will coexist, it is important to investigate the impacts of CAVs on the intersection systems under different market penetration rates (MPRs). Also, the currently used Highway Capacity Manual does not consider the impacts of CAVs when calculating the intersection capacity. Though highly needed, a new guideline for estimating the intersection capacity under different MPRs of CAVs is becoming a critical issue for transportation planners and engineers. Furthermore, combining the intersection traffic signal control (TSC) systems with deep reinforcement learning (DRL) provides a new potential solution to improve the efficiency, safety, and sustainability of the intersection system. However, the training procedure of the DRL TSC system requires large samples and takes a long time to converge. Furthermore, it is common to have several intersections along corridors or in networks. A single DRL agent is unable to control several intersections as this may result in exponential explosion in the action space. Hence, a modification of the DRL TSC framework to improve the training efficiency and a multi-agent control framework to control several intersections are needed.
To better prepare and guide both intersection planning and operations under different MPRs of CAVs and traffic demands, this dissertation provides an intensive evaluation of the impacts of CAVs in several signal intersection systems, as well as an in-depth analysis on intersection capacity adjustments that consider varying MPRs of CAVs. Also, a transfer-based DRL TSC framework is proposed and tested at different MPRs of CAVs and traffic demand levels. A multi-agent DRL TSC with shared traffic states between downstream and upstream intersections is investigated in a corridor. It is concluded that 100% MPR of CAVs can increase the saturation flow rate of the through-only lane by 126.8%. Meanwhile, transfer-based models could significantly improve training efficiency and model performance. The multi-agent DRL TSC also enables coordination between intersections. The insights of this research should be helpful and valuable to transportation researchers and traffic engineers in calculating intersection capacity, designing intelligent intersections, improving intersection efficiency, and implementing DRL-controlled traffic signals under the mixed flow with CAVs.