Transit Signal Priority (TSP) is a traffic signal control strategy that can provide priority to transit vehicles and thus improve transit service. However, this control strategy generally causes adverse effects on other traffic, which limits its widespread adoption. The development of Connected Vehicle (CV) technology enables the real-time acquisition of fine-grained traffic information, providing more comprehensive data for the optimization of traffic signals. Simultaneously, optimization algorithms in the field of TSP have been advancing at a rapid pace. Artificial intelligent (AI)-powered techniques, such as Deep Reinforcement Learning (DRL), have become promising approaches for addressing TSP problems recently. In this study, we developed adaptive TSP control frameworks for both isolated intersection scenarios and multiple intersection scenarios, assuming the implementation of CV technology. Leveraging the comprehensive traffic data obtained from CVs, our frameworks employ both single-agent DRL and multi-agent DRL techniques to address optimization problems. The controllers, based on our proposed frameworks, were tested in simulation environments and compared with various widely used traffic signal controllers across different scenarios, demonstrating superior performance.