Traffic Deep RL

A project aimed at optimizing traffic-light patterns for improved realistic throughput of intersections.

Description

As a capstone project for CS 490, my team and I worked alongside a professor from Purdue's department of Civil Engineering to optimize traffic patterns with deep-reinforcement learning principles.

Role | Member

As a member of a team of six other developers, I worked to aid the group in data preparation, modeling, evaluations, and general understanding, while documenting reports of our results and processes for further repeatability.

Modeling with Deep RL

In our case, given our use of reinforcement learning, it's important to note how we managed the state and environment. Based on 15 minutes of data from Lankershim boulevard, we created baseline aggregates, like turning volume, to use an a simulation tool known as SUMO -- this would end up being our way of creating a new data pool for our models. Modeling with a focus on deep-reinforcement learning in our case can be broken down into three groups: use of deep-q networks (DQN), proximal policy optimization (PPO), and soft-actor critic (SAC). Each of these models was tested with two value functions to optimize throughout, one with judgement on aggregate vehicle delay, and the other on total vehicle stops. Results for our initial findings can also be found below.

Key Takeaways

As a project heavy in development, understanding, and repeatability, it was important to cycle changes -- for instance, once we've developed a solution, but develop a new understanding of the data, it's crucial to cycle back and reflect the new understanding in the development process. As for repeatability, with things constantly changing, it's necessary to document the entire process to ensure any results can be matched by a reviewing third-party.