Automated & Cooperative Driving

AI for automated vehicles, from deep learning to large language and vision-language models, tested on a full-size vehicle.

The lab’s wrapped automated vehicle on the LTI test track beside a no-passing-zone sign

Why it matters

Automated vehicles can only deliver on safety and efficiency if they perceive the road reliably, predict what surrounding drivers will do, and make decisions that hold up in mixed traffic. Each of those steps breaks in conditions that matter to agencies: weak GPS, occluded lane markings, dense merges, and roads that were never designed for automated driving.

What we do

We develop the perception, prediction, and control algorithms behind automated and cooperative driving, with AI at the core: deep learning for localization and lane topology, reinforcement learning for control, and foundation models, including large language models (LLMs) and vision-language models (VLMs), for scene understanding and driving decisions. We are also building agentic AI that plans and cooperates toward goals. We keep analytical models where they give guarantees that learning alone cannot.

What sets the work apart is the test loop. Algorithms move from our CARLA–SUMO co-simulation platform to a full-size autonomous vehicle and then to the Larson Transportation Institute test track, one of 10 USDOT-designated Automated Vehicle Proving Grounds.

We also study the other side of deployment: how roads, pavement, and policy have to change for automated vehicles to operate at scale.

The forward camera and sensor mounts behind the windshield of the automated vehicle — Sensors behind the windshield.

Rear-seat displays showing the automation software — Rear-seat displays run the automation stack.

A test run on the LTI test track with a safety driver and an operator — A test run on the LTI test track.

Lab members in the driver and passenger seats of the vehicle — Ph.D. students on a test day.

What we work on

Perception and localization

Algorithms that let a vehicle know where it is and what the road looks like when GPS is weak and lanes are occluded: LiDAR-based cooperative localization between a leader and a follower vehicle, LiDAR SLAM for truck automation, monocular visual odometry, infrastructure-assisted cooperative perception, all-weather LiDAR segmentation, and lane graph prediction that stays reliable under occlusion.

Lightweight LiDAR-Based Cooperative Localization Model for Asymmetric Leader-Follower Cooperative Driving Automation System (IEEE T-ITS, 2026)
Confidence-Gated Topology Reasoning with Fiducial Marker Priors for Occlusion-Robust Lane Graph Prediction (ICRA 2026, 2026)
Infrastructure Sensor-Based Cooperative Perception for Early Stage Connected and Automated Vehicle Deployment (JITS, 2024)

Prediction, behavior, and scenario generation

Models of how human-driven and automated vehicles actually move, so that planners, digital twins, and test scenarios behave like real traffic: online lane-change inference, stochastic lateral wandering in mixed traffic, behavior-consistent digital twin trajectories, and collision-constrained generation of safety-critical scenarios.

CCFM: Collision-Constrained Flow Matching for Safety-Critical Scenario Generation (ECCV 2026, 2026)
Online Inference of Lane Changing Events for Connected and Automated Vehicle Applications with Analytical Logistic Diffusion Stochastic Differential Equation (TR Part C, 2022)
Behavioral Investigation of Stochastic Lateral Wandering Patterns in Mixed Traffic Flow (TR Part C, 2023)

Planning, control, and AI decision-making

Control methods that range from analytical to learning-based: freeway merging control with state-dependent discharge rates, deep reinforcement learning for longitudinal trajectories, Monte Carlo tree search at arterial intersections, platooning, and simulation-based verification of lane keeping assistance. Current projects use vision-language models (VLMs) to turn camera input into driving actions and agentic AI built on large language models (LLMs) for goal-directed, cooperative decision-making.

Analytical Formulation of Autonomous Vehicle Freeway Merging Control with State-Dependent Discharge Rates (IEEE T-ITS, 2025)
Online Longitudinal Trajectory Planning for Connected and Autonomous Vehicles in Mixed Traffic Flow with Deep Reinforcement Learning Approach (JITS, 2023)
A Monte Carlo Tree Search-Based Mixed Traffic Flow Control Algorithm for Arterial Intersection (TRR, 2020)

Infrastructure and policy for automation

What roads and rules need to change before automated vehicles operate at scale: how automated driving changes pavement roughness and long-term condition, AI-based pavement condition prediction, the PennDOT Act 130 assessment of CAV impacts on Pennsylvania, and an analysis of U.S. automated vehicle policy.

Highly Automated Vehicle Policy in the United States: Six Questions Exposing the Structural Misalignment Between Perception, Regulation, and Enforcement (TR Today, 2026)
A Machine Learning-Based Approach to Assess Impacts of Autonomous Vehicles on Pavement Roughness (Phil. Trans. R. Soc. A, 2023)
Bridging Multi-Level Vehicle Behavioural Patterns with Long-Term Pavement Condition: A Bidirectional Modelling Approach for Progressively Mixed Autonomous Traffic Flow (Transportmetrica B, 2024)

Videos

Penn State Student Researchers Demo Self-Driving Vehicle with President Neeli Bendapudi

Video: Penn State

Autonomous vehicle demo with real-time LiDAR and camera data

CARMA cooperative driving automation demonstration

Waypoint following on the LTI test track

Drive-by-wire, speed, and steering control demonstration

CARMA cooperative driving automation stack tested on the Penn State campus

Representative work

Confidence-Gated Topology Reasoning with Fiducial Marker Priors for Occlusion-Robust Lane Graph Prediction
Zirui Wu, Xianbiao Hu* (2026). IEEE International Conference on Robotics and Automation (ICRA).
Predicts the lane graph an automated vehicle needs for planning, and keeps the prediction reliable when lanes are occluded by gating topology reasoning on confidence and using fiducial markers as priors.
CCFM: Collision-Constrained Flow Matching for Safety-Critical Scenario Generation
K. Li, K. Liang, Yuxin Ding, Debojyoti Biswas, Xianbiao Hu, Ruwen Qin (2026). European Conference on Computer Vision (ECCV).
Generates safety-critical driving scenarios with flow matching under collision constraints, for testing automated driving systems.
Lightweight LiDAR-Based Cooperative Localization Model for Asymmetric Leader-Follower Cooperative Driving Automation System
Yuxin Ding, Chenxi Chen, Tianjia Yang, Xianbiao Hu* (2026). IEEE Transactions on Intelligent Transportation Systems, 27, 1650–1665.
A lightweight LiDAR-based model that lets the follower vehicle localize itself relative to the leader in an asymmetric leader-follower system, reducing reliance on GPS.
DOI
Analytical Formulation of Autonomous Vehicle Freeway Merging Control with State-Dependent Discharge Rates
Qing Tang, Xianbiao Hu* (2025). IEEE Transactions on Intelligent Transportation Systems, 26, 19935–19948.
An analytical formulation of autonomous vehicle merging control at freeway on-ramps, with discharge rates that change with the traffic state.
DOI
Highly Automated Vehicle Policy in the United States: Six Questions Exposing the Structural Misalignment Between Perception, Regulation, and Enforcement
Xianbiao Hu* (2026). Transportation Research Today.
Examines U.S. policy for highly automated vehicles through six questions and shows where public perception, regulation, and enforcement are structurally misaligned.
DOI Plain-language summary

Work with us

We work with agencies and companies that want to test automated driving algorithms on a real vehicle and a closed test track, or need an independent evaluation of how automation affects their roads.

Email Dr. Hu