We explore computational models to understand human driving behavior and devise new approaches to interact with human drivers. Diverse data sources can help us understand how to better interact with the driver. This includes different aspects of the driver, with different facets, from telltale signs about their gaze, situational awareness, and introspection of what their attention is focused on, to predicting and understanding how they drive and behave on the road, to understanding how cars can interact more broadly with the driver.

Research includes predictions on the driver and other road agents, estimations of where the driver is looking and what they know about the road, and approaches for sharing control of the car with the human driver. We use various methods to prototype novel interactions, including computational human-facing studies, hands-on simulation, and more.

Our team is devising approaches to better understand the driver, their state, intent, and characteristics so that we can have models that capture the driver and enable better teaming and interactions with them.

Our solutions incorporate data-driven machine learning (ML) models and cognitive insights that capture essential aspects of the driver’s behavior, such as their situational awareness, state, and intent, along with personal characteristics of their behavior and responses.

Our customers are downstream warning and intervention systems, as well as other approaches for teaming and interacting with the drivers. 

The challenges include the diversity of human behavior, the interaction between human factors, scene content, and the possible AI/vehicle actions, as well as the limited availability of large-scale human-vehicle interaction for critical events.

We are developing several models to better understand the human driver—including where their gaze and attention are directed, what affects their future behavior, and how the car can affect these aspects. We have numerous ML models to solve these challenges, including collaborations and the transfer of our technology to our partners at Woven by Toyota.

We further model driver intent and predict their possible actions as a basis for both safety and performance improvement—including reaction to vehicle intervention, information, AI teaching, and warning approaches.

• MAAD: A Model and Dataset for "Attended Awareness" in Driving

• Unsupervised Multi-View Gaze Representation Learning

• Look Both Ways: Self-supervising Driver Gaze Estimation and Road Scene Saliency | SpringerLink

• Can Pupillometry Be Used to Detect Driver Hazard Awareness

• Uncertainty-Aware Driver Trajectory Prediction at Urban Intersections

• TIP: Task-Informed Motion Prediction for Intelligent Vehicles

• Vehicle Trajectory Prediction Using Generative Adversarial Network With Temporal Logic Syntax Tree Features

• Trajectory Prediction with Linguistic Representations

• Drive Anywhere: Generalizable End-to-end Autonomous Driving with Multi-modal Foundation Models