Using Driver Gaze and On-Road Driving Data for Predicting Driver Maneuvers in Advanced Driving Assistance Systems
Abstract
With over 90\% of car crashes attributed to human errors, the focus on drivers as active participants in Advanced Driving Assistance Systems (ADAS) feedback mechanisms becomes crucial. The primary focus of our research is to model driver behavior for ADAS by utilizing gaze-based information in correlation with other vehicle sensor data with the aim of predicting driver maneuvers and assessing the utility of various types of information toward applying such models in driver assistance systems. The research utilized an instrumented vehicle equipped with diverse sensors, emphasizing the importance of calibrating these sensors to establish a common reference frame. The research focuses on three key objectives: mapping the driver's gaze to other sensors, creating a dataset of driving maneuvers and employing driver gaze and vehicle sensor data to develop predictive models. The methodology involves implementing robust cross-calibration algorithms to map the driver's gaze to the 3D vehicle surroundings. We generated a dataset of driving maneuvers by utilizing the instrumented vehicle to collect real driving data from different drivers, aligning different sensor outputs in spatio-temporal correspondence. This dataset serves as the foundation for developing predictive models of driver maneuvers. The study then employs two deep neural network models, Long Short-Term Memory (LSTM) and Gated Recurrent Units (GRU), to effectively handle temporal data and assess the impact of the driver's gaze in correlation with vehicle dynamics to predict the next driver's maneuvers. Our experimental results underscore the effectiveness of the proposed approach, showing that leveraging the driver's gaze data in correlation with vehicle dynamics data can improve prediction performance. The proposed approaches and findings in this thesis pave the way for the development of intelligent co-driver systems for on-road vehicles, ultimately leading to safer and more efficient transportation systems.