The rapid advancement of automated vehicles means the responsibility of the driving task is shifting toward the automated driving system, with the human driver becoming a ‘fallback’. This poses a safety conundrum, as it can entice drivers to perform non-driving related tasks which can hinder their ability to take over control of the vehicle if required. It is crucial for the driver to monitor the vehicle and road to ensure they are ready to take over.

Our Human Factors staff collaborated on a research project to evaluate how different head-up display designs can help the driver stay aware of the vehicle and road while performing a non-driving related task during automated driving. This blog provides a summary of the findings in the following paper:

Michael A. Gerber, Ronald Schroeter, Daniel Johnson, Christian P. Janssen, Andry Rakotonirainy, Jonny Kuo, and Mike G. Lenné. 2024. An Eye Gaze Heatmap Analysis of Uncertainty Head-Up Display Designs for Conditional Automated Driving. In CHI ’24, May 11–16, 2024, Honolulu, HI, USA. ACM, New York, NY, USA, 16 pages. https://doi.org/10.1145/3613904.3642219

The rapid advancement of automated vehicles means the driving task – including monitoring the driving situation, making decisions, and controlling the vehicle – is transitioning from being the sole responsibility of the human driver toward a shared responsibility with the automated driving system. This shift takes many of the vehicle control tasks out of the hands of the human, yet still requires drivers to monitor the automation and act as a backup when it fails or reaches limitations.

In the Society of Automotive Engineers (SAE) driving automation taxonomy, Level 3 (L3) – also called conditional automation – largely frees the driver from supervision duties. SAE L3 driving systems are defined as capable of performing the complete driving task within the intended domain of operation and should request the human driver to act as a ‘fallback’ and take control when needed.

However, this definition poses a safety conundrum, as without responsibility for driving-related tasks, humans are enabled and enticed to perform non-driving related tasks (NDRAs). Even when NDRAs are not allowed, humans tend to still engage in them, as we currently see with mobile phone use.

These NDRAs can degrade a driver’s performance when taking over control due to a lack of situational awareness, having potential safety implications. The driver should therefore be kept “in the loop” – at least until driving automation technology improves to the point where no safety-critical transitions are required. As of today, this technology does not exist for vehicles which operate on public roads. This means that it is crucial for drivers to maintain situational awareness. The best way for drivers to do this is by directing sufficient visual attention to the forward roadway, monitoring the automation, and generally remaining engaged enough to notice any vehicle-related failures, should they occur.

This study evaluated different interventions designed to help the driver maintain situational awareness while attending to an NDRA on a head-up display during automated driving. These interventions build on research that highlights the benefit of sharing the “uncertainty” of the automated driving system (the automation’s reliability or confidence in managing the current driving situation) to the driver while they engage in an NDRA. Specifically, the study explores how different uncertainty design intervention concepts a) influence the eye-gaze behaviour of drivers while engaging in a NDRA, and b) impact usability.

Using a high-fidelity driving simulator, drivers watched entertaining videos (i.e., an NDRA) while the researchers compared three different uncertainty intervention designs:

1. Continuous display – a bar visualisation of uncertainty placed close to the video
2. Interruption – strategically pausing the video during uncertain situations to notify about uncertainty
3. Combination – a combination of both

The study found that interruptions led participants to divide their attention between monitoring the driving environment and entertainment, as opposed to focusing excessively on the NDRA. This improvement to monitoring behaviour was more pronounced in the combination intervention compared to interruption alone, suggesting that pre-warning interruptions have positive effects. Continuous display only (without interruptions) had negative effects, whereby drivers appeared to be distracted by the display but ignored the perceived “suggestion” to check their surroundings.

Furthermore, the different interventions did not significantly impact what participants remembered about the video (NDRA), implying their viewing experience was not overly compromised. There were also no differences between ratings of usability for the different interventions.

Intermittent interruptions may have safety benefits over placing additional peripheral displays without compromising usability.

The research concluded that uncertainty visualisation (display only) in combination with everyday NDRAs may reduce driving-related attention and lead to misjudgment of the situation. Furthermore, it was discovered that interruption-based interventions lead to task-interleaving between NDRA and monitoring the road. Combining the interruptions with an uncertainty visualisation seems to be a promising way to maintain a driver’s situational awareness, resulting in more frequent and voluntary interruptions of the NDRA. This heightened situational awareness could result in improved takeover readiness and safety performance, however further research is needed. Understanding the modes of interaction that promote driver situational awareness has implications for vehicle design and demonstrates the benefits of driver monitoring systems in supporting drivers’ attention on the road.