Human-Piloted Drone Racing: Visual Processing and Control

Abstract

Humans race drones faster and more agile than algorithms, despite being limited to a fixed camera angle, body rate control, and response latencies in the order of hundreds of milliseconds. A better understanding of the ability of human pilots of selecting appropriate motor commands from highly dynamic visual information may provide key insights for solving current challenges in vision-based autonomous navigation. The aim of this study was to investigate the relationship between flight performance, control behavior, and eye movements of human pilots in a drone racing task. We collected a multimodal dataset from 21 experienced drone pilots using a highly realistic drone racing simulator, also used to recruit professional pilots. Our results showed task-specific improvements in drone racing performance over time. Gaze fixations not only tracked future waypoints but also anticipated the future flight path. Cross-correlation analysis showed a strong spatio-temporal relationship between eye movements, camera orienting behavior, and thrust vector control. These results highlight the importance of coordinated eye movements in human-piloted drone racing.

Publication
IEEE Robotics and Automation Letters