Open Access

Anke Fischer-Janzen, Thomas Wendt, Kristof Van Laerhoven

• Controlling robots through eye-tracking enables people with severe physical disabilities to perform everyday tasks independently. Interpreting the intent of users from their gaze alone, considering the presence of multiple objects and diverse tasks, poses a significant challenge that influences the design of this control modality. The objective of error-free object or task selection, whichControlling robots through eye-tracking enables people with severe physical disabilities to perform everyday tasks independently. Interpreting the intent of users from their gaze alone, considering the presence of multiple objects and diverse tasks, poses a significant challenge that influences the design of this control modality. The objective of error-free object or task selection, which enhances user safety, stands in contrast to the improved usability resulting from increased robot autonomy. This paper presents a task selection method that employs easily comprehensible pictograms as fiducial markers for robot control. The pictograms represent eight daily tasks that can be performed with a robotic arm. We present a pilot dataset that contains 1,008 in-the-wild images. We evaluated the performance of a YOLOv12n model trained on this dataset through a stress test to identify shortcomings in its real-world performance. The comprehensibility of the pictograms was evaluated in an online survey, resulting in an average correct response rate of 83%. Based on these results, we discuss the model’s usability, accessibility, scalability, and robustness in the context of the anticipated application. Our method provides a robust and scalable way to select daily tasks that can be performed in domestic and public environments.…