In social robots, visual attention expresses awareness of the scenario components and dynamics. As in humans, their attention should be driven by a combination of different attention mechanisms. In this paper, we introduce multi-modal saliency maps, i.e. spatial representations of saliency that dynamically integrate multiple attention sources depending on the context. We provide the mathematical formulation of the model and an open-source software implementation. Finally, we present an initial exploration of its potential in social interaction scenarios with humans, and evaluate its implementation.

Abstract The topic of how humans navigate using vision has been studied for decades. Research has identified the emergent patterns of retinal optic flow from gaze behavior may play an essential role in human curvilinear locomotion. However, the link towards control has been poorly understood. Lately, it has been shown that human locomotor behavior is corrective, formed from intermittent decisions and responses. A simulated virtual reality experiment was conducted where fourteen participants drove through a texture-rich simplistic road environment with left and right curve bends. The goal was to investigate how human intermittent lateral control can be associated with the retinal optic flow-based cues and vehicular heading as sources of information. This work reconstructs dense retinal optic flow using a numerical estimation of optic flow with measured gaze behavior. By combining retinal optic flow with the drivable lane surface, a cross-correlational relation to intermittent steering behavior could be observed. In addition, a novel method of identifying constituent ballistic correction using particle swarm optimization was demonstrated to analyze the incremental correction-based behavior. Through time delay analysis, our results show a human response time of approximately 0.14 s for retinal optic flow-based cues and 0.44 s for heading-based cues, measured from stimulus onset to steering correction onset. These response times were further delayed by 0.17 s when the vehicle-fixed steering wheel was visibly removed. In contrast to classical continuous control strategies, our findings support and argue for the intermittency property in human neuromuscular control of muscle synergies, through the principle of satisficing behavior: to only actuate when there is a perceived need for it. This is aligned with the human sustained sensorimotor model, which uses readily available information and internal models to produce informed responses through evidence accumulation to initiate appropriate ballistic correction, even amidst another correction.

Abstract Social communication between humans and robots has become critical as a result of the integration of robots into our daily lives as assistants. There is a need to explore how users intuitively understand the behavior of a robot and the impact of social context on that understanding. This study measures mental effort (as indexed by pupil response) and processing time, measured as the time taken to provide the correct answer, to investigate participants’ intuitive understanding of the robot’s gestures. Thirty-two participants participated in a charades game with a TIAGo robot, during which their eyes were tracked. Our findings show a relationship between mental effort and processing time, and indicate that robot gestures, congruence of speech and behavior, and the correctness of interpreting robot behavior influence intuitive understanding. Furthermore, we found that people focused on the robot’s limb movement. Using these findings, we can highlight what features contribute to the intuitive interaction with a robot, thus improving its efficiency.

Refractive errors are among the most common visual impairments globally, yet their diagnosis often relies on active user participation and clinical oversight. This study explores a passive method for estimating refractive power using two eye movement recording techniques: electrooculography (EOG) and video-based eye tracking. Using a publicly available dataset recorded under varying diopter conditions, we trained Long Short-Term Memory (LSTM) models to classify refractive power from unimodal (EOG or eye tracking) and multimodal configuration. We assess performance in both subject-dependent and subject-independent settings to evaluate model personalization and generalizability across individuals. Results show that the multimodal model consistently outperforms unimodal models, achieving the highest average accuracy in both settings: 96.207\% in the subject-dependent scenario and 8.882\% in the subject-independent scenario. However, generalization remains limited, with classification accuracy only marginally above chance in the subject-independent evaluations. Statistical comparisons in the subject-dependent setting confirmed that the multimodal model significantly outperformed the EOG and eye-tracking models. However, no statistically significant differences were found in the subject-independent setting. Our findings demonstrate both the potential and current limitations of eye movement data-based refractive error estimation, contributing to the development of continuous, non-invasive screening methods using EOG signals and eye-tracking data.

Driver gaze estimation is important for various driver gaze applications such as building advanced driving assistance systems and understanding driver gaze behavior. Gaze estimation in terms of gaze zone classification requires large-scale labeled data for supervised machine learning and deep learning-based models. In this study, we collected a driver gaze dataset and annotated it using three annotation approaches – manual annotation, Speak2Label, and moving pointer-based annotation. Moving pointer-based annotation was introduced as a new data annotation approach inspired by screen-based gaze data collection. For each data collection approach, ground truth labels were obtained using an eye tracker. The proposed moving pointer-based approach was found to achieve higher accuracy compared to the other two approaches. Due to the lower accuracy of manual annotation and the Speak2Label method, we performed a detailed analysis of these two annotation approaches to understand the reasons for the misclassification. A confusion matrix was also plotted to compare the manually assigned gaze labels with the ground truth labels. This was followed by misclassification analysis, two-sample t-test-based analysis to understand if head pose and pupil position of driver influence the misclassification by the annotators. In Speak2Label, misclassification was observed due to a lag between the speech and gaze time series, which can be visualized in the graph and cross-correlation analysis were done to compute the maximum lag between the two time series. Finally, we created a benchmark Eye Tracker-based Driver Gaze Dataset (ET-DGaze) that consists of the driver’s face images and corresponding gaze labels obtained from the eye tracker.

Workplace accidents and illnesses affect millions globally, underscoring the urgent need for effective strategies to design safer and healthier built environments. This study investigated the impact of lighting conditions, a critical environmental factor, on cognitive function and psychological well-being, focusing on workload, fatigue, attention, and stress. In a simulated work environment, participants followed a task protocol under two lighting conditions: 500 lux and 300 lux. Objective data were collected using Pupil Labs Core eye-tracking glasses, complemented by subjective self-reports via questionnaires. The findings revealed that 500 lux lighting with a lower colour temperature reduced fatigue, alleviated eye strain, and enhanced attention, demonstrating the role of proper lighting in promoting cognitive function and well-being. Conversely, the 300 lux condition led to increased fatigue and greater pupil constriction, highlighting potential negative effects of insufficient illuminance in workplace environments. Objective measures, such as pupil dilation, provided consistent and reliable insights compared to subjective self-reports, emphasizing the advantages of advanced eye-tracking technology in assessing environmental factors. The study also highlighted the limitations of subjective methods, which are susceptible to individual interpretation. These results underline the importance of integrating optimal lighting systems into building designs to improve worker productivity, mental health, and overall environmental quality.

Eye tracking is a tool that is widely used in scientific research, enabling the acquisition of precise and detailed data on an individual’s eye movements during interaction with visual stimuli, thus offering a rich source of information on visual perception and associated cognitive processes. In this work, a new software called SPEED (labScoc Processing and Extraction of Eye tracking Data) is presented to process data acquired by Pupil Lab Neon (Pupil Labs, Berlin, Germany). The software is written in Python which helps researchers with the feature extraction step without any coding skills. This work also presents a pilot study in which five healthy subjects were included in research investigating oculomotor correlates during MDMT (Moral Decision-Making Task) and testing possible autonomic predictors of participants’ performance. A statistically significant difference was observed in reaction times and in the number of blinks made during the choice between the conditions of the personal and impersonal dilemma.

Embryo transfer is a critical step of in vitro fertilization, the most effective treatment for infertility experienced by one in six people in their lifetime. To date, despite advances in optimizing embryo quality, an important variability of pregnancy rate remains between practitioners. In order to evaluate the key technical skills that might affect such behavioural differences, we conducted a preliminary multi-centric study on assisted reproductive technologies (ART) specialists using a Gynos Virtamed simulator for ultrasound guided embryo transfer (UGET) combined with a portable eyetracker (Neon, Pupil labs). Our first analyses demonstrate the capability of a recent portable eyetracker in tracking fine eye movements in an ecological (head unrestrained, dim light condition) embryo transfer condition. A dedicated processing pipeline was developed and gaze were analyzed on Areas of Interest (AoI) consisting of the ultrasound image, the uterine model (A, C or E) or the catheter. A separate analysis of the fixated anatomical subregions of the ultrasound image was also conducted. Preliminary analyses show two distinctive patterns of eye movements during UGET: a target based behaviour or a switching and tool following behaviour, suggesting more pro-active gaze behaviour in experts, in agreement with the literature on other image guided interventions.

In this work, an experiment was designed in which a defined route consisting of country roads, highways, and urban roads was driven by 20 subjects during the day and at night. The test vehicle was equipped with GPS and a camera, and the subject wore head-mounted eye-tracking glasses to record gaze. Gaze distributions for country roads, highways, urban roads, and specific urban roads were then calculated and compared. The day/night comparisons showed that the horizontal fixation distribution of the subjects was wider during the day than at night over the whole test distance. When the distributions were divided into urban roads, country roads, and motorways, the difference was also seen in each road environment. For the vertical distribution, no clear differences between day and night can be seen for country roads or urban roads. In the case of the highway, the vertical dispersion is significantly lower, so the gaze is more focused. On highways and urban roads there is a tendency for the gaze to be lowered. The differentiation between a residential road and a main road in the city made it clear that gaze behavior differs significantly depending on the urban area. For example, the residential road led to a broader gaze behavior, as the sides of the street were scanned much more often in order to detect potential hazards lurking between parked cars at an early stage. This paper highlights the contradictory results of eye-tracking research and shows that it is not advisable to define a holy grail of gaze distribution for all environments. Gaze is highly situational and context-dependent, and generalized gaze distributions should not be used to design lighting functions. The research highlights the importance of an adaptive light distribution that adapts to the traffic situation and the environment, always providing good visibility for the driver and allowing a natural gaze behavior.

Quiet eye training, a technique focused on optimizing gaze behavior during critical moments, has shown potential for enhancing motor skill acquisition. This study investigates the effects of quiet eye training in both virtual and real-world environments on dart-throwing learning. The participants consisted of 45 female students who were randomly divided into three groups: a control group (age: M  = 22.46 ± 2.89), real-world (age: M  = 23.80 ± 2.75), and virtual quiet eye training groups (age: M  = 24.33 ± 2.25). The training sessions spanned 2 days, with each session consisting of 60 dart throws divided into 20 blocks of three trials each. The virtual group used an Xbox Kinect motion sensor to throw virtual darts, while the real-world group threw real darts at a dartboard. Both experimental groups followed specific visual training protocols. The control group, on the other hand, threw real darts at a dartboard without receiving any visual training. Results showed that both experimental groups enhanced QE duration, but only the real-world group significantly improved throwing accuracy. These results highlight the importance of sensory information specific to the task in motor learning, supporting the specificity of practice hypothesis.