Introduction
This work was inspired by the observation that most research on eye gaze and speech has focused on behavior in unnatural situations (e.g., faces speaking single words). Furthermore, typical measures of eye gaze behavior that had been applied in previous research (e.g., fixation duration, saccade rate) only described a behavior averaged over listeners and over time. To be able to use eye gaze for audiological applications, we need methods that access a more nuanced and detailed description.
This work has been partially supported by the Swedish Research Council (Vetenskapsrådet, VR 2017-460 06092 418 Mekanismer och behandling vid åldersrelaterad hörselnedsättning).
Aims
This project aims at generating knowledge that would guide how eye gaze can be used in audiological applications. Potential applications included both the possibility for eye gaze as a control signal in hearing aids but also as a measure of listeners’ experience in real-world settings.
The purpose of this project is (1) to generate a basic understanding of how listeners use their gaze in realistic conversations, and (2) to gain experience generating and analyzing this type of data.
Methodology
We collected data from hearing-impaired participants while they followed a pre-recorded audio-visual conversation with two talkers. Participants carried out this task under different noise levels while wearing an eye-tracker to monitor their gaze.
We studied eye gaze behavior under different conversational turn taking categories using multi-level logistic regression.
Results
This project is still on-going. Here are some of our preliminary findings:
- Most of the variance in gaze behavior is explained from the differences between single conversational instances. This means that if you have a homogenous population subject to the same conversational video under the same noise conditions they will behave similarly.
- We also found out that listeners start looking at a new talker in a conversational turn 200 ms after they start talking. Considering the time it takes to plan and execute a saccade, this is initial evidence of a prediction effect.
This figure illustrates the probably (log odds) of the listener’s gaze to the new talker in a conversational turn (floor transfer) over time. We can observe that listeners start looking at a new talker 200 ms after they start talking
