This new study out in Cell reports study began with our striking finding from our 2023 PNAS Nexus paper that when we watch videos, our brains, eyes, pupil and even heart rate synchronize with that of other viewers, but only if we’re paying attention. But is it just the visual dynamics causing all this? or is there an underlying mechanism that can explain these dynamics, we believe its arousal. Listen to the podcast here Publications
This was the starting point for this study! To investigate the top-down effects we wanted to use auditory narratives to get rid of the visual confound of the videos while preserving some engaging content. To investigate any potential bottom-up effects of behavior or sensory input on the brain, we designed control experiments (this part of the study, we will make another blog post soon, for now we will concentrate on the top-down effects)
Our goal was to address three key questions:
- Pupil synchrony – Does pupil size synchronize due to luminance changes or cognitive processing? Prior auditory research suggests that pupil dilation reflects listening effort and emotional engagement, but what about the narrative drives this?
- Eye movements – Without visual stimuli, do subtle, non-traditional eye movements (beyond saccades, smooth pursuit, or vergence) still occur during story listening?
- Neural mechanism – Is there a common neural correlate with pupil size, heart rate and eye movements?
Key Findings
1. Eye movements during story listening
Even without visual input, participants’ eyes continued to move at a slow, synchronized pace, at the same time during the stories. Horizontal and vertical eye movements aligned across listeners, and the overall “envelope” of gaze position, which we call gaze variation fluctuated in unison, mirroring the narrative structure. See videos here
2. Pupil synchrony and attention
Pupil sizes synchronized even as participants stared at a static gray screen while listening to stories. Notably, the degree of pupil synchrony predicted test performance – though this exciting result ended up in the supplementary materials. Pupil dynamics appeared to reflect narrative features such as word onsets and semantic content. See videos here
3. Brain-Body Interaction
Seeking an overarching explanation for these correlations, we hypothesized that arousal fluctuations drive synchronized changes in brain, eye, and heart activity. To test this, we measured correlations between all synchronized signals – gaze variation, pupil size, heart rate – and neural activity. Across participants, we found consistent positive correlations between these physiological signals and front-central EEG patterns, suggesting a shared neural correlate tied to arousal.
Interestingly, we found that all synchronized signals clustered around a low-frequency range (0.1-2 Hz), slower than typical neural or physiological oscillations. This points toward a shared mechanism, possibly linked to the arousal system, known to influence these signals independently.
These findings highlight the remarkable interconnectedness of brain and body during shared experiences. Whether watching videos or listening to stories, our physiological signals synchronize in ways that reflect shared cognitive and emotional engagement.
Here we focused on the top-down part of this study, next we will look at the bottom-up part and the correlated between stimuli features and physiology.