New paper in PNAS: Synchronized eye movements predict test scores in online video education

I am immensely proud that almost 3 years of work has now been published in PNAS At the center of this paper is that we can measure if people are paying attention online by using web cameras to track peoples eye movements. Before you say “that sounds creepy”, then we can actually do this without sending any information about you to our server, thus preserving privacy. We can even predict how well students would do on a test, based on the material presented in the video. We did this with over 1000 students using their own webcams to track their eye movements.

So where did this all start? Ill tell you the journey we went through.

Just as Wikipedia has become a go-to source of information, so has instructional videos often found on YouTube. Everything from TED talks to videos about why stars are star-shaped? Online education has been growing rapidly, where you can take everything from fully fledged college degrees, to short courses on everything from language to knitting. When the NSF funded project I was hired in at the City College of New York started 3 years ago, we wanted to look in to online education. Specifically we wanted to look in to how this online format keeps students engaged, or maybe the lack of it, as the retention rates of students is staggeringly low. So how can we improve online learning? We believe one of the key challenged in education is the student teacher interaction. A good teacher can gauge whether students are lost or simply just not paying attention. Teachers react to this by changing teaching styles or give students some tasks that will hopefully engage students again. Some people are better at this than others of course, I remember too many super boring teachers, where sleeping might have been the only way to get through it (don’t tell anyone) but I stayed awake of course 😉

So how do we build tools that enable us to measure the level of attention of students online, that hopefully will enable both teachers and online education providers the possibility to react and make online education adaptive, just as a good teacher does in the classroom?

Working in the department of Biomedical Engineering at CCNY, in the Neural Engineering group for Prof. Lucas Parra, our go-to tools stems from Neuroscience. In the past we showed that the Inter-Subject Correlation (ISC) of EEG could actually measure the level of attention of students and in fact predict the students test scores based on the short videos they watched. But having an EEG set on every time you want to go to “class”, might not be so practical. So we looked for inspiration from what teacher do in a classroom. Teachers often have an easy time seeing whether they have lost their students. They simply look at their students eyes. If you are looking somewhere else, on your phone, out the window or at some of the other cute student in the class, then they are probably not paying attention to the teacher. If you are not paying attention, then you probably wont remember much of what the teacher was saying.

With this knowledge we did a first study and measured students eye movements as they watched 5 different short informal instructional videos, often found on YouTube. To then test whether or not students eye movements actually changed when they weren’t paying attention, we had the students watch the videos again, but this time we asked them to count backwards in decrements of 7 from a high prime number between 800 and 1000. The reason for the range was simply to remove any learning effect, as the students had to restart the counting backwards every time they watched a new video for the second time.

Attending condition
Distracted condition

(more to come)

New paper: Music synchronizes brainwaves across listeners with strong effects of repetition, familiarity and training

Our new paper on the effect of repetition in music is finally out. This has been a great joy to work with Lucas Parra and Elizabeth Margulis, merging the knowledge from music cognition, neuroscience and data science.

As a musical piece is repeated the listener’s “neural engagement” decreases for pieced that are composed in a familiar style, but pieces of unfamiliar style keeps the listener engaged. This effect is most pronounced for listeners that have some musical training. Musicians also seem to be significantly more engaged with the classical pieces than with non-musicians.

New paper “Neural engagement with online educational videos predicts learning performance for individual students”

Our paper “Neural engagement with online educational videos predicts learning performance for individual students” is out in “Neurobiology of Learning and Memory” Volume 155, November 2018, Pages 60-64 you can read the paper here

This is an exciting result which shows that with higher neural engagement, when watching educational videos, students perform better on questionnaires related to the video content, presented after the video. Neural engagement is measured as the Inter Subject Correlation of EEG signals recorded while students watch educational videos within the topic of Science, Technology, Engineering and Mathematics (STEM). This is a very intuitive result, if students pay attention to the videos, they also do better on the test.

Future research will look in to easier and more mobile methods to assess engagement in online education. Also investigating if our results generalize to other types of educational material and teaching styles. Stay tuned for updates.

1 Year in New York working in Parra lab at the City College of New York

In late 2017 i took the jump, from working in safe little Denmark to work in New York city at the City College of New York. From a lab that was focused on machine learning and modelling to a lab that goes from question, designing an experiment, collecting data, modelling and answering those questions (full cycle). Which has been a welcomed challenge.
I have been warmly welcomed in the lab by Prof. Lucas Parra and all the people working in the lab. It has been a big change, both in working environment (things are different in the US) but also the new field i am working in. From worrying about features, models etc. to now doing Neural Engineering working with EEG, ECG and eye tracking.

I have been hired in the NSF funded project “Assessing student attentional engagement from brain activity during STEM instruction”. Here we are exploring how to assess both the efficacy of different teaching styles in online education and different physiological measures to assess the attentional engagement of students. I have been hired for a 3-year postdoc and i am looking forward to the challenge. I will keep you updated with progress 🙂

Talk at From cognition to recommendation

I have been invited to talk at the “festival”, 9th of March @Vega in Copenhagen, Denmark. The program indicates that its going to be 20+ variants of Machine Learning!!

To break up the stream of “how deep is your network” i will talk a little about how we define the question that we want to solve using ML. Specifically i will explore the different both evolutionary and cognitive psychological mechanisms we can target to make better music services.
See you there 🙂

Article in “Alt om Data”

Last year i did an interview with Palle Vibe that works for the magazine “Alt om Data”, here are the efforts (its in danish)

It is always interesting to see how different people and journalist interpret your research. There is always a new angle and perspective, which makes it worth while and interesting to do these interviews 🙂

(click on the image to read the article)

Alt om Data – Nu har COMPUTEREN følelse for MUSIKKEN