ESR 12

Jauwairia Nasir

Automatic assessment of engagement during multi-party interactions


I am a Pakistani by origin and I like to consider myself as a person who has passion for research in the field of Artificial Intelligence and applying it to social domains like education and health. Being driven by this passion, as of September 2018, I am working as a PhD student at École polytechnique fédérale de Lausanne (EPFL) in Switzerland as one of the selected Early Stage Researcher (ESR) under the Marie Sklodowska Curie European Training Network ANIMATAS. At EPFL, I am working under the supervision of Professor Pierre Dillenbourg at Computer-Human Interaction in Learning and Instruction (CHILI) lab.


Before this, I worked for a while as a core consultant at an EU based AI healthcare startup. I got my M.S. in Electrical Engineering from Korea Advance Institute of Science and Technology (KAIST) where I was working at Robot Intelligence Technology Lab as a KGSP scholar. I received the Bachelors in Engineering degree in electronics engineering from School of Electrical Engineering and Computer Science at NUST, Islamabad, Pakistan where my interest in robotics surfaced under the supervision of Dr. Yasar Ayaz from RISE lab. I believe in technology for humanity, education for all, women empowerment, discrimination-free world and am trying to build my life around these causes. My hobbies include photography, blogging, travelling, arts, etc.

Main host institution

Ecole Polytechnique Fédérale de Lausanne (EPFL)


Pierre Dillenbourg (EPFL) in association with Mohamed Chetouani (SU) and Ginevra Castellano (UU)

Second institution

Sorbonne Université (SU) ; Üppsala Universitet (UU); Interactive Robotics (IR)


Robots in educational institutions are challenged with socially appropriate interactions with (i) previously unseen users and (2) multi-party interactions with changing numbers of partners. In such situations, advanced models of interpersonal interactions are required to effectively adapt to such complex social situations. Among the multimodal communication skills, the robot has to identify partner(s) to whom it should adapt, follow or either provide help or encouragements. Detecting engagement or disengagement in such situations is challenging and requires to continuously assess the dynamics of interaction, which is usually driven by the characteristics of individuals (role, social traits, social attitude, dominance) and the task to achieve. The concept of interpersonal synchrony will be employed to model group interactions. Using automatic behaviour understanding techniques, we will derive a number of quantifiers to characterize different aspects of synchrony between partners within an interaction as well as of the interaction itself. This could be performed at low-level (e.g., body movement synchrony) and high-level (e.g., contingency of engagement/attitudes in the learning task). This approach will be employed to predict engagement at individual and group levels. The adaptation of the robot will exploit these metrics. Various adaptation strategies will be investigated (robot’s role) and evaluated by a set of machine learning metrics and joint actions metrics achievement. The model will be implemented in educational settings in collaboration with Interactive Robotics. Evaluation will include users’ feedback.

Expected results