Advancing intuitive human-machine interaction with human-like social capabilities for education in schools




Europe Flag

This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 765955

Identifiez-vous pour accéder à la partie privée du site ou le modifier. Déconnexion.

ESR11 - Disfluencies and teaching strategies in social interactions between a pedagogical agent and a student

Early-Stage Researcher: Tanvi Dinkar

Hi, I’m Tanvi. One day, I took a linguistics class on a whim during my undergraduate in journalism and literature. This turned into a serious interest which lead me to do my Master of Science in linguistics from the University of Edinburgh. After this, I worked as a research assistant in ASR, and I realised that I had to push myself to learn more about computational linguistics. I then pursued a Master of Science in speech and language processing from Edinburgh. From there I joined Nuance as a language engineer to work on task-oriented dialogue systems. Luckily, working on dialogue let me work on a bit of everything. While the knowledge that I gained from the industry is invaluable; I knew that I wanted to pursue research. Things come full circle, and I hope that by working on spoken language understanding and HRI in my PhD, I will learn something more about human communication. In my spare time, I vicariously experience dramatic situations through reading, and love a good Hitchcock film."


Main host institution: IMT

Supervisor:  Chloé Clavel (IMT), in association with Catherine Pelachaud (UPMC)

Secondment institution: UPMC; JacobsUni

Objectives: Disfluencies (fillers, repetitions, auto-corrections, etc.) are spontaneous speech phenomena that can be linked to various factors such as speaker’s emotions, or speaker’s feeling of knowing. Studies have shown they can be markers of student’s stress, feeling of incompetence. Despite the potential of disfluencies to provide information on user’s emotion and mental state, the consideration of disfluencies in human-agent context has been so far rarely explored. This ESR project proposes to analyze the different types of disfluencies occurring in the student’s speech utterances during his/her interaction with the other students, a teacher or a pedagogical agent. This typology will be dedicated to develop models able to manage disfluencies within agent’s social interaction strategies and agent’s teaching strategy. Acoustic and linguistic realizations of the different types of disfluencies and their link with the accompanying gestures will be studied with a focus on disfluencies that can give information about the student’s feeling of competence and emotional state. The computational model will formalize the teaching strategies and the social interaction strategies and when to trigger them in order to help the student in his/her learning phase. Concretely, objectives include: O1: Pluridisciplinary review of literature ranging from sociolinguistic corpus studies to human-machine interaction on: i) disfluency studies; ii) teaching strategies (M8)


Expected results: Completed PhD dissertation; implementation of agent’s teaching strategy in the VIB platform for human-agent interaction, peer-reviewed publications in Interspeech, ACL, ACII, SIGDIAL, IVA, AAMAS, IEEE TAC, IJHCS 


For further information, contact:  Chloé Clavel

Deadline: December 15th, 2017

ANIMATAS – MSCA – ITN – 2017 - 765955 2