When words fail: Collaborative gestures during clarification dialogues

Abstract:

Research on co-speech gestures has primarily focussed on speakers. However, in conversation non-speaking addressees can also gesture. Often this is to provide concurrent feedback such as backchannels but sometimes it involves gestures that relate to the specific content of the speaker's turn. We hypothesise that non-speakers should contribute most actively during clarification sequences i.e. at the moments when mutual-understanding is threatened. We test this hypothesis using a corpus of story-telling dialogues, captured using video and motion capture. The results show that during clarification sequences speaker and non-speaker behaviours tend to merge. Non-speakers in particular move their hands faster and produce more than twice as many content-specific specific gestures in overlap with speaker turns. These results underline the collaborative nature of conversation, the strategic importance of non-verbal resources for sustaining mutual-understanding and the critical role of clarification and repair in successful communication.

Research areas:

• Dialogue
• Repair
• Gesture

Year:

2015

Type of Publication:

In Proceedings

Book title:

AAAI Spring Symposium Series: Turn-Taking and Coordination in Human-Machine Interaction

Month:

March