Who is laughing now? Laughter-infused dialogue systems

Abstract:

This thesis paves the way for including laughter in spoken dialogue systems in a domain-general and linguistically valid way using computational linguistics tools and methods. The thesis is concerned with three main areas. The first area concerns the placement of laughter in relation to speech and other behaviours. We show that convolutional and recurrent neural networks can effectively predict laughs from dialogue transcripts, whereas human perforance in this task is significantly worse. Such prediction models allow dialogue systems to predict user laughter and, if needed, put system laughter in an appropriate place. Further, we look at laughter placement in relation to gaze and show that laughter performing different pragmatic functions is related to different gaze patterns. These findings provide important implications for embodied conversational agents and social robots in regard to multimodal behaviour realisation and coordination. The second area is concerned with the interaction between laughter and the communicative intent of a user and system, as well as with the context in which the given intent occurs. We lay the groundwork for the central component of a spoken dialogue system by implementing a dialogue manager in a theoretically informed way using a proof-theoretic model based on linear logic. Our dialogue manager is then extended to support laughter functioning as feedback or as a signal accompanying system feedback, and an answer to polar questions. Additionally, we look at how laughter can modify or shape a dialogue act, and how the inclusion of laughter can improve Transformer-based deep learning models in the task of dialogue act recognition. The third area is humour. Even though humour is not necessary for laughter, they are closely related. We look at how humour is related to reasoning about social conventions and other learned and accommodated implicit assumptions. We show how these can be modelled in formally, suggesting one way in which the goal of situational and conversational creativity can be realised in artificial agents.

Research areas:

• IncReD
• Laughter

Year:

2022

Type of Publication:

Phd Thesis