Workshop to be held at ESSLLI 2017

July 17th-21st 2017

Toulouse, France

Motivation and description

Natural language use involves drawing information from different sources and fitting it together. For example, to understand the utterance “Take this and this and that and put it there” one has to be able to track the pointing device and be clear about the different referents of the deictic expressions. Similarly, for questions, a syntactic structure and an intonation contour must be aligned. In conversation, phenomena such as split utterances and other-repairs show that several speakers can co-produce single dialogue acts - even using non-standard phonetic, morphological and syntactic components.  

Language is a key component of interaction, and, as work in a variety of fields such as psycholinguistics (e.g. Pickering and Garrod, 2004; 2013) and conversation analysis (CA, see e.g. Schegloff, 2007) has emphasised, an account of interaction is also crucial in the analysis of language.  In the examples above, as with uses of natural language generally, different information - often in different modalities as in a gesture-speech context or in a speech-vision context - must be incorporated and produced or interpreted as and when it is encountered.

This poses challenges for formal approaches to language, which have traditionally abstracted away from the problems presented by the dynamic nature of linguistic interaction. Some researchers have therefore concluded that formalisation is inappropriate as a tool for the analysis of natural language (e.g. Cowley, 2011; Linell, 2009).

However, formal approaches are not just desirable but necessary - not only for a precise understanding of language phenomena, but also in order to enable the development of technology e.g. to meet the demands of language instruction necessary in an increasingly globalised society or to create conversational agents and robots.

Taking interaction seriously means acknowledging the importance of the dynamics in accounts of language. Languages can no longer be conceived of as static systems of individual processes with modules operating independently. This has consequences for the way we think about language at all levels - including phonological, lexical, syntactic, semantic and pragmatic components. Not only must we consider the interaction between modules within an individual, but we must also take into account the changes brought about by the interactions between speakers and communities.

Dynamic approaches are therefore crucial in handling multiparty interaction for example, where patterns of interaction can result in different levels of understanding between different participants in the same conversation (Eshghi and Healey, 2016). Taking interaction dynamics into account is also necessary to explain language change - including diverse phenomena such as diachronic change (Bouzouita, 2008) and semantic adaptation within a conversation (Mills and Healey, 2008). Theories of dynamics in linguistic interaction are also essential for accounting for language learning. This is the case for both first language acquisition in children, where interaction (in the form of e.g. turn-taking) precedes the acquisition of specific words (Bullowa, 1979) and second language learning where previously learned languages affect the learning of a new language (Ellis, 2008).

Formal approaches must model both the different types of information to be individuated and their interactions, setting up the structures algorithmically in a principled manner. The validity of formal mechanisms to relate the different types of information and compute the interactions can be evaluated against corpus data, experimental data or intuitions. Here, simple mappings will not do.  Instead we need dynamic tools such as update rules, joint building of incremental structure or shifting of information to structurally relevant places.

Recent work is beginning to tackle these issues from a formal perspective in a number of disciplines, for example, models of diachronic change (Kempson et al., 2016); speaker-hearer coordination (Howes et al., 2011; Poesio and Rieser, 2010; 2011; Healey et al., 2014); semantic update (Larsson and Cooper 2009; Cooper, 2012); language acquisition (Fernandez et al., 2011; Fernandez and Grimm, 2014); syntax for dialogue (Cann et al., 2005; Gregoromichelaki et al., 2013); information state models of dialogue (Ginzburg, 2012); embodied interaction (Hunter et al., 2015); human-agent interaction (Peltason et al., 2013; Purver et al., 2011; Schlangen, 2016); reasoning (Breitholtz, 2013; Piwek, 2015); the speech-gesture interface (Rieser, 2010; 2015; Lücking et al., 2012; 2015; Healey et al., 2015; Howes et al., 2016) and the semantics of gesture and prosody (Lascarides and Stone, 2006; 2009; Schlöder and Lascarides, 2015).

This workshop aims to bring together researchers working on different formal approaches to the dynamics of interaction to foster cross-disciplinary collaboration around these issues.  We encourage contributions dealing with material from typologically different languages and with different contexts of language use, to address a linguistic public with a variety of interests and working within different paradigms. Due to its formal orientation the workshop will also be relevant to participants with a focus on logic and computation. The organisers have extensive experience in working on dialogue theories, HCI, multimodal corpora and speech-gesture integration.

References

Bouzouita, M. (2008). At the syntax-pragmatics interface: Clitics in the history of Spanish. In Cooper, R., and Kempson, R. (editors) Language in flux: Dialogue coordination, language variation, change and evolution, 221-263 College publications, London.

Breitholtz, E. (2014). Reasoning with topoi–towards a rhetorical approach to non-monotonicity. In Proceedings of the 50th Anniversary Convention of the AISB.

Bullowa, M. (1979). Before speech: The beginning of interpersonal communication. CUP Archive.

Cann, R., Kempson, R., & Marten, L. (2005). The Dynamics of Language: An Introduction. Syntax and Semantics. Volume 35. Academic Press.

Cooper, R. (2012). Type theory and semantics in flux. Handbook of the Philosophy of Science, 14, 271-323.

Cowley, S. J. (Ed.). (2011). Distributed language (Vol. 34). John Benjamins Publishing.

Ellis, N. C. (2008). The dynamics of second language emergence: Cycles of language use, language change, and language acquisition. The Modern Language Journal, 92(2), 232-249.

Eshghi, A., & Healey, P. G. (2016). Collective Contexts in Conversation: Grounding by Proxy. Cognitive science, 40(2), 299-324.

Fernández, R., & Grimm, R. M. (2014). Quantifying categorical and conceptual convergence in child-adult dialogue. In Proceedings of the 36th Annual Conference of the Cognitive Science Society. Austin, TX: Cognitive Science Society.

Fernández, R., Larsson, S., Cooper, R., Ginzburg, J., & Schlangen, D. (2011). Reciprocal learning via dialogue interaction: Challenges and prospects. In Proceedings of IJCAI 2011 Workshop on Agents Learning Interactively from Human Teachers (ALIHT), Barcelona, Spain

Ginzburg, J. (2012). The Interactive Stance. Oxford University Press.

Gregoromichelaki, E., Kempson, R., Howes, C., & Eshghi, A. (2013). On making syntax dynamic: the challenge of compound utterances and the architecture of the grammar. In Wachsmuth et al. (editors) Alignment in Communication. Towards a New Theory of Communication, pp. 57-86, John Benjamins.

Healey, P. G. T., Plant, N., Howes, C., and Lavelle M. (2015). When words fail: Collaborative gestures during clarification dialogues. In AAAI Spring Symposium Series: Turn-Taking and Coordination in Human-Machine Interaction.

Healey, P. G. T., Purver, M., and Howes, C. (2014). Divergence in dialogue. PLoS ONE, 9(6):e98598.

Howes, C., Purver, M., Healey, P. G. T., Mills, G. J., & Gregoromichelaki, E. (2011). Incrementality in dialogue: Evidence from compound contributions. Dialogue and Discourse, 2(1), 279-311.

Howes, C., Lavelle, M., Healey, P. G. T., Hough, J. and McCabe, R. (2016). Helping hands? Gesture and self-repair in schizophrenia. In LREC-2016 Workshop: Resources and processing of linguistic and extra-linguistic data from people with various forms of cognitive/psychiatric impairments (RaPID-2016). Portoroz, Slovenia.

Hunter, J., Asher, N., & Lascarides, A. (2015). Integrating non-linguistic events into discourse structure. In Proceedings of the 11th International Conference on Computational Semantics (IWCS 2015), 184-194.

Kempson, R., Cann, R., Gregoromichelaki, E., & Chatzikyriakidis, S. (2016). Language as mechanisms for interaction. To appear in Theoretical Linguistics.

Larsson, S., & Cooper, R. (2009). Towards a formal view of corrective feedback. In Proceedings of the EACL 2009 Workshop on Cognitive Aspects of Computational Language Acquisition (pp. 1-9). Association for Computational Linguistics.

Lascarides, A. and Stone, M. (2006). Formal semantics of iconic gesture. In Schlangen D. and Fernández, R., (editors), Proceedings of the 10th Workshop on the Semantics and Pragmatics of Dialogue (Brandial), Potsdam. Universitätsverlag Potsdam, pp. 64–71.

Lascarides, A. and Stone, M. (2009). A formal semantic analysis of gesture. Journal of Semantics, 26(4), pp. 393–449.

Linell, P. (2009). Rethinking language, mind, and world dialogically: Interactional and contextual theories of human sense-making. IAP.

Lücking, A., Bergmann, K., Hahn, F., Kopp, S., & Rieser, H. (2012). Data-based analysis of speech and gesture: The Bielefeld speech and gesture alignment corpus (SaGA) and its Applications. Journal on Multimodal User Interfaces 7(1-2), pp. 5-18.

Lücking A., Pfeiffer T., Rieser H. (2015). Pointing and reference reconsidered. Journal of Pragmatics 77, pp. 56-79

Mills, G. J., & Healey, P. G. (2008). Semantic negotiation in dialogue: The mechanisms of alignment. In Proceedings of the 9th SIGdial Workshop on Discourse and Dialogue (pp. 46-53). Association for Computational Linguistics.

Peltason, J., Rieser, H., Wachsmuth, S. (2013). "The hand is no banana!" On communicating natural kind terms to a robot. In Wachsmuth et al (eds.) Alignment in Communication. Towards a New Theory of Communication, pp. 167-193, John Benjamins.

Pickering, M. J., & Garrod, S. (2004). Toward a mechanistic psychology of dialogue. Behavioral and brain sciences, 27(02), 169-190.

Pickering, M. J., & Garrod, S. (2013). An integrated theory of language production and comprehension. Behavioral and Brain Sciences, 36(04), 329-347.

Piwek, P. (2015). Two accounts of ambiguity in a dialogical theory of meaning. In Interactive Meaning Construction A Workshop at IWCS 2015 (p. 19).

Poesio, M. & Rieser, H. (2010). Completions, coordination, and alignment in dialogue. Dialogue and Discourse, 1, 1–89.

Poesio, M., & Rieser, H. (2011). An incremental model of anaphora and reference resolution based on resource situations. Dialogue and Discourse, 2(1), 235-277.

Purver, M., Eshghi, A., & Hough, J. (2011). Incremental semantic construction in a dialogue system. In Proceedings of the Ninth International Conference on Computational Semantics (IWCS 2011) pp. 365-369. Association for Computational Linguistics.

Rieser, H. (2010). On factoring out a gesture typology from the Bielefeld speech-and-gesture-alignment corpus (SAGA). In:Kopp, S., Wachsmuth, I. (editors) Gesture in Embodied Communication and Human-Computer Interaction. Springer, Berlin/Heidelberg

Rieser, H. (2015). When hands talk to mouth. Gesture and speech as autonomous communicating processes. In Howes, C. and Larsson, S., (editors), Proceedings of the 10th Workshop on the Semantics and Pragmatics of Dialogue (goDIAL), Gothenburg, pp. 122-131

Schegloff, E. A. (2007). Sequence organization in interaction: Volume 1: A primer in conversation analysis (Vol. 1). Cambridge University Press.

Schlangen D. (2016) Grounding, justification, adaptation: Towards machines that mean what they say
To appear in: Proceedings of the 20th Workshop on the Semantics and Pragmatics of Dialogue (JerSem).

Schlöder, J. & Lascarides, A. (2015). Interpreting English pitch contours in context. In Howes, C. and Larsson, S., (editors), Proceedings of the 10th Workshop on the Semantics and Pragmatics of Dialogue (goDIAL), Gothenburg, pp. 131-140