Language and Logic Courses
Facial Displays and Their Dialogical Meanings,
Jonathan Ginzburg (Paris Diderot University, France) and Ye Tian (Amazon Research Cambridge, UK & Laboratoire de Linguistique Formelle, Université Paris Diderot, France)
Week 2, 9:00 – 10:30,
Room 267 moved to Mirror Hall, Floor 2
The course aims at modelling the discourse impact of non-verbal social signals (nvss) such as laughter, smiling, frowning, and the like, but also including conventionalised pictorial representations (“emoji”) as textual manifestations. We will start with a brief overview of the two main theoretical sources for this course: first, work in computational and theoretical linguistics on the semantics and pragmatics of dialogue and second, psychological and computational approaches to emotion appraisal. The course will offer detailed argumentation, contrary to received wisdom until recently, as to the mutual interaction between nvss/emojis and content emanating from verbal material, in particular that nvss/emojis bear propositional content. We propose viewing nvss/emojis as akin to an event anaphors and show how to deduce various pragmatic functions from the basic meanings posited in combinination with enthymematic reasoning and emotional appraisal. The course will also include practical sessions on multidimensional laughter classification of speech using Praat and emoji data analysis in social media.
Lecture 1: Previous work on laughter and humour; motivation for a semantic/pragmatic approach; the KoS framework for dialogue. ginzburg-tian-esslli-2018-lek1
- Darwin, C. (1877). The expression of the emotions in man and animals. Oxford University Press, USA. With an introduction and notes by Paul Ekman.
- Morreall, J. (2016). Philosophy of humor. In Zalta, E. N. (Ed.), The Stanford Encyclopedia of Philosophy (Winter 2016 edition). Metaphysics Research Lab, Stanford University.
- Attardo, S. (2008). A primer for the linguistics of humor. The primer of humor research, 8, 101–157.
Spatharas, D. (2006). Persuasive ΓeΛΩΣ: public speaking and the use of laughter. Mnemosyne, 374–387.
- Cooper, R. & Ginzburg, J. (2015). Type theory with records for NL semantics. In Fox, C. & Lappin, S. (Eds.), Handbook of Contemporary Semantic Theory, Second Edition, pp. 375–407 Oxford. Blackwell.
Lecture 2: the KoS dialogue framework (cont); A minimal semantic/pragmatic theory of laughter. http://esslli2018.folli.info/wp-content/uploads/ginzburg-tian-fsm-essli18-lek2-1.pdf
1. Ginzburg, J. (2016). The semantics of dialogue. In M. Aloni and P. Dekker (Eds.), The Cambridge Handbook of Formal Semantics. Cambridge: Cambridge University Press.
2. Breitholtz, E. & Cooper, R. (2011). Enthymemes as rhetorical resources. In Artstein, R., Core, M., DeVault, D., Georgila, K., Kaiser, E., & Stent, A. (Eds.), SemDial 2011 (Los Angelogue): Proceedings of the 15th Workshop on the Semantics and Pragmatics of Dialogue.
Lecture 3: A minimal semantic/pragmatic theory of laughter. (Cont.) Classifying laughter in corpora using Praat.
Lecture 4: Cognitive theories of emotion; Incorporating cognitive theories of emotion into KoS; A formal analysis of emojis.
- Scherer, K. R. (2009), ‘The dynamic architecture of emotion: Evidence for the component process model’, Cognition and emotion 23(7), 1307–1351.
Gratch, J. & Marsella, S. (2004). Technical details of a domain-independent framework for modeling emotion. ICT tech report.
3. Herring, S., Stein, D., & Virtanen, T. (Eds.). (2013). Pragmatics of computer-mediated communication (Vol. 9). Walter de Gruyter.
4. Dresner, E., & Herring, S. C. (2010). Functions of the nonverbal in CMC: Emoticons and illocutionary force. Communication theory, 20(3), 249-268.
5. Novak, P. K., Smailović, J., Sluban, B., & Mozetič, I. (2015). Sentiment of emojis. PloS one, 10(12), e0144296.
6. Miller, H., Thebault-Spieker, J., Chang, S., Johnson, I., Terveen, L., & Hecht, B. (2016). Blissfully happy” or “ready to fight”: Varying Interpretations of Emoji. Proceedings of ICWSM, 2016.
- Lecture 5: Analyzing emojis in social media; A formal theory of non-verbal social signals in dialogue: laughter, smiling, eye rolling, frowning, sighing; Incorporating non-verbal social signals into the grammar.
Slides for part 1: http://esslli2018.folli.info/wp-content/uploads/201808_ESSLLI_emojis.pdf
Slides for part 2: http://esslli2018.folli.info/wp-content/uploads/fsm-esslli-lek5.pdf
Goodwin, M. H. & Alim, H. S. (2010). “whatever (neck roll, eye roll, teeth suck)”: the situated coproduction of social categories and identities through stancetaking and transmodal stylization. Journal of Linguistic Anthropology, 20(1), 179–194.
Niedenthal, P. M., Mermillod, M., Maringer, M., & Hess, U. (2010). The simulation of smiles (sims) model: embodied simulation and the meaning of facial expression. Behavioral and brain sciences, 33(6), 417–433.
4. Ginzburg, J. & Poesio, M. (2016). Grammar is a system that characterizes talk in interaction. Frontiers in Psychology, 7, 1938.