About MuMo Stance

Multimodal Stancetaking: Expressive Movement and Affective Stance:

Political Debates in the German Bundestag and the Polish Sejm

Research Project Objectives. The project advances a new framework to multimodal stancetaking. Initially developed as a theoretical and analytical concept for the analysis of metaphorical meaning-making in audiovisual media and in face-to-face interaction (Kappelhoff & Müller 2011; Müller & Kappelhoff 2018), expressive movement is the pathway we are offering to explain how stancetaking as a multimodal activity becomes a temporally-structured affective experience. By integrating ‘objective’ measures of speech prosody with the analysis of gesture (Karpiński et al. 2008; Karpiński 2009; Karpiński & Jarmołowicz-Nowikow 2010; Malisz & Karpiński 2010; Czoska et al. 2015; Karpiński i & Klessa 2018), body posture, head movements and speech on micro-, meso-, and macro levels of temporal unfolding of political speeches, a new angle is provided on the analysis of multimodal stancetaking as ‘expressive movement’. Despite the abundant and diverse literature on stance and stancetaking the role of gesture and body movement as well as prosodic properties of this communicative activity remain hitherto only scarcely studied.The project thus fills a gap and responds to the challenge of including multimodality in stancetaking research across different disciplinary fields. It addresses stancetaking as multimodal embodied process and investigates its realization in political debates in German and Polish parliaments. The project focuses on dynamic gestural patterns along with body-postures, head movements, and concomitant speech, including its prosodic properties, as ‘multimodal’ expressive movements (Kappelhoff & Müller 2011).
Methodology and added value of bilateral cooperation. The study will be based on a collection of recordings of political speeches. Stancetaking will be analyzed on the level of ‘expressive movement units’ (Kappelhoff & Bakels 2011; Müller & Kappelhoff 2018). Conceived as ‘multidimensional experiential gestalt’ that modulates and grounds processes of meaning-making affectively, Plessner’s notion of ‘expressive movement’ serves as theoretical and methodological reference point for the analysis of stancetaking. Expressive movement units will be tagged and independently analyzed for gestural and prosodic form as well as for co-occurring speech. A tight integration of the expertises of the two teams is central to the project. While the Polish team takes the lead in the speech prosody analyses, the German team leads with regard to linguistic gesture analysis and the theoretical and methodological aspects of the key concept of ‘expressive movement’. The two types of expertise are integrated to capture the dynamic complexity of multimodal forms and patterns of stancetaking as affective forms, i.e., as multimodal ‘expressive movements’ and their orchestration along the debates in the German Bundestag and the Polish Sejm. The complete debates will be transcribed, annotated, and submitted to analyses with regard to the three foci of the project: (1) segmentation of expressive movement units, (2) analysis of expressive forms and patterns with high levels of affectivity, (3) multimodal orchestration of affective stancetaking in terms of (a) affective stance (b) interaffective alignment.
Expected impact. The project extends the state of the art theoretically, methodologically, and concerning the research subject. It breaks new ground by: (1) offering a new approach to multimodal stancetaking as expressive movement, (2) complementing existing research on multimodal stancetaking by including prosody and gesture-speech synchronization, and by (3) investigating multimodal stancetaking in political debates across two languages (German and Polish). The project thus enhances our understanding of the multimodal complexity of stance as a ‘doing’ that rests upon the orchestration of “overt communicative means – speech, gesture, and other forms of symbolic action” (Du Bois 2007: 169). It provides the basis for future comparative research of multimodal stancetaking, be it across cultures, discourse genres, or different forms of political debates, and will offer open access to results and methodology.
Added value of the international cooperation. The collaboration between the Viadrina Gesture Center and the DiaGest Group (AMU) brings together two unique expert teams. Both teams investigate multimodal communication from a linguistic point of view. The Viadrina Group specializes in co-speech gesture, emergence and stabilization of meaning in gestures (recurrent gestures), as well as the experiential dynamics of metaphorical meaning in speech, gestures, and audiovisual media. The DiaGest Group on the other hand, specializes in a multimodal approach to „language in use”, both in interpersonal and human-machine communication. By integrating the analysis of gesture, speech, body-posture, and head-movements with prosodic parameters that are relatively well explored in terms of their relation to emotions or attitudes and of prosody-gesture synchronization, the project offers a unique perspective on stancetaking as multimodal expressive movement.

Project funded by

German Research Foundation (DFG)

National Science Fundation (NCN 2018/31/G/HS2/03633)