Prologi

Pääkirjoitus

Florence Oloff, Joonas Råman — Wed, 09 Oct 2024 00:00:00 +0300

Editorial

‘Am I saying it wrong?’ Progressivity-related troubles and instructional opportunities in child-robot L2 interaction

Wed, 09 Oct 2024 00:00:00 +0300

Participants’ orientation to progressivity of action sequences is a fundamental feature of human social interaction, but less is known about how progressivity is maintained in human-robot interaction (HRI). We explore this by drawing on c. 14 hours of video recordings showing small groups of primary school children interacting with Nao, a programmable humanoid robot. Facilitated by a teacher, the children in our data are completing a short robot-assisted language learning lesson aimed at training English vocabulary and oral skills at a Swedish-speaking school in Finland. We investigate how the teacher and the pupils address emerging troubles in a word repetition sequence which the robot is programmed to carry out with one pupil at a time. Our analysis focuses on two kinds of troubles related to sequence closure: the robot’s so-called ‘third’ turns that either 1) do not ratify the pupil’s just-prior word repetition as ‘correct’ or 2) are (treated as) incongruent within the sequential context. We show how the human participants make sense of such conduct, recruit the teacher’s assistance to secure closure of the activity sequence, and orient to pronunciation instruction in situated ways. The results shed light on how children accommodate to, and are socialised into, human-robot interaction.

Digital technology in physiotherapy consultations: Problem-solving sequences and recruitments

Sara Keel, Anja Schmid, Fabienne Keller — Wed, 09 Oct 2024 00:00:00 +0300

This contribution investigates consultations in which a physiotherapist compiles an exercise program on their computer that the patient can then use at home via a mobile application. It offers an analysis of moments in which physiotherapists encounter problems in locating a specific exercise and of the ways the interactants then achieve a solution. In day-to-day physiotherapy practice, problems with digital technology during the use of health applications or desktop computers occur often. Solving them is experienced as time-consuming and might cause perceived disruptions to workflows and interactions among professionals or between physiotherapists and patients. Adopting an Ethnomethodology and Conversation Analysis (EMCA) approach, our contribution is based on recordings of real-life consultations. It tackles the ways in which problems with a mobile health application (hereafter: the app) are treated in situ and from a members’ point of view. Our analysis reveals that identifying and solving problems with the app involve recruitments, i.e., methods through which seeking or volunteering assistance and/or cooperation is achieved. More specifically, it shows that depending on the moments and the ways recruitments are deployed and organized in physiotherapist-patient interaction, solving problems with the app during consultations creates opportunities for patient participation and thus cooperation between physiotherapists and patients.

Non players’ embodied practices of engagement in videogaming

Heike Baldauf-Quilliatre, Isabel Colón de Carvajal — Wed, 09 Oct 2024 00:00:00 +0300

Using technology in interaction means actively manipulating a technological device while interacting with others, but it can also refer to a situation where one person is employing a technological device in the presence of others and therefore in a potentially interactive situation. This is, for instance, the case when one acts on a screen in the presence of other physically co-present participants. Our paper deals with a particular situation of co-presence and technology use: a couple is sitting side by side on the sofa in the living room, one is playing a single-player adventure game on a large TV screen, the other is playing another game on a tablet. For about 40 years, the concept of “active spectators” developed in media sciences has highlighted the fact that spectators “do” something. But, few studies have dealt with the interactional practices used to display this “activeness”. This paper explores the construction of spectatorship with a special focus on spectating videogaming as a particular way of using technology in the sense of spectating at what is being done with a technological device. We propose a micro-analysis of the above-mentioned gaming-situation in order to show how the non-player’s engagement in the player’s gaming is co-constructed by both participants. A particular focus was placed on gaze and gaze shifts as a resource to display togetherness and potential availability.

Making new technology understandable through multimodal instruction: A digital mobility stick in customer training interaction

Tiina Räisänen, Niina Hynninen — Wed, 09 Oct 2024 00:00:00 +0300

With a variety of smart technologies on the market, health technologies have become an increasingly everyday phenomenon. This paper focuses on customer training in the use of a new technological device, a digital mobility stick, which is an exercise stick with a built-in haptic component. It can be used as a measuring and training tool to analyze the body’s ability to balance, bend, and rotate and to guide the trajectory of one’s exercise movements. We focus on the mobility stick as a training tool, investigating how the haptic technology is introduced to the customer in instructional interaction, and what roles the technology obtains in the process. The paper uses video-recorded customer training interaction data from a health technology company. It draws on multimodal conversation analytic research on instructions and instructed actions, objects, technologies, and touch in interaction. We show that the company representative’s specific orientation to the mobility stick was consequential to the instructed actions of the customers and their learning. The analyzed cases also illustrate that the mobility stick gained different roles in the interaction, ranging from a technological and sensorial object to a technology representing information to an active participant in interaction guiding human action. The study thus contributes to our understanding of the multimodality of instructional interaction and the potentially varied roles of technology in such interaction.

Digitaalisesta fyysiseen ja imitoituun – Uuden mobiilisovelluksen vaihtuvat roolit käyttöönottokoulutuksissa

Liisa Kääntä — Wed, 09 Oct 2024 00:00:00 +0300

Tutkimuksessa tarkastellaan multimodaaliseen keskustelunanalyysiin nojaten kouluttajien toimintaa, jolla he esittelevät työpaikan uutta mobiilisovellusta käyttöönottokoulutuksissa. Koulutustilanteet on tallennettu yhdestä suomalaisesta terveydenhuoltoalan organisaatiosta. Tutkimuksessa fokusoidaan kouluttajien multimodaaliseen vuorojen tuottamiseen eli siihen, miten he orientoituvat mobiilisovellukseen sekä fyysisenä laitteena että digitaalisena sovelluksena ja miten he tuottavat vuoronsa tätä kaksijakoisuutta silmällä pitäen. Mobiilisovelluksen esittelyä analysoitiin puheen ja kehollis-digitaalisen toiminnan yhteistyönä. Kouluttajat osoittivat mobiilisovellukselle erilaisia rooleja: siihen orientoiduttiin koulutuksen välineenä, koulutettavana objektina ja kanssatoimijana. Kaikki nämä roolit tukevat aikaisemmassa tutkimuksessa tehtyjä havaintoja. Kouluttajat orientoituivat mobiilisovellukseen myös imitoitavana elementtinä, mikä tuo uutta näkökulmaa teknologialle osoitettuihin rooleihin työelämän vuorovaikutuksen ja viestinnän tutkimuksessa. Tulosten perusteella teknologioilla voidaan nähdä olevan erilaisia materiaalisia muotoja, joita tehdään vuorovaikutuksessa näkyväksi. Vuorovaikutuksessa onkin mahdollista puhua ja digitaalis-kehollisesti todentaa näkyväksi teknologiaa, joka ei hahmotu selvästi tai jota on kuviteltava, jotta sitä voidaan tehdä tutuksi käyttäjille ja osoittaa heille sen hyödyllisyyttä. Lisäksi tutkimuksen avulla painotetaan ihmisten ja teknologioiden yhteistyön tärkeyttä, mikä on olennaista työelämän digitaaliseen murrokseen sopeutumisessa.

Bridging physical and virtual ecologies of action: giving and following instructions in co-located VR-gaming sessions

Margarethe Olbertz-Siitonen, Arja Piirainen-Marsh — Wed, 09 Oct 2024 00:00:00 +0300

Using multimodal conversation analysis, this study examines how communicative practices are adapted to the affordances for interaction in co-located encounters where participants are getting acquainted with VR-technology and games. The analysis focuses on instructional activities and investigates how an expert player guides others in learning how to handle VR equipment and getting to know the game mechanics during the initial moments of starting a new game.

The data comprises video-recordings of informal social gatherings of 3-4 young adults who take turns in trying out different VR games. The gaming situations were organized in a temporary game lab using consumer-grade VR equipment, a large screen that displayed video feed from the game console and loudspeakers for the game sound.

The analysis demonstrates how the experienced player who has no direct agency over the virtual world uses verbal and carefully placed tactile means to help novice players (who in turn have restricted access to the material world) navigate the initial stages of entering a game. Their complying actions, on the other hand, are characterized by finely tuned bodily adjustments in interplay with affordances of the technology. The instruction sequences, therefore, represent interactional moments in which the participants mutually attend to asymmetries, orienting to bridging physical and virtual ecologies of action.

Environmentally coupled gestures as a communicative resource in the word explanation activity: A multimodal analysis of interaction in social VR

Heidi Spets — Wed, 09 Oct 2024 00:00:00 +0300

This article contributes to our understanding of how participants use different resources to accomplish word explanations in social virtual reality (VR). The article draws on conversation analysis to examine audio-visual data of interaction on the Rec Room VR platform. A view of the physical space the participants inhabit has also been captured. There are twelve participants, and they have minimal experience with social VR. English is used as a lingua franca. The focus is on participants’ use of environmentally coupled gestures (EnCGs) during word explanation activity. The activity has two or more participants playing a word-guessing game, in which one participant explains a word using drawings and gesture as well as speech. Findings show that EnCGs that feature elements in the environment are more readily interpretable than EnCGs that feature elements over the avatar body. The latter can result in situations in which achieving the goal of a word explanation activity (correct guess) can be difficult. In addition, the explainer’s orientation to their physical body and the recipient’s orientation to the virtual body during the joint word explanation activity can create situations in which the gestures become difficult to interpret for the recipient. To conclude, the observations in this article reveal the importance of the alignment of virtual and physical gestures for the intelligibility of gesture in VR.