Evaluation in social media discourse

A corpus-assisted discourse study of evaluative images of the Covid-19 pandemic on the Finnish Twitter-sphere

Authors

• Jenna Saarni University of Turku https://orcid.org/0000-0003-4363-4911
• Otto Tarkka University of Turku https://orcid.org/0000-0001-8200-0319
• Veronika Laippala University of Turku https://orcid.org/0000-0002-7635-429X

DOI:

https://doi.org/10.61197/fjl.156423

Keywords:

corpus-assisted discourse studies, Covid-19, discourse analysis, evaluation, topic modelling, Twitter

Abstract

The way crises are discussed influences both personal and collective thinking (Seeger & Sellnow 2016). Therefore, it is essential to understand how the global health crisis of Covid-19 was represented on social media where different attitudes and emotions collide. Moreover, better grasping how people perceived the pandemic is vital for managing future health crises and other societal upheavals (Lindholm et al. 2023). In this study, we analyse how various discourse topics, i.e., themes, related to the Covid-19 pandemic were evaluated by Twitter users. We also examine the evaluative images that emerged from these discussions. With evaluation, we refer to expressions of language user’s attitudes and emotions through language (Bednarek 2010). Evaluative images, on the other hand, are created when these discourse topics and evaluations intersect. To tackle the vast amounts of data generated on social media, we employ the combination of the unsupervised machine learning method of topic modelling and the discourse analytical approach of evaluative parameters to study a large corpus of 374,978 Finnish tweets from January 2020 to August 2021. The results of our corpus-assisted discourse analysis reveal 35 distinct discourse topics, encompassing themes such as health, protective measures, briefings and support services. In addition, evaluative expressions of emotivity, mental state, importance and necessity are involved. From this analysis, two primary evaluative images emerge i) a focus on consistent responsibility and emotional reactions and ii) support for the groups most affected by the pandemic. These images reflect the situation’s complexity necessitating profound accountability and support across different layers of society.

References

Bednarek, Monika. 2008. Evaluation in media discourse: Analysis of a newspaper corpus. London and New York: Continuum.

Bednarek, Monika. 2010. Evaluation in the news. A methodological framework for analysing evaluative language in journalism. Australian Journal of Communication 37(2). 15–50.

Bednarek, Monika & Caple, Helen. 2012. News discourse. London: Bloomsbury.

Bednarek, Monika & Ross, Andrew, S. & Boichak, Olga & Doran, Y. J. & Carr, Georgia & Altmann, Eduardo, G. & Alexander, Tristram, J. 2022. Winning the discursive struggle? The impact of a significant environmental crisis event on dominant climate discourses on Twitter. Discourse, Context & Media 45. 1–13. https://doi.org/10.1016/j.dcm.2021.100564

Biber, Douglas & Finegan, Edward. 1989. Styles of stance in English: Lexical and grammatical marking of evidentiality and affect. Text – Interdisciplinary Journal for the Study of Discourse 9(1). 93–124.

Blei, David & Ng, Andrew & Jordan, Michael. 2003. Latent dirichlet allocation. Journal of Machine Learning Research 3. 993–1022.

Eklund, Anton. 2025. Evaluating document clusters through human interpretation. Umeå: Umeå University. (Doctoral dissertation.)

Fiskvik, Jannicke & Vik Bjarkø, Andrea & Grøtan, Tor Olav. 2023. Vaccine rhetoric, social media, and dissensus. An analysis of civic discourse between Norwegian health authorities and citizens on Facebook and Twitter during crisis. In Johansson, Bengt & Ihlen, Øyvind & Lindholm, Jenny & Blach-Ørsten, Mark (eds.), Communicating a pandemic. Crisis management and Covid-19 in the Nordic countries, 241–260. University of Gothenburg: Nordicom. https://doi.org/10.48335/9789188855688-11

Gearhart, Sherice & Kang, Seok. 2014. Social media in television news: The effects of Twitter and Facebook comments on journalism. Electronic News 8(4). 243–259. https://doi.org/10.1177/1931243114567565

Gillings, Mathew & Hardie, Andrew. 2022. The interpretation of topic models for scholarly analysis: An evaluation and critique of current practice. Digital Scholarship in the Humanities 38(2). 530–543. https://doi.org/10.1093/llc/fqac075

Halliday, M. A. K. & Matthiessen, Christian, M. I. M. 2014. Halliday’s introduction to functional grammar (4th ed.). London: Routledge.

Harambam, Jaron & Voss, Ehler. 2023. The corona truth wars: Epistemic disputes and societal conflicts around a pandemic — An introduction to the special issue. Minerva 61. 299–313. https://doi.org/10.1007/s11024-023-09511-1

Hunston, Susan & Thompson, Geoffrey. 2000. Evaluation in text: Authorial stance and the construction of discourse. Oxford: Oxford University Press.

Jacobs, Thomas & Tschötschel, Robin. 2019. Topic models meet discourse analysis: a quantitative tool for a qualitative approach. International Journal of Social Research Methodology 22(5). 469–485. https://doi.org/10.1080/13645579.2019.1576317

Jallinoja, Piia & Vento, Eetu & Väliverronen, Esa. 2024. Trust in scientific institutions and experts during the first 2 years of the COVID-19 pandemic in Finland. Scandinavian Journal of Public Health 53(5), 490–497. https://doi.org/10.1177/14034948241289633

Jallinoja, Piia & Sivelä, Jonas & Väliverronen, Esa. 2021. Valtavirtaa ja vastavirtaa – koronanäkemykset yhteydessä halukkuuteen ottaa koronarokotus. Duodecim 137(19). 2061–2068.

Johansson, Marjut, Kyröläinen, Aki-Juhani, Ginter, Filip, Lehti, Lotta, Krizsán, Attila & Laippala, Veronika. 2018. Opening up #jesuisCharlie anatomy of a Twitter discussion with mixed methods. Journal of Pragmatics 129. 90–101. https://doi.org/10.1016/j.pragma.2018.03.007

Knight, Naomi. 2010. Wrinkling complexity: Concepts of identity and affiliation in humour. In Bednarek, Monika & Martin, James, R. (eds.), New discourse on language: Functional perspectives on multimodality, identity, and affiliation, 35–58. London and New York: Continuum.

Lachlan, Kenneth, A. & Spence, Patric, R. & Lin, Xialing & Del Greco, Maria. 2014. Screaming into the wind: Examining the volume and content of tweets associated with Hurricane Sandy. Communication Studies 65(5). 500–518. https://doi.org/10.1080/10510974.2014.956941

Lehti, Lotta, Luodonpää-Manni, Milla, Jantunen, Jarmo Harri, Kyröläinen, Aki-Juhani, Vesanto, Aleksi & Laippala, Veronika. 2020. Commenting on poverty online: A corpus-assisted discourse study of the Suomi24 forum. SKY Journal of Linguistics 37. 7–47.

Lindholm, Jenny & Carlson, Tom & Albrecht, Frederike & Hermansson, Helena. 2023. Communicating Covid-19 on social media. Analysing the use of Twitter and Instagram by Nordic health authorities and prime ministers. In Johansson, Bengt & Ihlen, Øyvind & Lindholm, Jenny & Blach-Ørsten, Mark (eds.), Communicating a pandemic. Crisis management and Covid-19 in the Nordic countries, 149–172. University of Gothenburg: Nordicom. https://doi.org/10.48335/9789188855688-7

Logi, Lorenzo & Zappavigna, Michele 2021. A social semiotic perspective on emoji: How emoji and language interact to make meaning in digital messages. New Media & Society 25(12). 3222–3246. https://doi.org/10.1177/14614448211032965

Maier, Daniel & Waldherr, Annie & Miltner, Peter & Wiedemann, Gregor & Niekler, Andreas & Keinert, Alexa & Pfetsch, Barbara & Heyer, Gerhard & Reber, Ueli & Häussler, Thomas & Schmid-Petri, Hannah & Adam, Silke. 2018. Applying LDA topic modeling in communication research: Toward a valid and reliable methodology. Communication Methods and Measures 12(2–3). 93–118. https://doi.org/10.1080/19312458.2018.1430754

Martin, Brian. 2021. Covid information struggles. Social Epistemology Review and Reply Collective 10(7).16–26.

Martin, James, R. & White, Peter. 2005. The language of evaluation: Appraisal in English. London: Palgrave Macmillan.

Murakami, Akira & Thompson, Paul & Hunston, Susan & Vajn, Dominik. 2017. What is a corpus about?: Using topic modelling to explore a specialised corpus. Corpora 12(2). 243–277. https://doi.org/10.3366/cor.2017.0118

Mykkänen, Markus & Patron, Anri & Badham, Mark & Valentini, Chiara. 2024. Leveraging NLP for crisis communication management: A case study of news media analysis of the COVID-19 pandemic in two Nordic countries. Journal of Contingencies and Crisis Management 32(3). 1–19. https://doi.org/10.1111/1468-5973.12611

Nguyen, Minh, Van & Lai, Viet, Dac & Veyseh, Amir, Pouran, Ben & Nguyen, Thien, Huu. 2021. Trankit: A light-weight transformer-based toolkit for multilingual natural language processing. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations.

Niemiec, Emilia. 2020. COVID-19 and misinformation: Is censorship of social media a remedy to the spread of medical misinformation? EMBO Reports 21(11).

Partington, Alan & Duguid, Alison & Taylor, Charlotte. 2013. Patterns and meanings in discourse: Theory and practice in corpus-assisted discourse studies (CADS). Amsterdam: John Benjamins.

Pälli, Pekka & Lillqvist, Ella. 2020. Diskurssianalyysi. In Luodonpää-Manni, Milla & Hamunen, Markus & Konstenius, Reetta & Miestamo, Matti & Nikanne, Urpo (eds.), Kielentutkimuksen menetelmiä I–IV, 374–411. Helsinki: Finnish Literature Society. https://doi.org/10.21435/skst.1457

Řehůřek, Radim & Sojka, Petr. 2010. Software framework for topic modelling with large corpora. In Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks.

Saarni, Jenna & Laippala, Veronika. 2024. Health crisis communication in Finnish news media: Evaluative images of the Covid-19 pandemic in digital news headlines. Nordicom Review 45(1). 114–136. https://doi.org/10.2478/nor-2024-0015

Seeger, Matthew & Sellnow, Timothy, L. 2016. Narratives of crisis: Telling stories of ruin and renewal. Stanford: Stanford University Press.

Shir-Raz, Yaffa & Elisha, Ety & Martin, Brian & Ronel, Natti & Guetzkow, Josh. 2023. Censorship and suppression of Covid-19 heterodoxy: Tactics and counter-tactics. Minerva 61. 407–433. https://doi.org/10.1007/s11024-022-09479-4

Smyrnaios, Nikos & Ratinaud, Pierre. 2017. The Charlie Hebdo attacks on Twitter: A comparative analysis of a political controversy in English and French. Social Media + Society 3(1). 1–13. https://doi.org/10.1177/2056305117693647

Spence, Patric, R. & Lachlan, Kenneth, A. & Lin, Xialing & del Greco, Maria. 2015. Variability in Twitter content across the stages of a natural disaster: Implications for crisis communication. Communication Quarterly 63(2). 171–186. https://doi.org/10.1080/01463373.2015.1012219

Spencer, Anthony. 2023. International communication. In Croucher, Stephen, M. & Diers-Lawson, Audra (eds.), Pandemic communication, 215–230. London: Routledge.

Törnberg, Anton & Törnberg, Petter. 2016a. Combining CDA and topic modeling: Analyzing discursive connections between Islamophobia and anti-feminism on an online forum. Discourse & Society 27(4). 401–422. https://doi.org/10.1177/0957926516634546

Törnberg, Anton & Törnberg, Petter. 2016b. Muslims in social media discourse: Combining topic modeling and critical discourse analysis. Discourse, Context & Media 13. 132–142. https://doi.org/10.1016/j.dcm.2016.04.003

van der Meer, Toni, G. L. A. 2016. Automated content analysis and crisis communication research. Public Relations Review 42(5). 952–961. https://doi.org/10.1016/j.pubrev.2016.09.001

van Dijk, Teun A. 1977. Sentence topic and discourse topic. Papers in Slavic Philology 1. 49–61.

van Dijk, Teun A. 1980. Macrostructures: An interdisciplinary study of global structures in discourse, interaction, and cognition. New Jersey: Lawrence Erlbaum Associates.

van Dijk, Teun A. 1997. Introduction: The study of discourse. In van Dijk, Teun (ed.), Discourse Studies: A Multidisciplinary Introduction, 1–7. London: SAGE.

White, Peter. 2002. Appraisal. In Verschueren, Jef & Östman, Jan-Ola & Blommaert, Jan (eds.), Handbook of pragmatics, 1–27. Amsterdam: John Benjamins.

Zappavigna, Michele. 2012. Discourse of Twitter and social media: How we use language to create affiliation on the web. London: Bloomsbury.

Zappavigna, Michele. 2014. Enacting identity in microblogging through ambient affiliation. Discourse & Communication 2. 209–228. https://doi.org/10.1177/1750481313510816

Zappavigna, Michele. 2017a. Evaluation. In Hoffmann, Christian, R. & Bublitz, Wolfram (eds.), Pragmatics of social media, 435–458. Berlin and Boston: De Gruyter Mouton.

Zappavigna, Michele. 2017b. Twitter. In Hoffmann, Christian, R. & Bublitz, Wolfram (eds.), Pragmatics of social media, 201–224. Berlin and Boston: De Gruyter Mouton.

Zappavigna, Michele & Dreyfus, Shoshana. 2022. “In these pandemic times”: The role of temporal meanings in ambient affiliation about COVID-19 on Twitter. Discourse, Context & Media 47. 1–11. https://doi.org/10.1016/j.dcm.2022.100595