Ihmisarvioijien näkemyksiä suullisen kielitaidon automaattisesta arvioinnista, digitaalisesta arviointiprosessista sekä puhesuorituksista arvioitavista ulottuvuuksista

Human raters' perceptions of the automated assessment of oral language skills, the digital assessment process and the dimensions to be assessed from speaking performances

Kirjoittajat

• Anna von Zansen Helsingin yliopisto https://orcid.org/0000-0002-6444-7667
• Heini Kallio Jyväskylän yliopisto https://orcid.org/0000-0002-1263-9624
• Milla Sneck Helsingin yliopisto
• Mikko Kuronen Jyväskylän yliopisto https://orcid.org/0000-0001-5971-7063
• Ari Huhta Jyväskylän yliopisto https://orcid.org/0000-0003-3124-449X
• Raili Hildén Helsingin yliopisto https://orcid.org/0000-0002-5114-5600

DOI: https://doi.org/10.30661/afinlavk.114821

Avainsanat:

automaattinen arviointi, kielitaidon arviointi, suullinen kielitaito

Abstrakti

This study investigated human raters’ perceptions of automated assessment of oral language skills. The raters (n = 37) participated in three assessment rounds organized by the DigiTala research project using Moodle and Zoom. The raters assessed Finnish and Swedish learners’ speech samples using one holistic and five analytical rating scales created in the project. After the assessment, the raters responded to a questionnaire that included Likert-scale and open-ended questions. Numerical responses were analyzed with descriptive statistics, open responses with content analysis. The raters think that automated scoring could support human rating. The assessment rounds were carried out successfully. The selected dimensions proved to be essential parts of the speaking performances. The results will benefit those working on automated assessment and oral language assessment.