Speaker: Aysel Sarıcaoğlu Aygan
Venue: G160
Time & Date: 10:30 – 19.11.2026
GenAI-Based L2 Writing Assessment:
Evidence from Construct-Oriented Qualitative Analyses
Aysel Sarıcaoğlu Aygan
Generative artificial intelligence (GenAI) tools are increasingly used in second language (L2) writing assessment practices — both summative and formative. Several studies have demonstrated the potential of large language models (LLMs) for automated scoring and feedback (e.g., Bui & Barrot, 2024; Mizumoto & Eguchi, 2023; Pfau et al., 2023; Shin & Lee, 2024; Yamashita, 2024); however, existing evidence is derived largely from quantitative comparisons between GenAI-generated scores and human scores and between GenAI-detected errors and human-detected errors. Construct-oriented qualitative evaluations of GenAI-based L2 writing assessment that could increase our understanding of how LLMs analyze the L2 writing construct across specific genres, dimensions, and rhetorical and linguistic features remain considerably limited. Such evidence is crucial for validating the intended uses of GenAI-based assessment and informing decision-making in L2 writing pedagogy.
This presentation provides an overview of qualitative analyses of GenAI-based L2 writing assessment, particularly what LLMs can and cannot identify in L2 writing across various genres, dimensions, and rhetorical and linguistic features. It highlights the importance of evaluating LLMs’ assessment performance in relation to the target construct and genre and discusses empirical evidence from studies on the assessment potential of GPT for argumentative writing (Saricaoglu & Bilki, 2025), cause-and-effect writing (Saricaoglu & Bilki, 2026), and critical reflections (Saricaoglu & Yilmaz, 2025). Together, these studies extend the focus of investigations on GenAI-based L2 writing assessment from scoring reliability and error-detection capacity to (a) the more complex dimensions of L2 writing such as content development and coherence and cohesion, (b) genre-specific rhetorical functions through which meaning is constructed, and (c) the fine-grained linguistic features of written texts. The presentation concludes with theoretical implications and methodological and pedagogical considerations for using GenAI in L2 writing assessment.
Biodata:
Aysel Saricaoglu is an associate professor in the Department of English Language and Literature at the Social Sciences University of Ankara, Türkiye. She is interested in teaching and investigating L2 writing development and assessment, syntactic complexity, automated feedback, research writing, corpus linguistics, and teacher research literacy. Her work has appeared in journals such as Studies on Second Language Acquisition, Annual Review of Applied Linguistics, International Journal of Applied Linguistics, Assessing Writing, Journal of English for Academic Purposes, ReCALL, and CALL. She is currently involved in several national and international research collaborations, language test development projects, and professional development activities for language teachers. She is also co-editing a book on reflective writing for language teachers to be published by Routledge in 2028.