Do AI Writing Tools Improve L2 Writing Performance? A Meta-Analysis of Experimental Evidence, Moderators, and Methodological Quality

Abstract

Artificial intelligence (AI) writing tools—including large language model (LLM)-based systems, automated writing evaluation (AWE) platforms, and AI-powered writing assistants—have become an increasingly prominent feature of second language (L2) writing pedagogy. Yet whether these tools reliably improve L2 writing performance, and under what instructional conditions, remains an open question. This study reports a systematic meta-analysis of experimental and quasi-experimental evidence on the effects of AI writing tools on L2 writing performance. A multi-database search of Scopus, Web of Science, and ERIC, supplemented by forward citation tracing and targeted hand-searching, identified 34 eligible focal effect sizes from studies published between 2021 and 2026. Effect sizes were expressed as Hedges’ g. A verified subset of 22 effects with directly extractable statistics was estimated using restricted maximum-likelihood (REML) with Knapp–Hartung adjustment; a model (k = 34) incorporating conversion-based effects served as a robustness analysis. The verified model produced a pooled effect of g = 0.714, 95% CI [0.592, 0.836], with low observed heterogeneity (I² = 24.3%). Exploratory moderator analyses indicated that multidimensional feedback designs, sustained interventions (≥ 8 weeks), and university-level contexts were associated with larger effects, while single-session and lower-order-only feedback showed smaller gains. Methodological quality was generally adequate but uneven: many studies lacked delayed transfer measures and transparent effect-size reporting. These findings position AI writing tools as pedagogically contingent rather than inherently effective—their educational value depends critically on how feedback literacy, teacher mediation, and instructional design shape opportunities for noticing, revision, and transferable writing development.