AI FOR IELTS WRITING PREPARATION
REVIEWING CHATGPT-GENERATED ANSWER MOCKUPS AND ITS TEXT EVALUATION
Abstrak terlihat: 0 / PDF terunduh: 0DOI:
https://doi.org/10.37147/eltr.v9i2.260Keywords:
ChatGPT, General Training Writing Task 1, Generative AI (GAI) Tools, IELTS, writing mockupAbstract
Generative AI (GAI) tools, such as ChatGPT, are increasingly used as accessible, fast, and consistent alternatives for language proficiency exam preparation, especially for writing tasks. While GAI tools are seen as efficient in providing feedback, little research explored their role in generating answer mockups for IELTS writing tasks and evaluating the mockups they create. This study investigated the use of ChatGPT for IELTS General Training Writing Task 1 preparation by addressing these research questions: “How did ChatGPT generate answer mockups for IELTS General Training Writing Task 1?” and “How did ChatGPT evaluate the answer mockups it created?” This research used qualitative content analysis, analyzing 15 mockups generated for five IELTS writing prompts across score bands 6, 7, and 8. The findings showed that ChatGPT could generate mockups by adhering to IELTS task prompts and criteria in most cases. ChatGPT could also provide detailed evaluations and scores for each writing criterion. However, discrepancies between the generated mockups' scores and the evaluation scores raise questions about the consistency and accuracy of the assessments. The implications of this study are for ChatGPT as a more accessible option for IELTS preparation and as an evaluation on GAI tools’ responses for future development.
Downloads
References
Alberth. (2023). The use of ChatGPT in academic writing: A blessing or a curse in disguise? Teflin Journal, 34(2), 337–352. https://doi.org/10.15639/teflinjournal.v34i2/337-352
Alexopoulou, T., Michel, M., Murakami, A., & Meurers, D. (2017). Task effects on linguistic complexity and accuracy: A large-scale learner corpus analysis employing natural language processing techniques. Language Learning, 67(1), 180–208. https://doi.org/10.1111/lang.12232
Andriani, E., & Bram, B. (2021). Syntactic analysis of sentence patterns and types of BBC news articles. LiNGUA 16(2), 25-34. https://doi.org/10.18860/ling.v16i1.10832
Andriani, E., & Ena, O. T. (2022). Teacher’s use of code-switching types and functions in teaching TOEFL and IELTS preparation classes. Pedagogy: Journal of English Language Teaching, 10(2), 153-168. https://doi.org/10.32332/joelt.v10i2.4379
Ariyanti, A., & Anam, S. (2021). Technology-enhanced paraphrasing tool to improve EFL students’ writing achievement and enjoyment. Journal of English Language Teaching and Linguistics, 6(3), 715-726. https://doi.org/10.21462/jeltl.v6i3.698
Assad, A. (2024). Exploring the use of ChatGPT among media students in Egypt: Opportunities and challenges. Online Journal of Communication and Media Technologies, 14(2), 1-14. https://doi.org/10.30935/ojcmt/14416
Athanassopoulos, S., Manoli, P., Gouvi, M., Lavidas, K., & Komis, V. (2023). The use of ChatGPT as a learning tool to improve foreign language writing in a multilingual and multicultural classroom. Advances in Mobile Learning Educational Research, 3(2), 818–824. https://doi.org/10.25082/amler.2023.02.009
Barrot, J. S. (2023). Using ChatGPT for second language writing: Pitfalls and potentials. Assessing Writing, 57, 1-6. https://doi.org/10.1016/j.asw.2023.100745
Budjalemba, A. S., & Listyani, L. (2020). Factors contributing to students difficulties in academic writing class: Students perceptions. UC Journal: ELT, Linguistics and Literature Journal, 1(2), 135–149. https://doi.org/10.24071/uc.v1i2.2966
Buriak, J. M., Akinwande, D., Artzi, N., Brinker, C. J., Burrows, C., Chan, W. C. W., Chen, C., Chen, X., Chhowalla, M., Chi, L., Chueh, W., Crudden, C. M., Di Carlo, D., Glotzer, S. C., Hersam, M. C., Ho, D., Hu, T. Y., Huang, J., Javey, A., … Ye, J. (2023). Best Practices for Using AI When Writing Scientific Manuscripts. ACS Nano, 17(5), 4091–4093. https://doi.org/10.1021/acsnano.3c01544
Hong, W. C. H. (2023). The impact of ChatGPT on foreign language teaching and learning: Opportunities in education and research. Journal of Educational Technology and Innovation, 5(1), 37-45. https://doi.org/10.61414/jeti.v5i1.103
Creswell, J. W. (2012). Educational research: Planning, conducting, and evaluating quantitative and qualitative research. Pearson Education.
Emran, A. Q., Mohammed, M. N., Saeed, H., Abu Keir, M. Y., Alani, Z. N., & Mohammed Ibrahim, F. (2024). Paraphrasing ChatGPT answers as a tool to enhance university students’ academic writing skills. 2024 ASU International Conference in Emerging Technologies for Sustainability and Intelligent Systems, ICETSIS 2024, 501–505. https://doi.org/10.1109/ICETSIS61505.2024.10459386
Etikan, I. & Bala, K. (2017). Sampling and Sampling Methods. Biometrics & Biostatistics International Journal, 5(6), 215-217. https://doi.org/10.15406/bbij.2017.05.00149
Fitria, T. N. (2021). QuillBot as an online tool: Students’ alternative in paraphrasing and rewriting of English writing. Englisia: Journal of Language, Education, and Humanities, 9(1), 183-196. https://doi.org/10.22373/ej.v9i1.10233
Gayed, J. M., Carlon, M. K. J., Oriola, A. M., & Cross, J. S. (2022). Exploring an AI-based writing assistant’s impact on English language learners. Computers and Education: Artificial Intelligence, 3, 1-7. https://doi.org/10.1016/j.caeai.2022.100055
Ginting, P., Muda Batubara, H., & Hasnah, Y. (2023). Artificial Intelligence Powered Writing Tools as Adaptable Aids for Academic Writing: Insight from EFL College Learners in Writing Final Project. International Journal of Multidisciplinary Research and Analysis, 6(10), 4640-4650. https://doi.org/10.47191/ijmra/v6-i10-15
Huang, A. Y. Q., Lu, O. H. T., & Yang, S. J. H. (2023). Effects of artificial intelligence–enabled personalized recommendations on learners’ learning engagement, motivation, and outcomes in a flipped classroom. Computers & Education, 194, 104684. https://doi.org/10.1016/j.compedu.2022.104684
Imran, M., & Almusharraf, N. (2023). Analyzing the role of ChatGPT as a writing assistant at higher education level: A systematic review of the literature. Contemporary Educational Technology, 15(4), 1-14. https://doi.org/10.30935/cedtech/13605
Jeanjaroonsri, R. (2023). Thai EFL learners’ use and perceptions of mobile technologies for writing. LEARN Journal: Language Education and Acquisition Research Network, 16(1), 169-193. https://so04.tci-thaijo.org/index.php/LEARN/index
Kohnke, L., Moorhouse, B. L., & Zou, D. (2023). ChatGPT for language teaching and learning. RELC Journal, 54(2), 537–550. https://doi.org/10.1177/00336882231162868
Koraishi, O. (2023). Teaching English in the age of AI: Embracing ChatGPT to optimize EFL materials and assessment. LET Journal: Language Education & Tehcnology, 3(1), 55-72. http://langedutech.com
Kurniasari, M. D. (2017). Tolerating structural ambiguity in grammar learning. IJIET (International Journal of Indonesian Education and Teaching), 1(1), 85–94. https://doi.org/10.24071/ijiet.v1i1.330
Kurniati, E. Y., & Fithriani, R. (2022). Post-graduate students’ perceptions of Quillbot utilization in English academic writing class. Journal of English Language Teaching and Linguistics, 7(3), 437-451. https://doi.org/10.21462/jeltl.v7i3.852
Le, C., & Mohd, T. K. (2024). A comparative study of text comprehension in IELTS reading exam using GPT-3. International Journal of Computer Science and Information Technology, 16(1), 45–52. https://doi.org/10.5121/ijcsit.2024.16104
Lichtinger, E. (2018). Gap between self-efficacy and college students’ writing skills. Journal of College Reading and Learning, 48(2), 124–137. https://doi.org/10.1080/10790195.2017.1411213
Lin, M. P.-C., & Chang, D. (2020). A mixed-method classroom study. Journal of Educational Technology & Society, 23(1), 78–92. https://www.jstor.org/stable/26915408
Lin, X. (2023). Exploring the role of ChatGPT as a facilitator for motivating self-directed learning among adult learners. Adult Learning, 35(3), 156–166. https://doi.org/10.1177/10451595231184928
Maghamil, M. C., & Sieras, S. G. (2024). Impact of ChatGPT on the academic writing quality of senior high school students. Journal of English Language Teaching and Applied Linguistics, 6(2), 115-128. https://doi.org/10.32996/jeltal
Maharani, A. A. P., & Santosa, M. H. (2021). The implementation of process approach combined with Screencast-o-matic and Bookcreator to improve students’ argumentative writing. LLT Journal: Journal on Language and Language Teaching, 24(1), 12–22. https://doi.org/10.24071/llt.v24i1.2516
Marzuki, Widiati, U., Rusdin, D., Darwin, & Indrawati, I. (2023). The impact of AI writing tools on the content and organization of students’ writing: EFL teachers’ perspective. Cogent Education, 10(2), 1-17. https://doi.org/10.1080/2331186X.2023.2236469
Meniado, J. (2023). The impact of ChatGPT on English language teaching, learning, and assessment: A rapid review of literature. Arab World English Journal, 14(4), 3–18. https://doi.org/10.24093/awej/vol14no4.1
Mogavi, R., Deng, C., Juho Kim, J., Zhou, P., D. Kwon, Y., Hosny Saleh Metwally, A., Tlili, A., Bassanelli, S., Bucchiarone, A., Gujar, S., Nacke, L. E., & Hui, P. (2024). ChatGPT in education: A blessing or a curse? A qualitative study exploring early adopters’ utilization and perceptions. Computers in Human Behavior: Artificial Humans, 2(1), 1-20. https://doi.org/10.1016/j.chbah.2023.100027
Rahman, B. I. R, Laiya, R. E., & Sarumaha, M. S. (2023). Navigating English writing proficiency tests in the era of artificial intelligence. Journal of English Education and Teaching, 7(3), 480-498. https://doi.org/10.33369/jeet.7.3.480-498
Shaib, C., Elazar, Y., Li, J. J., & Wallace, B. C. (2024). Detection and measurement of syntactic templates in generated text. Computation and Language, 1-16. https://doi.org/10.48550/arXiv.2407.00211
Silva, C. (2017). Research design - The new perspective of research methodology. British Journal of Education, Society & Behavioural Science, 19(2), 1–12. https://doi.org/10.9734/bjesbs/2017/30274
Song, C., & Song, Y. (2023). Enhancing academic writing skills and motivation: assessing the efficacy of ChatGPT in AI-assisted language learning for EFL students. Frontiers in Psychology, 14, 1-14. https://doi.org/10.3389/fpsyg.2023.1260843
Su, Y., Lin, Y., & Lai, C. (2023). Collaborating with ChatGPT in argumentative writing classrooms. Assessing Writing, 57, 100752. https://doi.org/10.1016/j.asw.2023.100752
Sun, T. (2023). The potential use of generative AI in ESL writing assessment: A case study of IELTS writing tasks. Irish Journal of Technology Enhanced Learning, 7(2), 42-51. http://www.ilta.ie/
Susanto, A. I. (2023). ChatGPT vs. human IELTS tutor: A comparative study of IELTS task 2 writing sample. 20th Technology for Second Language Learning Conference. https://apling.engl.iastate.edu/conferences/technology-for-second-language-learning-conference/tsll-2023/abstracts/
Sutrisno, D. B. (2023). Practicing IELTS writing for L2 writers with ChatGPT; An exploratory self-study. English Department of UMMU Journal (EDU Journal, 3(2), 41-55. http://www.jurnal.ummu.ac.id/index.php/edu
Thorp, H. (2023). ChatGPT is fun, but not an author. Science, 379, 313. https://doi.org/10.1126/science.adg7879
Triana, H., Abustang, P. B., Utomo, E., Rakhman, G. G. F., & Fahrurrozi, F. (2023). Assessment evaluation using solo taxonomy for measuring levels of critical thinking skills: PYP international baccalaureate case study. Prisma Sains: Jurnal Pengkajian Ilmu Dan Pembelajaran Matematika Dan IPA IKIP Mataram, 11(2), 581-595. https://doi.org/10.33394/j-ps.v11i2.7894
Utami, S. P. T., Andayani, Winarni, R., & Sumarwati. (2023). Utilization of artificial intelligence technology in an academic writing class: How do Indonesian students perceive? Contemporary Educational Technology, 15(4), 1-13. https://doi.org/10.30935/cedtech/13419
Van Horn, K. R. (2024). ChatGPT in English language learning: Exploring perceptions and promoting autonomy in a university EFL context. TESL-EJ, 28(1), 1-26. https://doi.org/10.55593/ej.28109a8
Vicente-Yagüe-Jara, M. I., López-Martínez, O., Navarro-Navarro, V., & Cuéllar-Santiago, F. (2023). Writing, creativity, and artificial intelligence. ChatGPT in the university context. Comunicar, 31(77), 47–57. https://doi.org/10.3916/C77-2023-04
Wafa, N., Samia, M., Mohamed, H., Amine, E., & Fatiha, K. B. (2024). Citation: Nouari Wafa, et al, (2024) Innovating narratives or stifling creativity? Assessing the impact of generative pre-trained transformer (GPT) on science fiction writing skills among master students in Algeria. Educational Administration: Theory And Practice, 30(6), 1494–1507. https://doi.org/10.53555/kuey.v30i6.5526
Wale, B. D., & Kassahun, Y. F. (2024). The transformative power of ai writing technologies: Enhancing EFL writing instruction through the integrative use of writerly and google docs. Human Behavior and Emerging Technologies, 2024, 1-15. https://doi.org/10.1155/2024/9221377
Wang, C., Li, Z., & Bonk, C. (2024). Understanding self-directed learning in AI-Assisted writing: A mixed methods study of postsecondary learners. Computers and Education: Artificial Intelligence, 6, 1-10. https://doi.org/10.1016/j.caeai.2024.100247
Wang, C., Wang, Y., & Zou, B. (2024). Revolutionising EFL pedagogy: Innovative strategies for integrating GAI (ChatGPT) into language teaching. Journal of Language Teaching, 4(1), 1–7. https://doi.org/10.54475/jlt.2024.004
Wardat, Y., Tashtoush, M. A., AlAli, R., & Jarrah, A. M. (2023). ChatGPT: A revolutionary tool for teaching and learning mathematics. Eurasia Journal of Mathematics, Science and Technology Education, 19(7), 1-18. https://doi.org/10.29333/ejmste/13272
Wilson, J., Myers, M., & Potter, A. (2022). Investigating the promise of automated writing evaluation for supporting formative writing assessment at scale. Assessment in Education: Principles, Policy & Practice, 29, 1–17. https://doi.org/10.1080/0969594X.2022.2025762
Wilson, J., & Roscoe, R. D. (2019). Automated writing evaluation and feedback: Multiple metrics of efficacy. Journal of Educational Computing Research, 58(1), 87–125. https://doi.org/10.1177/0735633119830764
Xiao, Y., & Zhi, Y. (2023). An exploratory study of EFL learners’ use of ChatGPT for language learning tasks: Experience and perceptions. Languages, 8(3), 1-12. https://doi.org/10.3390/languages8030212
Yan, D.(2023). How ChatGPT’s automatic text generation impact on learners in a L2 writing practicum: An exploratory investigation. Education and Information Technologies, 28, 13943–13967. https://doi.org/10.1007/s10639-023-11742-4
Zhang, Z. (2024). ChatGPT: Enhancing learner agency and multilingual practices in English writing class. Journal of Education, Humanities and Social Sciences 26, 1153-1160. https://doi.org/10.54097/2cwdzn44
Zhao, X., Hu, J., & Cox, A. (2024). Incorporating artificial intelligence into student academic writing in higher education: The use of Wordtune by Chinese international students. Proceedings of the 57th Hawaii International Conference on System Sciences, 2726-2735. https://hdl.handle.net/10125/106712
Zhou, D., & Zhang, Y. (2024). Red AI? Inconsistent responses from GPT models on political issues in the US and China. Computation and Language, 1-21. https://doi.org/10.48550/arXiv.2312.09917
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Erina Andriani
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
