EXPLORING THE EFFECTIVENESS OF CHATGPT AS A CONVERSATIONAL PARTNER ON EFL STUDENT’S SPEAKING SKILL: A QUASI-EXPERIMENTAL STUDY IN INDONESIA HIGHER EDUCATION

Zalfa Cabrina Maharani; Bambang Eko Siagiyanto; Refai Refai

doi:10.24127/pj.v14i3.12338

EXPLORING THE EFFECTIVENESS OF CHATGPT AS A CONVERSATIONAL PARTNER ON EFL STUDENT’S SPEAKING SKILL: A QUASI-EXPERIMENTAL STUDY IN INDONESIA HIGHER EDUCATION

Zalfa Cabrina Maharani^(1*), Bambang Eko Siagiyanto⁽²⁾, Refai Refai⁽³⁾,

(1) English Department, Universitas Muhammadiyah Metro, Indonesia
(2) English Department, Universitas Muhammadiyah Metro, Indonesia
(3) English Department, Universitas Muhammadiyah Metro, Indonesia
(*) Corresponding Author

Abstract

This study explores the effectiveness of ChatGPT in enhancing the speaking skills of EFL undergraduate students in the Hospital Administration program at Universitas Muhammadiyah Metro. A quasi-experimental research design was employed, involving 42 first-semester students divided into two groups: 21 students in ARS 1 as the experimental group and 21 students in ARS 2 as the control group. The experimental group received treatment using specific prompts designed to initiate conversations with ChatGPT. Data were collected through oral tests conducted before and after the intervention (pre-test and post-test). The results indicate a statistically significant improvement in the speaking skills of students in the experimental group. Hypothesis testing showed a significance value of <0.000 for the first hypothesis and 0.004 for the second, both below the threshold of 0.05, indicating that the alternative hypotheses were accepted. In conclusion, the use of ChatGPT had a positive and significant effect on improving the speaking skills of EFL students in the Hospital Administration program. The results highlight the potential of ChatGPT to supplement classroom speaking activities, while contributing to research on AI in EFL learning

Keywords

ChatGPT, Speaking skill, EFL student

References

Akhter, S. (2021). Exploring the Significance of Speaking Skill for EFL Learners. Sir Syed Journal of Education & Social Research, 4(3).

Alsalem, M. S. (2024). EFL Students’ Perception and Attitude towards the Use of ChatGPT to Promote English Speaking Skills in the Saudi Context. Arab World English Journal, 15(4), 73–84. https://doi.org/10.24093/awej/vol15no4.5

Brown, H. D. (2004). Language Assesment: Principles and Classroom Practice. Longman.

Desnaranti, L., & Putra, F. P. (2022). Undergraduate Students Perceptions’ Analysis About Difficulties In English Speaking. Research and Development Journal of Education, 8(2), 469. https://doi.org/10.30998/rdje.v8i2.12862

Du, J., & Daniel, B. K. (2024). Transforming language education: A systematic review of AI-powered chatbots for English as a foreign language speaking practice. In Computers and Education: Artificial Intelligence (Vol. 6). Elsevier B.V. https://doi.org/10.1016/j.caeai.2024.100230

Huang, J. (2024). Enhancing EFL Speaking Feedback with ChatGPT’s Voice Prompts. International Journal of TESOL Studies, 6(3), 4–13. https://doi.org/10.58304/ijts.20240302

Huy, N. Van, Nam, N. T., & Bon, B. N. (2024). The Importance of Speaking in English as a Foreign Language between Skillful and Thoughtful Competencies: Studying Sociolinguistics Perspectives. International Journal of English Language Studies . https://doi.org/10.32996/ijels

Islam, M. S., & Roy, S. (2024). Challenges in Developing Learners’ English-Speaking Skills at the Tertiary Level in the EFL Context: Teachers’ and Learners’ Perceptions. https://doi.org/10.21203/rs.3.rs-4491944/v1

Ismailia, T. (2021). Performance Assessment Using Rubric To Improve Students’ Speaking Skill. Journal of Applied Linguistics and Literacy, 5(2). https://doi.org/https://dx.doi.org/10.25157/jall.v5i2.5786

Kim, S., & Lee, J. (2023). The Effect of Voice-enabled ChatGPT on Speaking Proficiency, Affective and Metacognitive Domains of Korean EFL Learners. Journal of Research in Curriculum & Instruction, 27(4), 344. https://doi.org/10.24231/rici.2023.27.4.344

Kohnke, L., Moorhouse, B. L., & Zou, D. (2023). ChatGPT for Language Teaching and Learning. RELC Journal, 54(3).

Long, M. H. (1996). The Role of the Linguistic Environment in Second Language Acquisition.

Muniandy, J., & Selvanathan, M. (2024). ChatGPT, a partnering tool to improve ESL learners’ speaking skills: Case study in a Public University, Malaysia. Teaching Public Administration. https://doi.org/10.1177/01447394241230152

Nhu, P. T. T. (2024). Investigating the effectiveness of practicing English conversation with ChatGPT in improving non-English majored students’ English speaking skills at Nguyen Tat Thanh University. Journal of Science and Technology, 7(4). https://doi.org/10.55401/b1rnww21

Nuñez, A. A. C., Nuñez, M. S. C., Pachay, J. F. Z., & Bosquez, A. M. C. (2025). Using ChatGPT Voice to Improve Speaking Skills in English Language Learners. Ciencia Latina Revista Científica Multidisciplinar, 9(1), 7143–7161. https://doi.org/10.37811/cl_rcm.v9i1.16390

OpenAI. (2023). ChatGPT: Optimizing language models for dialogue. OpenAI. https://openai.com/blog/chatgpt

Phuong, N. T. K. (2024). The Role of ChatGPT in Teaching Speaking Skills for English Majored Students: A Research Perspective. International Journal of Social Science and Human Research, 07(10). https://doi.org/10.47191/ijsshr/v7-i10-60

Pratiwi, N., Efendy, A. G., Rini, H. C., & Ahmed, N. A. (2024). Speaking Practice using ChatGPT’s Voice Conversation: A Review on Potentials and Concerns. Journal of Language Intelligence and Culture, 6(1), 59–72. https://doi.org/10.35719/jlic.v6i1.149

Raihan. (2017). Metodologi Penelitian. Universitas Islam Jakarta.

Salsabil, A. D., & Rakhmawati, L. A. (2025). From Silent Learners to Confident Speakers: The Effect of AI Voice Chat with ChatGPT on EFL Speaking Skills. Journal of English Education, 3(1), 38–50. https://doi.org/10.61994/jee

Siyoto, S., & Sodik, A. (2015). Dasar Metodologi Penelitian. Literasi Media Publishiing.

Sugiyono. (2022). Metode Penelitian Kuantitatif , Kualitatif, dan R&D (26th ed.). Penerbit Alfabeta.

Szabó, F., & Csépes, I. (2022). Constructivism in language pedagogy. Hungarian Educational Research Journal, 13(3), 405–417. https://doi.org/10.1556/063.2022.00136

Üstünbaş, Ü. (2024). Hey, GPT, Can We Have a Chat?: A Case Study on EFL Learners’ AI Speaking Practice. International Journal of Modern Education Studies, 8(1). https://doi.org/10.51383/ijonmes.2024.318

Zakiyah, ‎. (2025). AI-driven Language Learning: Can ChatGPT Improve Student Speaking Proficiency? KnE Social Sciences, 10(14), 20–37. https://doi.org/10.18502/kss.v10i14.19071

DOI: http://dx.doi.org/10.24127/pj.v14i3.12338