Language Learning through AI Chatbots: Effectiveness and Cognitive Load Analysis
Downloads
Background. The use of AI chatbots in language learning has gained popularity due to their accessibility and ability to provide interactive practice. However, concerns regarding their effectiveness in improving language skills and the potential cognitive load they place on learners remain underexplored. With the increasing integration of AI in education, understanding how these tools impact language acquisition is critical for optimizing learning outcomes.
Purpose. This study aims to evaluate the effectiveness of AI chatbots in language learning and analyze the cognitive load experienced by learners during chatbot interactions.
Method. A mixed-methods approach was used, combining quantitative assessments of language proficiency before and after using an AI chatbot for a three-week period, with qualitative interviews to assess learner perceptions of cognitive load. Participants included 60 university students learning a new language.
Results. The results indicate that AI chatbots significantly improved language skills, particularly in conversational fluency and vocabulary acquisition. However, a moderate level of cognitive load was reported by learners, primarily due to the need to simultaneously engage in real-time conversation and process feedback. While the cognitive load was not overwhelming, it varied based on individual learner characteristics, such as prior language proficiency and familiarity with AI tools.
Conclusion. In conclusion, AI chatbots offer an effective method for enhancing language learning, particularly in improving fluency and vocabulary. However, managing cognitive load is crucial to maximizing their educational potential. Further research is recommended to explore adaptive chatbot designs that can tailor interactions to individual learner needs.
Adiguzel, T., Kaya, M. H., & Cansu, F. K. (2023). Revolutionizing education with AI: Exploring the transformative potential of ChatGPT. Contemporary Educational Technology, 15(3). Scopus. https://doi.org/10.30935/cedtech/13152
Aeni, N., Khang, A., Al Yakin, A., Yunus, M., & Cardoso, L. (2023). Revolutionized Teaching by Incorporating Artificial Intelligence Chatbot for Higher Education Ecosystem. Dalam AI-Centric Modeling and Analytics: Concepts, Technologies, and Applications (hlm. 43–76). CRC Press; Scopus. https://doi.org/10.1201/9781003400110-4
Ahadi, N., Zanjanab, A. G., Sorooshian, S., Monametsi, G., Virutamasen, P., Wongpreedee, K., Deebhijarn, S., & Taghipour, A. (2023). The outlook of ChatGPT, an AI-based tool adoption in Academia: Applications, challenges, and opportunities. IEEE Reg 10 Annu Int Conf Proc TENCON, 15–20. Scopus. https://doi.org/10.1109/TENCON58879.2023.10322527
Al-Abdullatif, A. M., Al-Dokhny, A. A., & Drwish, A. M. (2023). Implementing the Bashayer chatbot in Saudi higher education: Measuring the influence on students’ motivation and learning strategies. Frontiers in Psychology, 14. Scopus. https://doi.org/10.3389/fpsyg.2023.1129070
Armaselu, F. (2024). Playing the Imitation Game: Human-AI Simulators in Pedagogic Design. Dalam Lorig F., Tucker J., Lindstrom A.D., Dignum F., Murukannaiah P., Theodorou A., & Yolum P. (Ed.), Front. Artif. Intell. Appl. (Vol. 386, hlm. 46–54). IOS Press BV; Scopus. https://doi.org/10.3233/FAIA240181
Balcombe, L. (2023). AI Chatbots in Digital Mental Health. Informatics, 10(4). Scopus. https://doi.org/10.3390/informatics10040082
Belda-Medina, J., & Calvo-Ferrer, J. R. (2022). Using Chatbots as AI Conversational Partners in Language Learning. Applied Sciences (Switzerland), 12(17). Scopus. https://doi.org/10.3390/app12178427
Casciato, D., Mateen, S., Cooperman, S., Pesavento, D., & Brandao, R. A. (2024). Evaluation of Online AI-Generated Foot and Ankle Surgery Information. Journal of Foot and Ankle Surgery. Scopus. https://doi.org/10.1053/j.jfas.2024.06.009
Cavallaro, A., Romano, M., & Laccone, R. (2024). Examining User Perceptions to Vocal Interaction with AI Bots in Virtual Reality and Mobile Environments: A Focus on Foreign Language Learning and Communication Dynamics. Dalam Degen H. & Ntoa S. (Ed.), Lect. Notes Comput. Sci.: Vol. 14734 LNAI (hlm. 20–30). Springer Science and Business Media Deutschland GmbH; Scopus. https://doi.org/10.1007/978-3-031-60606-9_2
Chan, M. M., Amado-Salvatierra, H. R., Hernandez-Rizzardini, R., & De La Roca, M. (2023). The potential role of AI-based Chatbots in Engineering Education. Experiences from a teaching perspective. Proc. Front. Educ. Conf. FIE. Proceedings - Frontiers in Education Conference, FIE. Scopus. https://doi.org/10.1109/FIE58773.2023.10343296
Chetradevee, S. L., Anushka Xavier, K., & Jayapandian, N. (2022). Artificial Intelligence Technological Revolution in Education and Space for Next Generation. Dalam Sharma H., Shrivastava V., Kumari Bharti K., & Wang L. (Ed.), Lect. Notes Networks Syst. (Vol. 461, hlm. 371–382). Springer Science and Business Media Deutschland GmbH; Scopus. https://doi.org/10.1007/978-981-19-2130-8_30
Faruqui, S. H. A., Tasnim, N., Basith, I. I., Obeidat, S. M., & Yildiz, F. (2024). Board 46: Integrating AI in Higher-Education Protocol for a Pilot Study with’SAMCares An Adaptive Learning Hub’. ASEE Annu. Conf. Expos. Conf. Proc. ASEE Annual Conference and Exposition, Conference Proceedings. Scopus. https://www.scopus.com/inward/record.uri?eid=2-s2.0 85202055079&partnerID=40&md5=0bc78f35f2fdaaa4d98596d20bd6a702
Haristiani, N. (2019). Artificial Intelligence (AI) Chatbot as Language Learning Medium: An inquiry. Dalam Ifdil I., Yohandri Y., Krismadinata K., & Rahim R. (Ed.), J. Phys. Conf. Ser. (Vol. 1387, Nomor 1). Institute of Physics Publishing; Scopus. https://doi.org/10.1088/1742-6596/1387/1/012020
Hwang, Y., Lee, S., & Jeon, J. (2024). Integrating AI chatbots into the metaverse: Pre-service English teachers’ design works and perceptions. Education and Information Technologies. Scopus. https://doi.org/10.1007/s10639-024-12924-4
Ivanova, M., Grosseck, G., & Holotescu, C. (2024). Unveiling Insights: A Bibliometric Analysis of Artificial Intelligence in Teaching. Informatics, 11(1). Scopus. https://doi.org/10.3390/informatics11010010
Jose, J., & Jose, B. J. (2024). Educators’ Academic Insights on Artificial Intelligence: Challenges and Opportunities. Electronic Journal of E-Learning, 22(2), 59–77. Scopus. https://doi.org/10.34190/ejel.21.5.3272
Kajiwara, Y., & Kawabata, K. (2024). AI literacy for ethical use of chatbot: Will students accept AI ethics? Computers and Education: Artificial Intelligence, 6. Scopus. https://doi.org/10.1016/j.caeai.2024.100251
Krishnam, N. P., Bora, A., Swathi, R. S. V. R., Gehlot, A., Talwar, S., & Raghu, T. (2023). AI-Based advanced Talk-chatbot for Implementation. Int. Conf. Adv. Comput. Innov. Technol. Eng., ICACITE, 1808–1814. Scopus. https://doi.org/10.1109/ICACITE57410.2023.10182611
Lee, H.-Y., & Huang, Y.-M. (2024). Bridging STEM Education and Ubiquitous Learning: A Case Study on Developing a LINE Chatbot with Google’s Gemini for Virtual Peer Collaboration. Dalam Cheng Y.-P., Pedaste M., Bardone E., & Huang Y.-M. (Ed.), Lect. Notes Comput. Sci.: Vol. 14786 LNCS (hlm. 237–246). Springer Science and Business Media Deutschland GmbH; Scopus. https://doi.org/10.1007/978-3-031-65884-6_25
Misra R., Misra R., Rajarajan M., Veeravalli B., Kesswani N., & Patel A. (Ed.). (2023). 7th International Conference on Internet of Things and Connected Technologies, ICIoTCT 2022. Lecture Notes in Networks and Systems, 616 LNNS. Scopus. https://www.scopus.com/inward/record.uri?eid=2-s2.0-85171994140&partnerID=40&md5=f9792d433d729745718d5334934a720b
Neo, M. (2022). THE MERLIN PROJECT: MALAYSIAN STUDENTS’ ACCEPTANCE OF AN AI CHATBOT IN THEIR LEARNING PROCESS. Turkish Online Journal of Distance Education, 23(3), 31–48. Scopus.
Sabeh, H. N. (2024). What Drives IT Students Toward ChatGPT? Analyzing the Factors Influencing Students’ Intention to Use ChatGPT for Educational Purposes. Int. Multi-Conf. Syst., Signals Devices, SSD, 533–539. Scopus. https://doi.org/10.1109/SSD61670.2024.10548826
Samala, A. D., Zhai, X., Aoki, K., Bojic, L., & Zikic, S. (2024). An In-Depth Review of ChatGPT’s Pros and Cons for Learning and Teaching in Education. International Journal of Interactive Mobile Technologies, 18(2), 96–117. Scopus. https://doi.org/10.3991/ijim.v18i02.46509
Sawlowicz, Z. (2024). Artificial Intelligence in the education of geology students – the hero or the killer? Przeglad Geologiczny, 72(5), 209–214. Scopus. https://doi.org/10.7306/2024.8
Shahzad, M. F., Xu, S., & Zahid, H. (2024). Exploring the impact of generative AI-based technologies on learning performance through self-efficacy, fairness & ethics, creativity, and trust in higher education. Education and Information Technologies. Scopus. https://doi.org/10.1007/s10639-024-12949-9
Tang, C. M., Ng, V. S. C., Leung, H. M. F., & Yuen, J. C. H. (2024). AI-Generated Programming Solutions: Impacts on Academic Integrity and Good Practices. Dalam Poquet O., Ortega-Arranz A., Viberg O., Chounta I.-A., McLaren B., & Jovanovic J. (Ed.), International Conference on Computer Supported Education, CSEDU - Proceedings (Vol. 2, hlm. 478–485). Science and Technology Publications, Lda; Scopus. https://doi.org/10.5220/0012563600003693
Terzic, I., Drobnjak, A., & Boticki, I. (2023). Designing Educational Personas using Generative AI. Dalam Shih J.-L., Kashihara A., Chen W., Chen W., Ogata H., Baker R., Chang B., Dianati S., Madathil J., Yousef A.M.F., Yang Y., & Zarzour H. (Ed.), Int. Conf. Comput. Educ., ICCE - Proc. (Vol. 2, hlm. 961–963). Asia-Pacific Society for Computers in Education; Scopus. https://www.scopus.com/inward/record.uri?eid=2-s2.0-85181770328&partnerID=40&md5=456b4da0924a418e136900cbdf5b2edc
Vera, M. C. S., & Palaoag, T. D. (2023). MedPlantBot: AI Chatbot Architectural Design Framework for Responsible Use of Medicinal Plants. IEEE Int. Conf. ICT Bus. Ind. Gov., ICTBIG. 3rd IEEE International Conference on ICT in Business Industry and Government, ICTBIG 2023. Scopus. https://doi.org/10.1109/ICTBIG59752.2023.10456018
Veras, M., Dyer, J.-O., Rooney, M., Silva, P. G. B., Rutherford, D., & Kairy, D. (2023). Usability and Efficacy of Artificial Intelligence Chatbots (ChatGPT) for Health Sciences Students: Protocol for a Crossover Randomized Controlled Trial. JMIR Research Protocols, 12(1). Scopus. https://doi.org/10.2196/51873
Wu, H.-S. (2024). Subverting the Future of Teaching: Artificial Intelligence Innovation in Nursing Education. Journal of Nursing, 71(2), 20–25. Scopus. https://doi.org/10.6224/JN.202404_71(2).04
Wu, T., & Yu, Z. (2024). Bibliometric and Systematic Analysis of Artificial Intelligence Chatbots’ Use for Language Education. Journal of University Teaching and Learning Practice, 21(6). Scopus. https://doi.org/10.53761/6jmnst54
Zaphiris P., Ioannou A., & Ioannou A. (Ed.). (2024a). 11th International Conference on Learning and Collaboration Technologies, LCT 2024, held as part of the 26th International Conference on Human-Computer Interaction, HCII 2024. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 14724 LNCS. Scopus. https://www.scopus.com/inward/record.uri?eid=2-s2.0-85196846290&partnerID=40&md5=946683875ab7d914f13c0747a1cfc8a9
Zaphiris P., Ioannou A., & Ioannou A. (Ed.). (2024b). 11th International Conference on Learning and Collaboration Technologies, LCT 2024, held as part of the 26th International Conference on Human-Computer Interaction, HCII 2024. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 14722 LNCS. Scopus. https://www.scopus.com/inward/record.uri?eid=2-s2.0-85197235626&partnerID=40&md5=4e7c89ade785b98f89bdfb556e51ec79
Zarei, M., Zarei, M., Hamzehzadeh, S., Oliyaei, S. S. B., & Hosseini, M.-S. (2024). ChatGPT, a Friend or a Foe in Medical Education: A Review of Strengths, Challenges, and Opportunities. Shiraz E Medical Journal, 25(6). Scopus. https://doi.org/10.5812/semj-145840
Zhang, Z., & Huang, X. (2024). The impact of chatbots based on large language models on second language vocabulary acquisition. Heliyon, 10(3). Scopus. https://doi.org/10.1016/j.heliyon.2024.e25370
Copyright (c) 2024 Muh. Safar, Dini Anggraheni
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
