Can chatbots improve student progression and achievement? This question builds on the following observations:
• Students seem to perform better when they have frequent and diverse forms of interaction within their programme of study, in the form of prompt answers to their queries, validation of their understanding of the topic, and feedback. They also appreciate opportunities to self-regulate their study. This argument is supported by scholarly works that empirically investigated the positive relationship between students’ interactions, self-regulation and performance (Kommaraju et al. 2010, O’Keeffe 2013, Paolini 2015, Pintrich and De Groot 1990).
• During the last years, advances in artificial intelligence (AI) and natural language processing (NLP) opened up novel possibilities to use chatbots for enhancing the interaction between service providers and users. Examples include assisting students to select university courses at Leeds Beckett University
, providing mental health assistance
and teaching foreign languages
. Contemporary open source and commercial offerings provide the opportunity to employ chatbots in a convenient and inexpensive way. Some academic works have been made on the use of digital artefacts in higher education, and on their effects on students’ engagement and learning (Goda et al. 2014, Jia and Chen 2009, Pereira 2016, Popenici and Kerr 2017, Song et al. 2017).
• The advances within inexpensive AI and NLP combined with the fact that student’s interaction and self-regulation seem to improve their performance, makes it possible to investigate how the use of chatbots can affect distance learning students on progression and achievement. Chatbots provide an innovative and convenient way to fulfil some functions like providing prompt response to students’ queries, offering further clarifications and examples about topics covered in the programme, and help students test their understanding through questions or quizzes.
My research in this area aims to explore whether and how these functions can be effectively integrated with the existing teaching approaches and resources in ways that result in the enrichment of the students’ learning experience with beneficial effects to their progression and achievement. This aim is attained by developing and testing the use of a chatbot in distance learning modules that are taught at CeFiMS of SOAS University of London. The chatbot will provide to students an online learning resource on the specific subject areas of the modules. The test will result in some evidence about students’ use of the chatbot, their attitudes towards the chatbot, and then their progression and achievement in the module.
The use of chatbots has attracted an increased amount of attention during the last a few years. As a rough indication, Google Scholar reports that the term “chatbot” is found in 310 publications per year on average (excluding patents and citations) in the period 2012-2016, while the term is present in 1,710 publications in 2017 and in 2,330 publications in 2018 at the time of writing.
During the last couple of years, a niche stream of literature started focusing on the use of chatbots in teaching & learning applications. Part of these works highlighted the potentials of chatbots (Dibitonto et al. 2018, Dousai and Hall 2018, Molnár and Szüts 2018), while others also warned that chatbots should be designed and implemented in appropriate ways otherwise their influence may be limited to a mere novelty effect (Fryer et al. 2017). Other works that seem relevant to the use of chatbots for teaching & learning applications include those that focus on the features and effectiveness of human-chatbot interaction, including forms of gamification (Asquer 2013, Benotti et al. 2014, Du Vernet et al. 2016). These works provide various recommendations for the interaction design of chatbots, including, for example, avoiding the ‘uncanny valley’ effect of making users uneasy with fake human-like features of the digital artefact (Ciechanowski et al. 2018).
This research project differs in various respects from existing studies. First, this project tests the use of a chatbot in the context of distance learning rather than in the one face-to-face course content delivery. Second, this project tests the use of a chatbot in the context of non-language higher education programmes, while existing studies mainly focused on the use of chatbots in the teaching of languages and to college students. Finally, this project aims to explore the relationship between the use of a chatbot and substantive effects in terms of student progression and achievements, apart from exploring users’ use and satisfaction with the chatbot.
This project also builds on a prototype chatbot that has been developed on the foundations of accounting, which can be accessed on the Facebook Messenger. The accounting chatbot provides some instances of bi-directional interaction between the chatbot and the user. It includes a conversational flow that is driven by the chatbot, a short quiz at the end of the conversation, and the possibility for the user to pose questions (such as, for example, “what is depreciation?”). The accounting chatbot prototype has no practical or research use at present. However, this preparatory work has been done to display the functionalities that can be included in the project chatbot.
The project chatbot will allow gather data on the frequency and duration of usage of the chatbot by the students, on the kind of interactions that they have, on the results of the quizzes, and on the questions that students pose to the chatbot. Further data will be collected through a focus group and a survey on students’ satisfaction with the use of the chatbot. Finally, additional data will be collected on students’ progression and achievement.
Asquer, A. (2013). Not just videogames: Gamification and its potential application to public services. In Digital public administration and E-government in developing nations: Policy and practice (pp. 146-165). IGI Global.
Benotti, L., Martínez, M. C., & Schapachnik, F. (2014, June). Engaging high school students using chatbots. In Proceedings of the 2014 conference on Innovation & technology in computer science education (pp. 63-68). ACM.
Ciechanowski, L., Przegalinska, A., Magnuski, M., & Gloor, P. (2018). In the shades of the uncanny valley: An experimental study of human–chatbot interaction. Future Generation Computer Systems.
Dibitonto, M., Leszczynska, K., Tazzi, F., & Medaglia, C. M. (2018, July). Chatbot in a Campus Environment: Design of LiSA, a Virtual Assistant to Help Students in Their University Life. In International Conference on Human-Computer Interaction (pp. 103-116). Springer, Cham.
Dousay, T. A., & Hall, C. (2018, June). Alexa, tell me about using a virtual assistant in the classroom. In EdMedia+ Innovate Learning (pp. 1413-1419). Association for the Advancement of Computing in Education (AACE).
DuVernet, A., Asquer, A., & Krachkovskaya, I. (2016). 16 The gamification of education and business: a critical analysis and future research prospects. Research Handbook on Digital Transformations, 335.
Fryer, L. K., Ainley, M., Thompson, A., Gibson, A., & Sherlock, Z. (2017). Stimulating and sustaining interest in a language course: An experimental comparison of Chatbot and Human task partners. Computers in Human Behavior, 75, 461-468.
Goda, Y., Yamada, M., Matsukawa, H., Hata, K., & Yasunami, S. (2014). Conversation with a chatbot before an online EFL group discussion and the effects on critical thinking. The Journal of Information and Systems in Education, 13(1), 1-7.
Jia, J., & Chen, W. (2009). The further development of CSIEC project driven by application and evaluation in English education. British Journal of Educational Technology, 40(5), 901-918.
Komarraju, M., Musulkin, S., & Bhattacharya, G. (2010). Role of student–faculty interactions in developing college students' academic self-concept, motivation, and achievement. Journal of College Student Development, 51(3), 332-342.
Molnár, G., & Szüts, Z. (2018, September). The Role of Chatbots in Formal Education. In 2018 IEEE 16th International Symposium on Intelligent Systems and Informatics (SISY) (pp. 000197-000202). IEEE.
O'Keeffe, P. (2013). A sense of belonging: Improving student retention. College Student Journal, 47(4), 605-613.
Paolini, A. (2015). Enhancing Teaching Effectiveness and Student Learning Outcomes. Journal of effective teaching, 15(1), 20-33.
Pereira, J. (2016, November). Leveraging chatbots to improve self-guided learning through conversational quizzes. In Proceedings of the Fourth International Conference on Technological Ecosystems for Enhancing Multiculturality (pp. 911-918). ACM.
Pintrich, P. R., & De Groot, E. V. (1990). Motivational and self-regulated learning components of classroom academic performance. Journal of educational psychology, 82(1), 33.
Popenici, S. A., & Kerr, S. (2017). Exploring the impact of artificial intelligence on teaching and learning in higher education. Research and Practice in Technology Enhanced Learning, 12(1), 22.
Song, D., Oh, E. Y., & Rice, M. (2017, July). Interacting with a conversational agent system for educational purposes in online courses. In Human System Interactions (HSI), 2017 10th International Conference on (pp. 78-82). IEEE.