Using a RAG-enhanced large language model in a virtual teaching assistant role: Experiences from a pilot project in statistics education

Németh, Renáta and Tátrai, Annamária and Szabó, Miklós and Tamási, Árpád (2024) Using a RAG-enhanced large language model in a virtual teaching assistant role: Experiences from a pilot project in statistics education. HUNGARIAN STATISTICAL REVIEW, 7 (2). pp. 3-27. ISSN 2630-9130

Preview

Text 2024_02_003.pdf - Published Version Download (338kB) | Preview

Abstract

The role of artificial intelligence (AI) in education is expected to grow, but how it transforms teaching and learning remains unclear. This study explores the use of an AI tutor that is similar to ChatGPT enhanced with retrieval-augmented generation (RAG), in a pilot project at the Faculty of Social Sciences of Eötvös Loránd University in Budapest, Hungary. The tutor provided a searchable knowledge base for students preparing for admission to the MSc in Survey Statistics and Data Analytics. Instructor feedback highlighted the tutor’s ability to deliver accurate, textbook-based responses, but noted limitations in addressing real-world complexities. Student feedback, which was gathered through focus groups and surveys, showed high satisfaction and many used the tool for active learning such as comparing concepts and organising material. Students had the flexibility to adapt the tutor to their own learning strategy, and they also noted the importance of the tutor as a time-saving supplement rather than a replacement for comprehensive study. Approximately 15% of student queries demonstrated critical thinking, where students used the AI tutor to confirm their own interpretations. Similarly, around 15% showed active learning, seeking explanations and comparisons or generated study guides, while nearly 30% engaged directly with course material, referencing specific concepts and theories from their readings. Instructor evaluation revealed that 76% of the AI tutor’s responses were fully correct, 17% mostly correct and only 6% were misleading. The findings suggest that RAG models hold promise for enhancing learning by offering reliable, interactive and efficient support for students and educators.

Actions (login required)

Edit Item