Colloquium: How Students Use AI – Evidence from a University Chatbot

Abstract:
We study the extent, nature, and malleability of students’ AI use at a German distance-learning university. To this end, we link full texts of conversations with a GPT-based tutor chatbot to administrative student records and a pre-adoption survey. A preregistered randomized encouragement campaign aims at increasing students’ AI exposure and usage intensity. We document selective baseline AI adoption, concentrated among students with prior AI experience and higher digital competence. Supervised text classifications of conversation content reveal that higher-order uses that more directly resemble learning augmentation account for only a small share of interactions. Instead, most classified requests concern concept explanations or task solving, pointing to a pattern of AI use that is closer to cognitive substitution than to complementarity in skill formation. Students treated with the encouragement campaign substantially increased usage intensity, but did not change the cognitive composition of queries. Preliminary results for educational downstream outcomes show positive effects on study progress without affecting students' GPA.