Results from a LA pilot: providing teachers information about students’ consultation of course materials
Many teachers at Utrecht ľϸӰ (UU) use some form of the flipped classroom: they provide reading or video materials for students to prepare before the teacher-guided classes on campus. But how well do students engage with these preparatory materials? And can insight into the use of these materials help teachers to support students?
Recently, our learning analytics (LA) team was approached by two teachers with a the following request: “Can team LA help us gain insight students’ consultation of the course materials, e.g. via a dashboard?” Their goal was to be able to better support students who might be falling behind by identifying which materials were being accessed, and using that information to fine-tune the content of the teacher-guided classes. Teachers expected they could also offer additional recommendations for supplementary materials based on that information.
As we do for all LA initiatives at the UU, we followed the LA roadmap for this project as well. During that process, we specify the exact end result the teachers had in mind and made sure to involve all relevant stakeholders. Because the teachers requested information at individual student level, we put extra effort into hearing students’ opinion and performed an elaborate privacy assessment.
Once we obtained the green light from all stakeholders, we were good to go with respect to the technical realization of the dashboard (start of the project). The figure below shows the dashboard that was used by the teachers. The dashboard consists provides an overview of whether students accessed the course (per week), and which course materials were accessed by which students (also per week).
An import step in the LA roadmap is to evaluate if the intended goal of the project was reached. Both teachers and students were interviewed about how they perceived the use of the dashboard during the course. The teachers mentioned that they consulted the dashboard every week during the course. They found the dashboard generally easy to use and valuable. They consulted the dashboard to track student activity, which helped tailor tutorial sessions based on material engagement and also used the dashboard to engage students, offer reminders, and highlight key resources. For the teachers, it was an eye-opener to have insight into which materials were opened by students and which were not. However, while there was an expected benefit of the dashboard at individual and group level, in practice most benefit was experienced at the group level, allowing teachers to infer patterns in student consultation of materials. Based on that information, the teachers adapted the content of tutorial sessions, allowing them to put effort into those materials and activities that would benefit students most.
Students were also positive about the project, indicating they noticed teachers’ adaptation of tutorials, and there is some evidence that students’ motivation went up. Students also saw the dashboard’s potential for personalizing education but expressed concerns about how it could affect teacher-student dynamics, though in this case, no negative effects were noted.
Overall, the project highlighted the potential of LA to improve teaching strategies and student engagement. Because UU are changing their learning management system (LMS) from BlackBoard to BrightSpace, we are currently investigating if the LA pilot will be continued with BrightSpace.