Looking back on four years of Learning Analytics
Part 2: Effective implementation in the institution
Over the past four years, we heavily invested in Learning Analytics (LA) at Utrecht ľ¹Ï¸£ÀûÓ°ÊÓ (UU). The aim of the LA project was to investigate the added value of LA for education at the UU and to create the facilities needed for LA. Now that this period is coming to an end, it is time to look back: what have we learned and what will we take with us into the future?
1. Getting everyone on the same page
Our first tip is: devote a lot of time and attention to that broad group of stakeholders.
In the first two years, we were hardly able to implement any LA projects in education. We first had to create the conditions necessary to start using LA in a responsible, safe, and widely supported manner. This led to our LA roadmap, which takes into account the educational, privacy, ethical, and security aspects of LA.
Some experts advise starting small with LA, for example with a pilot in a single course, in order to generate management buy-in. We didn't need to convince administrators, but rather connect people on the work floor who need to embrace LA, such as teachers, policy officers, and educational support staff. Think of:
- Functional and system administrators (ITS): indispensable for the rapid technical implementation of an LA initiative.
- Educational directors: they can roll out LA more broadly within a faculty and create space in education to experiment with and learn about LA.
- Privacy officers: they know what potential ethical and legal objections or challenges may arise. If you explain the purpose of LA to them at an early stage, you can take these objections into account and prevent them.
2. Make sure there's enough support for end users
Our second tip is: focus on one or two larger initiatives where you can really make an impact.
To implement LA successfully, you need enthusiastic end users. But these are often the people with the busiest schedules. That is why it is essential to involve stakeholders who can ensure that sufficient time and resources are available to get started with LA in earnest, such as educational directors or educational support staff.
In practice, we find that we spend a lot of time implementing and testing LA initiatives. Our LA team has taken on a large part of these tasks, but this is not a sustainable solution for future projects. As an LA team or educational institution, do you expect that implementing LA will also take a lot of time? If so, we recommend not starting too many LA projects at the same time.
3. Data maturity
Our third tip is to first conduct a data maturity check of the institution before starting LA.
A sufficiently high level of data maturity is a prerequisite for the successful implementation of LA. Is the level of data maturity not yet sufficient? Then make sure you have sufficient resources and support to improve this, so that scaling up LA is a realistic goal. This often requires cooperation between different departments. So start the conversation with all parties involved in early on, so that you have the right people, resources, and facilities in place. This also requires devoting a lot of time and attention to the right stakeholders (see lesson 1).
4. LA has multiple focus areas – which ones do we choose?
Our fourth tip is to link your LA program to strategic ambitions in order to increase the likelihood of relevant and sustainable results.
During our LA pilots in educational practice, we worked closely with teachers, students, and study advisors. They had a significant say in the design of the final LA solutions. This meant that we regularly adjusted the dashboards on request, tailoring them to the wishes of the end users.
When scaling up, such an ad hoc approach is not feasible. The challenge is therefore to find the right balance: on the one hand, you want solutions that are scalable and widely applicable, but on the other hand, they must offer sufficient customization to remain truly useful and valuable to the end user. It is therefore advisable to focus on one or two projects that have the potential to be relevant to a large group of users. The choice of specific projects depends on the institution's policy: in which LA areas do we want to continue investing in order to advance our education?
The next step in Learning Analytics at UU
After four years of working on LA within the UU, we have built up a lot and learned a great deal. We started by creating the preconditions—policy, organization (LA roadmap), infrastructure, data maturity—before implementing LA projects in education. It was important to involve stakeholders at an early stage, from educational directors to privacy officers.
Successful implementation of LA requires vision and practical support, time, attention to end users, and realistic expectations. Our advice is to start small with a few impactful projects, rather than multiple smaller initiatives.