Using learning analyses in universities

The project aimed to address how learning analytics can support learning processes in higher education across stakeholders and contexts, in line with learning theories. Rather than designing new machine learning algorithms, the goal was to build learning analytics tools (metrics, visualisations, feedback) that enable stakeholders (students, teachers, administrators) to engage in improved educational processes that positively impact learning.

The project had three main objectives:

1. To investigate what data can effectively support stakeholders' educational goals in different contexts.
2. To investigate how to report this data to stakeholders in a beneficial way in terms of form and level of support.
3. To investigate the impact of these learning analytics on educational processes and outcomes.

The project focused on anchoring learning analytics in learning theories, proactively developing tools with stakeholders and empirically measuring their educational impact in controlled and real-world settings.

Learning analytics tools aim to provide data-driven insights to improve teaching and learning in higher education. However, adoption of these tools has been limited. This project sought to understand the barriers to adoption and develop strategies and tools to overcome them. The research drew on frameworks like Hattie's Visible Learning, which emphasises the importance of feedback in making learning visible. It also employed approaches like Design-Based Implementation Research to bridge the gap between research and practice. The original plan was for the project to focus on three stakeholder groups – teachers, students and administrators; however, the scope was strategically narrowed to teachers and students to accelerate the development of a key tool.

The project aimed to:

1. identify obstacles to learning analytics adoption in higher education
2. develop strategies to overcome adoption barriers
3. create and evaluate learning analytics tools aligned with stakeholder needs and teaching practices
4. provide empirical evidence for the impact of learning analytics on teaching and learning

The project has several key implications:

1. Professional development is needed to support instructors in shifting to more student-centred, data-responsive teaching approaches that can leverage learning analytics.
2. Learning analytics tools should be integrated into existing educational platforms to reduce barriers to adoption, rather than developed as standalone systems.
3. Agile, human-centred design methodologies are crucial for developing learning analytics tools that meet stakeholder needs and mitigate risks.
4. Further research is needed on the quantitative impact of learning analytics tools on teaching and learning outcomes.
5. Careful consideration must be given to potential negative effects of learning analytics on classroom dynamics and student behaviours.
6. Open-source tools like Jupyter Analytics offer opportunities for wider adoption and research on learning analytics in higher education.

These implications suggest a need for changes in higher education policy, teacher training, educational technology development practices and further research to realize the potential of learning analytics while mitigating risks.

Three main messages

1. Training stakeholders on how to use learning analytics tools is not enough.
   The first barrier to adoption is that learning analytics tools are often incompatible with the existing teaching practices of many instructors. To utilise learning analytics effectively in a course, an instructor has to teach in an adaptive and flexible way. This is because learning analytics tools can provide feedback at unexpected times and about unanticipated issues. However, many instructors teach in fixed ways, such as preparing all of their lecture materials before the start of the semester. The research team found that instructors who taught in fixed ways held a “teacher-centred” mindset, while instructors who taught in adaptive ways had a more “student-centred” mindset. It is not enough to simply show teachers how to use learning analytics tools, since their reasons for not using those tools lie deeper. Instructors in higher education need professional development that supports them in transitioning from a “teacher-centred” to a “student-centred” approach.
2. To accelerate the development and adoption of learning analytics, institutions should shift to digital education platforms that support plugins or extensions.
   The second barrier to adoption is that the costs of adopting learning analytics tools are too high for most stakeholders. In higher education, time is the most precious resource, and most learning analytics tools are very time-consuming for users. The costs associated with finding, installing, learning and using new software demand more resources than most stakeholders in higher education can afford. This is what motivates our second recommendation. Rather than developing new software from scratch, learning analytics researchers and developers should aim to build their tools into existing, widely adopted platforms, such as Moodle, Tableau or Jupyter notebooks. This can be thought of as a Trojan horse approach for the adoption of learning analytics. The adoption costs of installing a plugin into an already known system like Moodle are far lower than the costs of adopting entirely new software. If institutions provide stakeholders with digital platforms that are open source and easily extensible with plugins, they also neutralise a barrier to adoption by opening a door for designers of learning analytics tools to pass through.
3. Agile and human-centred design methodologies help ensure that learning analytics tools will be useful to stakeholders and can help reveal and mitigate risks.
   The research team designed and developed a new learning analytics platform called Jupyter Analytics, which is the first end-to-end analytics platform integrated into Jupyter notebooks. They co-created Jupyter Analytics with instructors and teaching assistants, and they have invested great effort into making it open-source, easy to install and use, secure and mindful of privacy. Jupyter Analytics is already being used in multiple classrooms in Switzerland, and the research team is working together with international instructors who wish to incorporate it into their teaching.

Uni Analytics: What, How, and Why Do Different Educational Stakeholders Use Learning Analytics in Higher Education?