Open Universiteit

Please use this identifier to cite or link to this item: http://hdl.handle.net/1820/7932
Title: Design, Implementation and Evaluation of two Widgets for a MOOC’s Learning Analytics Dashboard’
Authors: Toisoul, Christian
Keywords: Learning Analytics
MOOC
EFLA
xAPI
widgets
LA-tools
tool-evaluation
Issue Date: 30-Mar-2017
Publisher: Open Universiteit
Abstract: One of the most important sources of feedback in a Massive Open Online Course (MOOC) is the learning analytics dashboard (LAD) that provides the learners and teachers with appropriate feedback about the learning process. Kim, Jo, and Park (2016) conclude that a LAD should be built meticulously towards the needs of the users if the LAD replaces the feedback from traditional teachers or tutors as it is the case in MOOCs. The goal of this Technology Enhanced Learning study was threefold. First an extended literature review was conducted on LADs for MOOCS. A second goal was to design and develop two LA widgets according to state-of-the-art research: A first widget that monitors the learners’ activity and compares it to that of other participants; a second widget that gives a detailed overview of the activities per learner. The third goal of the research was to evaluate those widgets with the Evaluation Framework for Learning Analytics questionnaire (EFLA). In addition to the EFLA questions we also asked for interacting factors like ‘academic achievement’ and ‘intention towards the MOOC’ and looked at their effect on the evaluation. For the current study only the learners were taken into account for the analysis. Due to the real setting in which this research took place, it was not possible to actively contact the MOOCs’ learners and teachers. This resulted in not enough teachers (n=2) returning the questionnaire. MOOC learners were selected based on their activity in the ECO-learning environment at two specific moments. In September 2016 the learners that were active on the platform (n=284) after June 1, 2016 were asked to answer a questionnaire about the current LAD. A month later the dashboard was altered by adding the new developed widgets. After that the learners were again asked to answer the evaluation questionnaire and were explicitly asked to focus on the two new developed widgets. The analysis of the widget evaluation was performed with the answers from respondents (n=8) that answered both the first and the second round questionnaire. The analysis of the interacting factors were done with the results from all respondents (n = 44). For the evaluation of the widgets the EFLA (Scheffel & Drachsler, 2016) was used. The results indicate that for all criteria the averages of the new developed widgets were improved but only significantly on the Data criterion: t(7) = -3.21, p = .015. A MANOVA performed on the complete dataset showed interaction effects on academic achievement, intentions towards the MOOC and the different set of tools. The new tools seemed to be rated better by people with a Master degree that have the intention in finishing the course. This could be related to greater self-regulation skills. We can conclude this study with a positive evaluation of the new developed widgets. However, further research with more evaluations and a more controllable environment is needed to consolidate the findings of this study. The research also focuses the attention on some important LAD design issues; e.g. the possibility to provide different dashboard visualisations to different type of users. The new developed widgets will be available under Open Source licence via the Open University of the Netherland’s DSpace…..
Description: Toisoul, C. (2017). Design, Implementation and Evaluation of two Widgets for a MOOC’s Learning Analytics Dashboard’. Maart, 30, 2017, Heerlen, Nederland: Open Universiteit.
URI: http://hdl.handle.net/1820/7932
Appears in Collections:MSc Learning Sciences

Files in This Item:
File Description SizeFormat 
OWCToisoul-30032017.pdf635.37 kBAdobe PDFView/Open


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.