Better Learning Analytics: Lessons from EdX, Hack Education, Mozilla + P2PU


Screen Shot 2014-08-20 at 10.42.40 AM


Yesterday, Audrey Watters of Hack Education,  Andrew Sliwinski at Mozilla Foundation, Justin Reich of HarvardX and Berkman Center, Vanessa Gennarelli of P2PU came together to discuss the current obsession with the field of learning analytics. 

Check out the conversation here as we cover:

  • Whether learning analytics are a new field
  • What sorts of data can be collected (engagement versus comprehension/recall)
  • Difficulties in acquiring/sharing reliable data
  • Power and learner agency
  • Ethics, privacy and the impact of research on results

We invite you to weigh in with your opinion at

Articles and Blog Posts Referenced:

 Our Takeaways and Reflections:

Andrew: In the past we’ve tried to bite off more than we can chew – this time is no exception, particularly when it comes to ethics! But we covered a broad range of topics, at a skin-deep level, which is kind of a bummer, because we have great people here, but for the purposes of this call, that was great. 
  • Working on the questions a few days before was very helpful for moderation purposes – gave me a bit of background, and a sense of who to call on for which questions, to get the conversational ball rolling. 
  • I also love how the conversation turned to the humanities vs sciences discussion, which consumes an enourmous amount of my headspace in real life – seems the issues are completely transferable. And I’m not sure if any new solutions will come out of the digital world. The kinds of things you learn reading Plato will never be as measurable as those you learn in Physics 101 and I’m not sure it even matters. (Although of course it does, if Classics departments have no money…)
  • I’m surprised at how interwoven these topics are getting–we can’t seem to talk about learner pathways without analytics, can’t talk about analytics without talking about scale. 
  • Audrey spoke about how those on the panel generally agree about *most* things, it became clear how nuanced the differences are here in the approaches we take. Really enjoyed that about the conversation. 
  • Obviously we at P2PU have more thinking to do about privacy–how to inform learners and where that bumps up against influencing the data we’re collecting.
  • None of us come from a ed-tech for-profit/start up, which is a mistake in hindsight. Those are the folks who I’d encourage to do this kind of deep thinking. Maybe we should send it to them 🙂
  • For me, engagement data is inextricably linked with other sorts of learning data–motivation, community health and learning are all interleaved bits
Audrey: We share values around these things, but this call showed how valued the nuances are – for instance, I find the whole psychometrics thing quite terrifying, but I found the conversation quite subtle. I think we probably disagree about a lot here. Is that a difference between individuals and institutions? I wonder if we still need to define more clearly what we mean by “learning” — or if, as we did with this discussion, show that it’s not a clear-cut thing at all. (In other words, I think there’s a value to the messiness of some of this discussion too.) 
Justin: focussing more on a couple of areas might have been helpful – but we don’t know what the audience is. Several of us are working on different platforms which have different periods of engagement, but giving a bit of into on the platforms and the differences between them would have been useful. While definitions and frame setting are boring, I also think it may be important. Should we define learning analytics more carefully when talking about it? Does it just mean “educational research with data from computers?” Or something more specific?
Andrew: +1 on diving into the details. We keep generalizing due to the time / speed at which we are covering large areas which leads to a false sense of simplicity. I believe that this is also reflected in the (seeming) lack of differences in values between speakers. Nathan and I, for example, come from WILDLY different approaches to learning (and even how we define learning) but very often have common ground in terms of the problems we face. These differences become more acute when we get into the details of implementation.




Back to Blog home