Pro training vs academical MOOCs

I've touched on it slightly before (mostly in off-the-cuff remarks about how Udacity seems to reorient itself towards professional training); I find it rather fascinating how different “academical” MOOCs feel from “professional” ones.

Let's introduce our terms, first. I'm using “academical” to qualify courses given by a teaching institution, with an outward goal of teaching fundamental concepts and practices, sometimes through the use of a given product. “Professional” courses are provided by companies and aim at giving immediately useable skills focused on a specific product. For instance: “Introduction to Biology, the Secret of Life” from MIT is a academical course; “Introduction to Hadoop and MapReduce” by Cloudera through Udacity is professional training.

Sometimes the distinction is a bit blurry, as some “academical” courses steer very close to “pro” − I'm thinking of UC Berkeley's “Software as a Service” course, which could very well be renamed “Introduction to Ruby on Rails”. That's not a bad thing in itself: I am definitely not arguing that “professional” training is somehow “lesser than” academical courses. Keeping oneself updated on specific products and techniques is very important in intellectual professions.

I am currently following a “professional” course from MongoDB, Inc.: “Advanced deployment and operations”, focused − obviously − on the intricacies of deploying and administering MongoDB instances. This course uses the OpenEdX platform, in a more or less classical way: video lectures, quick questions, then homework. Some of the homework uses locally-installed software that evaluates hands-on manipulations, and that's one of the great things about this course. Overall the course is great, the lectures are clear and detailed, and indeed quite advanced.

And then, there are the discussion forums. To put it bluntly… a lot of “pro” training students are, I don't know, hopeless? How can they hope to complete an advanced course on database operations when they obviously can't be bothered to do more than cut-and-paste commands without trying to understand what they're doing − and then go whine on the forums? This week I've been trying to help out − after all, not everybody who's taking the course is supposed to be proficient at using Linux (which is the platform used for the homework). But I find that a lot of people don't even understand the concepts of host names, TCP ports, or the difference between one and three. (Yes, the numbers.) And then, when things don't work, they post angry comments on the forums following the lines of “the instructions are crappy, I followed them and it gives me an error!”

I really admire the patience of the TAs.

What I find slightly disquieting is that, through help obtained on the forums (it seems the “do not post full answers” / “do not ask for homework answers”), these people will complete the course, and they will get the certificate, and they will be hopeless nonetheless. I guess I should be happy, this means I'll have no problem finding employment fixing the obvious mistakes others have made… but seriously, it means that people will be handed the keys to big databases when they really shouldn't be allowed anywhere near a root account.

Anyway. Rant off.

What I find interesting is that, by and large, we don't get the same clueless types in academical MOOCs. Oh, we do get a handful of 15-year-olds with more enthusiasm than understanding of the underlying ideas, and we do get people complaining about homework that's too difficult or “unfair”, but not nearly as many, and they are not nearly so obnoxious. And that, indeed, also applies to borderline courses − I just went back to check the forums for the Berkeley Rails course: the overall level is much higher. The course homework was to be done on a Linux virtual machine: there were overall very few issues with using that.

Now why is that? Why does a course titled “MongoDB advanced deployment and operations”, explicitly aimed at people with significant systems experience, attract so many people who lack the very basics, while a course called “Software as a Service” aimed at undergraduates, mostly gets clued-up students? I'm guessing that, in essence, a “pro training” course, being more concrete, attracts students a priori less at ease with more abstract topics. Also, pro training is more immediately useful, it is something that can be boasted about on a resume. I guess the employment value of “I completed both modules of MongoDB DBA courses” is greater than “I completed UC Berkeley's overview of building software as a service” (although, having done both courses, I'd rank them equally − actually, the Berkeley homework being harder, I'd prefer candidates with that on their CVs. But most employers will not have done all the courses.)

Of course, the a priori less technical MOOCs also get their share of less qualified people. Based on the discussion forums, I'd say a number of students who took the Copenhagen Diabetes course weren't very well armed to deal with that kind of advanced matter − but still, while I didn't use the forums as much, I didn't find them plagued with so many whines and complaints. I guess it's because the people there had a real willingness to learn, as opposed to rack up a certificate to boost up their career prospects.

Or something. I don't know, really.

I'll just avoid the MongoDB course's forums, I guess.

More certificates!

Yesterday I've collected my certificates / statements of accomplishment for 15.071x The Analytics Edge, and Diabetes: A Global Challenge.

So that's a round 12 certificates obtained, with two more on the way (Astro and Probabilities). Yay, I guess (says the guy in a mock-blasé tone of voice.)

Notes from the trenches: Epigenetics at Melbourne

[ed: it's been a while since I've posted, mostly because not much happened.]

While it's officially Week 5 in a 7-week course, I'm actually well into Week 6 (weekly content is released a week in advance), with only one quiz to take − I expect to do it later today − then it's down to the dreaded peer-reviewed essay.

All this to say that I feel capable of discussing the course.

(to digress, there's something I thought about yesterday: MOOCs are supposed to be “communities” for students, but they are transient − lasting only a handful of weeks. For the same reason it's hard to be able to discuss a course in general terms based on only a couple of weeks' content, it's difficult to get to know people − fellow students − in such a short time. I wonder if multiple-course sequences, like the ASTROx year-long series at ANU, will actually foster such a “community”.)

Anyway − epigenetics is the study of the process through which genes are stably turned on or off. The keyword here is “stably” − while a given cell will express genes differently at various times (for instance insulin secretion is turned up when glucose enters the pancreatic beta-cells), this is not an epigenetic change; rather, some genes are permanently turned off (or on) for the whole life of the cell, barring exceptional circumstances, and this state is preserved when the cell multiplies.

Something which the course makes very clear is that this is a field under active research. In other words, there is very little that we know for certain − some processes are well-understood, but most are not, and a large part of what is thought is “very controversial”, that is to say, researchers disagree strongly on how the mechanisms work, and even on whether they actually exist in the first place. So that's pretty exciting, if somewhat confusing: one doesn't quite expect to walk into a classroom and be told “okay, so we think there is some epigenetics here, but we're not sure, and we don't really know how it works anyway, so if you're thinking of doing some research of your own in the future, that's not a bad place to start.”

In keeping with this bleeding-edge focus, the course places a strong emphasis on reading scientific papers, over at PMC or PLOS − we're actually quizzed on the papers. In a way, the video lectures are only an introduction, the real meat of the course being the papers. That's the hardest part for me: reading and understanding jargon-laden, dry papers is a specific skill that I, erm, need to work on (I find my eyes glaze over pretty quickly). It's also not an activity that can easily fit into my normal MOOCing times (on the bus/train? No way, requires much more concentration − in the evenings? Nope, requires a freshness of mind that I just don't have after a full day's work − ideally I'd get up an hour earlier and read during breakfast, but er… I value sleep, too).

In terms of content: the first few weeks are about the well-understood mechanisms (DNA methylation, chromatin structure and histone modifications, X chromosome inactivation, epigenetic reprogramming), then we get on to more controversial topics (environmental disruption of epigenetic state, for instance the effects of tobacco smoke, or diet, at crucial periods of time). The last week of the course is about cancer, which is pretty interesting (epigenetic modifications are one of the hallmarks of cancer − it's actually one of the very few common points of all cancer types: not knowing anything about the subject, I'll refrain from qualifying cancer as an “epigenetic disease”, but it's certainly tempting.)

The lectures are basically slides with an embedded shot of the lecturer (Marnie Blewitt from Melbourne University) in the corner. While I usually dislike this format, here it works well, partly because Dr Blewitt is a great speaker with a clear voice, but mostly because the slides are only outlines / supporting material for the course itself. In fact I find I hardly read the slides − I just skim them and concentrate instead on the audio.

Every week there's a quiz, which is fairly difficult − you have three tries, but the questions change between each try. As I said, the quizzes are often about specific points raised not in the lecture, but in the required readings, forcing us to read the papers, not a bad thing. At the end of the course, there is a peer-reviewed essay; I'm not certain how it will play out, it's been ages since I wrote essays (and then again, never in a scientific subject). I'll keep you posted about how it turns out.

So, in general, I like this course quite a lot, and it's certainly given me a lot of things to think about.

I guess I should have studied Astrophysics…

… as my final grade for the ANU ASTRO1x course is, well, reasonably good.

(That said, the grading scheme was rather lenient. While I do know a lot more about astronomy than I used to, I don't really deserve a 100% score. But still, it's always nice to get good grades.)

Notes towards an edX / Coursera comparison

A I wrote elsewhere, three major platforms seem to occupy much of the MOOC landscape: Udacity, Coursera, and edX. I've been meaning to do a write-up of my various impressions of the platforms based on the courses I've taken or am taking (counting the ones I've dropped, that's something like 15 on edX-based platforms, 6 on Coursera, and 1 on Udacity). Doing this write-up will take a while though, because I want it to be as good as I can make it; which will probably mean taking more courses, too.

In the meantime, I want to gather my thoughts, and sort of draft the main ideas I want to include in the final write-up.

Introducing the players

I'll mostly talk about edX and Coursera; I haven't really got enough to say about Udacity, and the model is different enough that it's not too easily compared to the other two. A few words of introduction though:

Coursera and Udacity are both commercial companies tracing their roots to Stanford University in California. Coursera has taken most of the limelight, gathering $85 million in funding and a very wide portfolio, over 600 courses, usually (but not exclusively) online versions of courses offered by various universities around the globe. Since they are commercial startups, a lot of the discussion around them revolve around business plans, licencing agreements, etc. Their models are different: Coursera is closer to classical university courses, with a predefined schedule, lectures, homework, etc. Udacity courses are offered on a subscription basis: you can take them whenever you like and take however much time you need, with a monthly fee.

In contrast, edX is a nonprofit organization. Its origins are at MIT, which was quickly joined by Harvard. It is governed by the edX Consortium, comprising the two founding institutions plus a large number of organizations (mostly, but not only, academicals).

Courses at edX are mostly organized in the same way as Coursera's, with predefined schedules and so on. A few courses are available on a self-paced basis, though.

Note that “edX” can have multiple meanings: either it is the edX organization itself, or the edX software that powers the platform. The latter is free and open source, and in wide use by multiple organizations, such as France Université Numérique (the French MOOC platform) or MongoDB University. Interestingly, it seems that Google is a contributor to the OpenEdX platform, and that Stanford (from where both Coursera and Udacity have originated) has also put its weight behind it.

Institutions

Coursera has by far the widest course catalog, more than 600, from many places around the world. They seem to have a great deal of partners in the US state university systems, in Australia, in China, etc. but also from Europe (hello, Centrale Paris!), which is a feat. Partners are mostly universities, but there are a handful of non-universities as well, like the World Bank and the National Geographic society.

Topics covered by the courses are extremely varied, and range from advanced mathematics to “do you have what it takes to become a vet?” Chances are good that, if you are interested in a topic, you will find a relevant course on Coursera.

Udacity seems to have refocused on hands-on professional training built in partnership with companies (Google, Facebook, etc. have all classes on Udacity), although there are a few university-sponsored courses too.

edX courses come from its partners, first and foremost MIT, Harvard and UC Berkeley. Presently, most courses seem to come from American institutions, although that's changing, with courses from India, Latin America, Japan, Australia, coming on. Europeans are lagging a bit: the edX powerhouses seem to be TU Delft (Netherlands) and Université Catholique de Louvain (Belgium). Overall, I think it's something like 175 courses that are offered on edX; generally, you will find introductory courses in about any subject, but more advanced courses are sometimes lacking (unless you're in IT).

Software

Coursera's software is very polished. It is quite obvious where all the funding has gone: the platform is very easy to use, it's simple to browse through the catalog to find interesting courses, one can “watch” courses for upcoming sessions, the platform itself does some simple analytics to suggest courses, etc. There are even mobile applications for iOS and Android which allow students to download lectures and watch them at their leisure, something which is extremely handy for people like me who watch lectures while commuting to work.

Course delivery is also quite polished, it's fast, there are few or no bugs, but I kind of dislike the layout. It's organized by activity type, so you have separate menu entries for lectures, quizzes, peer-reviewed assignments, surveys, etc.; each of these pages will have content added to it as course segments are released. On the one hand, students are free to organize themselves, although it seems to me that they'll just go through the elements sequentially (first watch all the lectures in one go, then browse the required readings, then take the quizzes, etc.) On the other hand, it's messier, I feel, and it makes it harder to keep track of what's to be done at which time, and in what order.

The course dashboard helps: some courses have counters on it recapping activities (“you have watched 16 out of 23 lectures”). Interestingly, the latest/most active forum threads are summarized on the dashboard too, so it feels more like the “course hub” that it's supposed to be.

Course activities seem to be limited to lectures, readings, quizzes and peer-reviewed assignments. It may be because I didn't take too many courses − I suppose programming courses have support for sandboxes, for instance. But generally, I find the “work” part of Coursera courses rather disappointing: it's often limited to checking boxes in quizzes.

By contrast, edX feels a lot more open-sourcey[1], very flexible but rough at the edges. It may be that they just haven't had as much money to invest in the software as Coursera have; being a non-profit funded by universities, they're unlikely to have $85 million to spend. So the “home site” experience is far from as good. Browsing through the catalog is a mostly-manual chore (there's no search engine, just categories). The student dashboard is a shambles. You have to keep track of what courses start when manually (I use a spreadsheet…), same for deadlines.

The course experience however is a lot better. The courseware is organized linearly, mixing blocks of various nature. So you can, and do, have lecture sections interspersed with knowledge-checks, followed by worked examples, practice problems (which can be of any type), then homework assignments. Lectures are still lectures and so, a mostly passive experience, but the homework can be basically anything: quizzes, formula input (with a nice MathJax integration), advanced interactive tools (7.00x had us cross fruit flies in order to produce populations with a specific phenotype), code execution (CS169x ran unit tests on our code to check its compliance), etc. This makes for a much, much richer learning experience.

The edX forum software sucks big time, but it has a redeeming quality: it's possible for course designers to embed discussion forums in courseware pages, thereby fostering discussion of the topic under scrutiny.

The overall feeling that I have is that course designers have a lot more latitude with edX, when Coursera has more of a “one size doesn't quite fit all” approach. That, and the fact that edX courses tend to come from higher-profile institutions (MIT, Harvard, Berkeley…) that may have more effort to invest in a particular course, means that as a rule of thumb, I go to edX first and only fall back to Coursera when I can't find what I need / want on edX.

[1] disclaimer: I am a big open-source user and advocate. I'm typing this from a Linux Mint desktop.

Stat2.2X is passed!

So, I can do Probabilities.

To be honest, I didn't think I'd get this one − I mean, I joined on the third week of a five-week course (so I'd missed 40% of the course); worse, there was a midterm on the second week which I'd already missed. I registered anyway, because I wanted to learn about the subject (I am still convinced that statistics and probabilities are the most important maths one should learn at school after arithmetic) and because I intend to take 2.3X, “Inference”.

But thanks to the grading scheme (50% of the grade is contributed by the final alone, plus only the four best homework assignments, out of five, count towards the grade) I made it. Since edX have adopted the policy of handing out certificates to everyone, that's my, let's see, 11th edX certificate secured.

(I made a grand total of 2 errors in the course; both of which I could have avoided if I'd paid a little attention.)

EdX's enrollment options

At edX, when you enroll on a course, you pick your track between two options… although there are often, in reality, three. Let's review them:

• “Audit” is the most basic. Gives you access to the courseware, the exams, etc. but you don't get a certificate of completion should you “pass” the course − what it means is, you don't feel the pressure to do exams (which can actually be detrimental to the learning process). It's free.
• “Honour code” (or “Honor” as they write it, but hey, I'm European and half-Brit) is the basic certificate-granting level. You do everything, then if you do well you get a nice little PDF saying that someone purporting to be you completed the course. It's also free.
• “Verified Certificate” means that you send official proof of your identity (in fact you hold up your passport to your webcam, you take a picture of yourself with the same webcam, someone at edX matches the pictures. You may have to take additional pictures at random times during the course to prove it's still you, but that's not happened to me yet.) So the PDF now says that someone took the course and they did check it was you. There's a fee involved, which is decided on a per-course basis. The cheaper courses start at $25 a pop, and I've seen courses asking for ten times that. Besides, that's a minimum fee − remember, edX is a nonprofit. When you pick the Verified Certificate track, you can pay whatever you like as long as it's above the minimal fee.

Not all courses offer Verified certificates; not all offer Honour code certs either, but nearly all do so we won't mention U. Washington's “deal with stress” course.

Now when you sign up for a course that does offer the Verified option, you get a choice between “Audit” (free) and “Verified” (paid). But what a lot of people don't realize is that the Honour code option is still available! Just pick “Verified”, but instead of choosing an amount to pay, click the checkbox saying “What if I can't pay? Choose an Honour code certificate instead.”

So that's how it works. Or worked.

A couple of days ago, edX rolled out a release including a number of improvements, including a Dashboard update that now states clearly what track you're on for each course. So it became very clear to a lot of people that they were “auditing” courses − and that they wouldn't get the coveted certificate after all. A couple of days' worth of grumbling, and voilà: I saw in two places (including a Course Info note by the actual instructor) mention of “a change in policy”: auditing students will get certificates, too.

So, dunno if that's due to the uproar, or if the whole three-level registration is too complex anyway. It may be the end of the distinction between Audit and Honour Code.

It's also a case of convergence towards the model of the competition: Coursera has only two levels, “join for free” and “Signature Track”. You get certificates (sorry, Statements of Accomplishments) either case, but only by signing on the paid-for Signature Track is your certificate verifiable online (edX provides authenticity validation to all − so if you see an edX cert, you know that someone really took the course; you can't be certain who. If you see a Coursera statement, you only have an easily-tampered-with PDF that proves nothing.)

So, what to make of it? Well, I didn't really see the point of Audit anyway (maybe, I thought, if you don't go through with the homework and all but only watch the lectures, then you don't get the big bold “you only scored 9% on this course, while 85% is required for a pass” message on the archived course on the dashboard). So, kind of glad if that's going away.

In the short term, it means I still have a good chance of clinching that Stat 2.2 certificate (though I'm auditing the course, having joined late). So I'll be trying the final this weekend after all.

15.071x The Analytics Edge is over; now what?

I had passed the, well, passing threshold a while ago, but it's always nice to get a high overall score. What I'm surprised about is my competition score, I thought I'd done quite poorly but I still got 82%.

So in the end, the final took me about four hours, and I made a number of stupid mistakes. Mostly because I'm absolutely knackered − I shouldn't, really, have attempted the final when I am well less than firing on all pistons, so to speak, but hey, aside from my amour-propre, there was no risk as I'd already secured a pass.

In total, it wasn't the most fun course I've taken, but it's certainly one in which I've learned a lot. While certainly not fluent in R, I have enough basic skills to fool around and try and analyze data; plus I now have a decent understanding of what data analysts do, which is a plus. While I'll never become an actuary or something like that, there is little doubt those basic skills will be of use in my future life.

So… this being out of the way, what now? This week also finish the Diabetes course from Copenhagen and the Statistics course from Berkeley; I've secured a pass for the first one, as for the second, edX's recent release reminded me that I registered as an auditor (because since I'd joined late, I didn't think I could actually pass the course − while I perfectly could). So I think I'll give the final a miss.

Ongoing courses are now: MongoDB M202, Epigenetics in Melbourne, and Astrophysics in Canberra. And that's all until June 2nd, when the Georgetown “Genomics Medicine Gets Personal” and the third part of Berkeley's Statistics course start, followed a week later by MIT's 7.QBWx.

But overall, the MOOC pressure is going down. I'll be doing 3 courses for the rest of May, 4 courses in June, then it drops back to 3 in July; August has only one course scheduled (Exoplanets). Compared with my current 6 (or 7, if you really want to count the joke that is Social Physics), it'll feel like holidays. Nice; I could do with a rest.

Postmortem: MAS.S69X Big Data and Social Physics

So I've “taken” this “course”.

Which is to say, I watched five short (under ten minutes each) videos introducing the content of some chapters of a book. And answered “yes” to the question, “did you watch the videos?”

For that I'll get a certificate. Huh.

Basically, this “course” feels like a trailer for a book. The ideas mentioned are enticing; I may buy the book sometime in the future. But huh, I thought I'd signed up for a big data course, where I'd get to learn some skills and/or theory, about, y'know, big data applied to social behaviours. I feel kind of cheated.

I suppose it's a kind of experiment − by providing a social reward (the edX/MITx certificate, the occasion to discuss big ideas on the edX forums) and wrapping it under the guise of an “MIT class”, maybe Alex Pentland and team are trying to see if the sales numbers for the book skyrocket? It's one of their basic tenets anyway, that social rewards are better incentives than economic rewards.

So anyway, don't go there. It's not worth it, unless you're curious about the Social Physics book and considering buying it anyway − you can view this “course” as a kind of glossy brochure for the book.

PS: though that's an edX course finished and a certificate I'll get, I'm not entering it in the Completed courses page on this blog; it's so far from deserving the title of “course” that putting up that certificate on my wall would be intellectual fraud on my part.

edX turns 2

So edX will be exactly two years old on May 15th. I'll be celebrating my own way, I guess by catching up on Astrophysics or revising for the Statistics final.

Veni, vedi, t-shirti.

So, what's edX for me? Quite a lot.

I jumped on the MOOC bandwagon in September 2013; well under a year ago, though the story begins in the summer of 2013. Having read a couple of pop-econ books and sort of got into my mind that I'd like, one day, to get some sort of formalish background in the subject, I was made aware that MIT had this OpenCourseWare thing going − and hey, what did you know, they had an Introduction to Microeconomics up! So I'd loaded up my smartphone with lectures, and watched them on my daily commute, and loved every single one of them.

(Now you people of MIT and edX, if you can convince Jon Gruber to host an economics MOOC, that'd be wonderful − Caltech's Principles of Microeconomics with Calculus, being way more austere and technically oriented, cannot fill in the role of an Econ 101 for everybody.)

So when I was done with 14.01SC and found my commute rather empty, I browsed through OCW and found a Fundamentals of Biology class, and thought hey, I've always regretted choosing math/physics over bio when picking a higher ed path (mostly because I wasn't properly aware that yes, you can study biology and not end up either in med school or working for Nestlé), so I started rooting around, until the big banners did get through to me that hey, a more up-to-date version of this course, with added homework material and a certificate at the end, was available through something called “edX”.

Boy did I know what I was getting into.

7.00x was great, so great that I signed up for Harvard's SPU27x (a smashing course that changed the way I envision food, and y'know, I'm French and all that) and UT Austin's Age of Globalization (thinking I would get a 14.01SC-like mathematically-minded analysis of globalization; sadly that was not to be). Then Berkeley's CS169 came along, and well, I'd wanted to find out what this Ruby on Rails thing was like for years. I signed up for MCB80.1x because neuroscience is this kind of hip thing right now, isn't it?

When that batch was over, I was hooked. I signed up to Ec1011x out of nostalgia for the MIT introduction to microeconomics, and this is to date the course I've worked the hardest on. I registered on Coursera to take the Epigenetics course that's just started − because there weren't any courses on edX to follow up to 7.00x. The idea that I might reorient my career towards a mix of computer science and biology/medicine started to grow in my mind, so I grew more methodical in my choices: I knew I needed a thorough statistics refresher so I picked Berkeley's Stat 2.1x (and later, 2.2x). The R statistical environment seemed like a good thing to learn − MIT's Analytics Edge course, here I am. Bioinformatics looked like the thing, so (there being nothing on edX) I took Peking U's course over on Coursera. And so on. (I tend to favour edX courses over Coursera / Udacity ones because I prefer the site layout, and more importantly because I feel that as a non-profit, it's intrinsically better − there is much less chance of seeing my profile data resold, or free stuff becoming paying, to start with.)

Eight months after signing up to my first MOOC, I am doing six courses in parallel, half of which through edX. I have 8 edX certificates to my name, two more are already secured and at least another one should be mine by the end of May. My edX dashboard, mixing past, current and future courses as it does, is overflowing with 20 entries (I am seriously considering forking OpenEdX on Github, refactoring the dashboard so it works for people with > 10 courses, and doing a pull request.) I brought back my old laser printer from the cellar so I could print out Immunology outlines and revise them on the bus. I plan my holidays around course schedules. I have started a blog about MOOCs. I generally finish each week mentally exhausted.

Less than a year into this MOOCing game, edX has already had a bigger impact on my life than, I don't know, any Web organization besides Google (to give some perspective, I've dropped Facebook to make more time for MOOCs, and feel all the better for it.)

All this to say, happy birthday edX!

(I'm unlikely to make it for the livestream, unfortunately, though I'll give it a try.)

MOOC forums - why they suck and how they could suck less

In my recent Notes from the Trenches about the Copenhagen Diabetes MOOC, I digressed a bit about the rather irksome “guided forum discussion” (anti-)pattern one sees occasionally in classes; sometimes, it's even compulsory and participation (any participation) counts towards your grades, such as in UT's Age of Globalization course.

Why is this irksome? Because it plain doesn't work. Encouraging or forcing people to communicate is awkward at best; when the population count is in the tens of thousands, it means you end up with a lot of one-liners, “me too”s, clichés and platitudes.

Take the Diabetes MOOC. One of the discussion topics this week was “should we systematically sequence the genomes of newborn children to identify risks?” Within a couple of days, roughly two-thirds of the answers were along the lines of “no that's eugenist, you big Nazi you”. About half of the remaining posts were from one single “superposter” who apparently took it as his holy duty to answer as close as every single post as possible. The rest may have been thoughtful, intelligent responses but by that time my eyes had glazed over. I didn't even try to raise the level of the debate (yet, based only on the lectures, there was a lot to say about the − poor − predictive abilities of genome sequencing, false positives, etc.)

UT's Age of Globalization had mandatory forum participation. Well, that's a humanities course, I guess student interaction is pretty much an integral part of the point, so pointing towards the only interaction medium there is seems logical; however in the end we had the same behaviour: most people would just upvote an entry they liked, or respond with a one-line “me too!” or “no you're wrong”, and get the 1 point credit towards the certificate.

What it all leads to is a terrible signal/noise ratio, which is a powerful disincentive for quality participation. Thinking through an issue, weighing the pros and cons, imagining the various points of views takes time and effort; seeing that all this effort is systematically rewarded by either nothing or a short “thanks, that was very interesting” (or worse “that doesn't make me change my opinion”) that doesn't bring anything to the discussion, is disheartening.

The problem is compounded when the forum software itself is not so great. Take a look at edX's forum:

Some glaring issues:

• Very, very poor use of screen real estate. The screenshot above shows the totality of the forum's landing page for 15.071x, on my desktop computer which has a 27", 1920×1080 pixel display. Two million available pixels and edX only shows 8 posts − half of which are pinned staff announcements that once read, are of no interest whatsoever to students. So that leaves… yeah, four posts being displayed. Hurray.
• Actually when you click on a post, the height of the forum, including navigation pane, increases to the minimum of display height (minus some padding) and total post (with responses) height. Huh, what? That's really inconsistent from a UX point of view − the navigation pane's display should never be dependent on the detailed view. Makes the display jump around unpredictably.
• The scrolling behaviour between the two panes is a mess. It's such a mess that I don't even want to think it through.
• The forum structure (subsections) are hidden from view; by default the forum shows “all discussions”. It's far from obvious that there even are subsections, and that one should use them.
• There is no moderation worth of the name: basically the TAs just delete posts that violate the honour code (such as posting full homework solutions) but don't do the necessary housekeeping (such as moving threads to the correct subsection). Maybe there's moderation of flamewars, as I didn't see any, but I would pessimitically just think people can't be bothered to flame each other.
• The navigation pane's entries are: subject, number of upvotes, number of comments. And that's all. No mention of first post date, last comment date, first post author, last post author, whether staff have answered the post, etc.: all elements that can pique the interest of potential readers.
• Sorting is very basic. The most obvious missing options are “unanswered posts” and “most active discussions”.
• There is upvoting of posts, but no downvoting.

What this means is that the most active threads are the “Eeeee I'm from Wherever and really excited about this course!”, hello-introduce-yourself ones, and a couple of staff-originated course-related threads. Not so great for a community.

Some edX courses also suffer from their being split into multiple parts − meaning that the forum dynamic, if and when it has been established, has to restart from scratch every five weeks. Huh, a month is not enough for a community to assert itself, you know?

As usual, Coursera's software is a little better finished (either Coursera is much better funded − which is likely, the company has raised something like 85 million dollars according to Wikipedia; edX being a non-profit can't raise venture capital and relies on contributions from, as I gather, MIT, Harvard, UCBerkeley and Google) so most of these “glaring” issues are avoided, but there are still no “view unread” / “most active” filters; one has to navigate to the subforum one wants to check out the discussions there. Generally the end result is the same: slogging through the forums is a chore, and the signal-to-noise ratio is abysmal.

To be fair, the edX forums have one redeeming virtue, and that is the possibility to link subforums to courseware pages. What this means is that in practice, there are mini-forums embedded in the courseware pages, specific to that content − for instance, in ANU's ASTRO1x course, the weekly “mystery” page has its own subforum; in a limited way, that lets student skip the tedious forum navigation. (They also have good MathJax integration, letting people enter beautiful formulae).

Part of the problem is that the platforms just recreated (from scratch, with limited means… nuff said) forums following a dying model anyway. Online forums in the traditional sense have had their glory days from about the early 2000s (when they finally took over from newsgroups) to the early 2010s (when they've been displaced by Facebook, Google+/Hangouts, etc.) In 2014, for good or worse, they're not an appropriate medium anymore. Users are not used to the discipline of finding out the best subforum to post to, then slogging through the forum archives to find unanswered posts, watch out for replies, etc. They're used to the little red thingummy on Facebook telling them someone's replied to their posts.

Others have already written about how to solve the problem. I'm not certain that just waving the Web2.0 wand will magically make all troubles go away, but certainly, one can take a leaf off some social sites. Embedding live chat can be a good idea; it was tried for CS169 part 2, I think, unfortunately they picked a heavyweight JS IRC client that slowed down the pages tremendously and often made them crash altogether (it was kind of ironic to see such crappy, badly-tested software being put together for a course about software quality…), so after a few days a TA put up a note on the course forums giving instructions for how to disable the chat component completely. A worse admission of failure I have rarely seen; but the idea has some merit. A component integrated with the platform (rather than third-party) might work, especially if some thought is given to screen real estate usage (small font sizes, less whitespace, and you could fit a narrow chat column to the right of an edX courseware page even on a 1280px wide display − let's not even mention responsive design).

Having a look at how StackExchange works might yield some ideas − cleanly separate random chat from serious questions/answers. Make it easy for people to find if a question has already been asked, find unanswered questions, kill  (through downvoting for instance) duplicates and obviously worthless posts (like the individuals demanding special treatment). Have a look at how Discourse do things. Throw in some social media features (add specific posters to watch list, to make sure one never misses a post from JonPowles on the ASTRO1x forums; conversely, blacklist people who tend to be irritating) to be in the mood of the 2010s. Perhaps add direct messaging if you must.

Connect courses together. It's highly likely that students who took Introductory Biology will move on to more advanced biology courses; letting people follow through with their mini-social network already set up would be nice − and the possibility to refer to discussions that've already been had in other courses would be handy. I was quite surprised when taking an end-of-course survey, and one of the questions was “Do you intend to keep in touch with fellow participant?” − I realized that I couldn't actually name a single fellow participant. Also, why would I want to keep in touch? I don't know anything about them, in all likelihood we have nothing in common apart from a shared interest, at a specific point in time, in a specific course.

(I thought that segmenting the forums − basically create “bags” or “classes” of a few thousand students − might help too, but probably would do more harm than good: all of the dynamic online communities I've seen have had a core of a handful of stalwarts. I'm not sure the number of high-value individuals scale linearly with the size of the community, so segmenting would prevent these individuals from connecting with each other and create a good group dynamic. Besides, cutting up a 200,000-strong cohort into a hundred 2,000-strong classes would mean TAs have 2,000 times the moderation / participation workload.)

So, yeah, building decent forum/community software is a significant endeavour. It takes time, and effort, which the various providers may prefer spending on other things (e.g. fixing edX's dashboards, or the horribly slow math input fields; or indeed work on XBlocks so future courses may use more advanced tools). And I know, edX's open source, so why don't I just fork the code, fix it, and do a pull request? Well, apart from the fact that it is a significant endeavour and that even if I were to find the time and motivation to build such a thing (not to mention that my Python skills are inexistant), my PR would almost certainly not be merged (it is highly unusual that far-ranging changes that come out of the blue are accepted by open source projects, especially ones as active as edX − there were 31 merges on May 9th alone).

As a final note, I'd be interested to see how NovoEd does things. They're branding themselves as a “social environment for online courses” rather than as a “platform for creating and hosting MOOCs”; they will have given much thought to community-building. Unfortunately, the courses they're offering (mostly around finance, entrepreneurship, and startup creation) are of little to no interest to me, and I'm not quite ready to register to a boring course just to see how they're doing forums.

Random musings: On Platforms

MOOCs are here to stay; that is a certainty[1]. But we're still in the early stages of the MOOC “revolution”: a new ecological niche has opened up, we are in the middle of a diversity explosion, but sooner or later the number of players will go down and each will differentiate.

(To continue with the bad analogy, the dinosaurs have died out and we're in the diversification phase: Coursera's the birds, edX's the mammals, OpenEdX's the metatherians − closely related to placental mammals but different − Udacity's the crocodylians, FutureLearn's the bony fish, etc.)

Sooner or later, the number of platforms will go down as “the industry” converges to a handful, sort of like happened in the social network space: gone are the Orkuts, LiveJournals, and MySpace of the past; only Facebook and Twitter (and Baidu in China) remain, and the two are sufficiently different to not be frontal competitors.

Listing the MOOC platforms out there is something many people have done, so I won't replicate for replication's sake. Safe to say there are many. They are subtly different, though in reality, the variability of individual MOOCs is often greater than the variability in the platforms hosting them.

If we take a cursory look at the megafauna in this landscape, we have two major beasts competing frontally, a third player which is different enough to carve its own niche, and a slew of minor players. A full review isn't what I intend to do here (I do mean to do a writeup of edX vs Coursera at some point, though), I'll just give some highlights on the platform types out there.

Coursera and edX are the big beasts, very similar in terms of structure. Between them, they must hold down a massive proportion of the MOOC market (70%, 80%? I have no idea.) Universities create courses with a mix of video lectures, text resources, homework and exams; students enroll to specific sessions of a course, on pre-set dates, and if they pass a certain threshold, they get a certificate. For certain courses, students may opt for a “verified” certificate (for which the platform takes minimal steps to ascertain the student's identity, for instance by comparing webcam images with a picture ID like a passport). There are even a handful of “sequences” of courses leading up to an overall certification in a specific field (they are called “Specializations” in Courserese and “XSeries” in EdXian). The similarity is sufficient for there being cross-talk: Caltech's Principles of Economics with Calculus course ran once on Coursera then migrated to edX.
There are differences between the two (Coursera's more mature overall, edX's more flexible) but switching from one to the other is mostly transparent for students − it's the actual course that's important.

Udacity has a different model. First off, it's a for-profit business (Coursera's one too, but less obviously so as they're at the gain-market-share-lose-money stage of startup development; edX's a non-profit); the focus is much more clearly on paying customers. In terms of content, it means the courses tend to be more oriented towards building specialized professional skills than towards acquiring general knowledge of a field − Udacity teams up with businesses to offer classes on specific products, for instance. (There are also more general, less immediately “useful” courses, but they do seem a bit lost in there.) The classes themselves are self-paced: students may take them whenever they want, over as much time as they need. To access the “full learning experience” (which I understand means access to TAs and certification) one can pay a monthly subscription fee.

Then there are national endeavours such as the UK's FutureLearn, France Université Numérique (actually built on top of edX software), Australia's Open2Study, etc. I am sort of confused why these are actually needed, and indeed some universities hedge their bets (it looks like most Australian universities have courses up on either edX or Coursera). I guess the point is to integrate into the national higher-education system, so that a − say − French student can mix and match courses for her local university and FUN[2] and still gain credit for both? We're not there yet; if that is indeed the point then that will be an exciting development, but in the meantime the national platforms look like something very unnecessary indeed.

Saylor Academy deserves a special mention. They don't actually create course materials, but rather use the wealth of material that's out there (from OpenCourseWare to YouTube videos via freely-accessible textbooks) to package coherent courses and full curricula, going a full step further than Coursera/edX's “sequences”: by mixing and matching existing resources, you can actually get the full equivalent (content-wise) to a traditional US college 16-course major. Saylor does the packaging, exams, and certification.

Possibly what is missing (or which I haven't found) is a “MOOC navigator” site, something that will suggest pathways through MOOCs (e.g. for a biology education, “take edX's 7.00x for introductory biology, then hop on to Coursera's Useful Genetics, pick up a couple of biochemistry courses here and there, a stats course here…”) − that is to say, go above the level of individual courses and establish curricula, taking into account things such as starting dates, etc. Actually, a “skill tree” as in many MMORPGs might do it. Kind of a metaplatform, really. Hmm. That might be an interesting side-project for when the days finally decide to last 36 hours to give me time to actually do stuff.

[1] What I am not so certain about − and that's the exciting part − is the impact the MOOC “movement”, if you want to call it that, will have on both the educational system as a whole, and the workplace. Speaking from a French, occasional recruiter point of view, one of the first things I look at when looking at a resume is the school the applicant graduated from. It's not the only thing and it's not discriminatory, but it's a better-than-null-hypothesis proxy for the overall “quality” of the applicant, especially if they graduated less than five or ten years ago.

That said, the French system is intensely competitive, with a lot of engineering schools (204 are officially allowed to deliver a diplôme d'ingénieur, not counting universities), often quite small (most have classes in the ballpark of 50-100 students a year), admission to which is based on a handful of nation-wide competitive examinations. Specialized (and not-so-specialized) media maintain a nationwide ranking of these schools, updated on a yearly basis. By this system, the “better” schools (i.e. better-ranked) naturally attract the “better” students (i.e. those that are better than the rest at taking gruelling exams they have spent the previous two or three years preparing for).

All in all the French system is exclusive − only the very best go to ENS or Polytechnique, the next best go to Mines or Centrale, etc. It makes a decent guarantee that a graduate from a toppish-tier school is a very clever person with a great capacity for hard work. This is fine and dandy, but what it doesn't do is guarantee that graduates from a lower-ranked school or a mainstream university are not good and indeed many very apt students are excluded (in technical terms, the sensitivity is decent but the specificity is awful; there are few − though not zero − false positives but a great many false negatives.)

MOOCs by nature are inclusive. I tend to believe that people who are willing to stick to the sometimes heavy schedule of a MOOC − and even better, many MOOCs − show themselves to be highly-motivated individuals which may be of great interest to companies. But that's a gut feeling; only time will tell, if and when MOOCs go mainstream (I've only once seen a resume mentioning a MOOC).

[2] I wonder whether the “France Université Numérique” name was picked specifically to bring a little FUN in the university system?

Random musings: What makes a good MOOC?

All MOOCs are born equal in dignity, but not in much else

Having completed or being well on my way to complete about 15 MOOCs, and tried a handful more, it's evident to me that no two MOOCs are the same; which is right and proper, the organization of the course being, after all, down to the professor's preferences − and also of course subject to the specificities of the subject matter.

All MOOCs are not created equal, then, from a purely objective point of view. Subjectively, they are not all the same either. Some MOOCs make a strong impression, others are bound to just fade away, others still are fated to be dropped. It is therefore a valid question to ask what makes a good MOOC.

The ideal analytical method

Scientifically, the correct way to answer that question would be to describe each MOOC with a fair number of objective characteristics (we'll come to that), then ask students their impressions on the MOOC at least three times: once in the first two weeks (before the “second-wave drop-off”[1] kicks in), once around the end, and once a couple of months afterwards (once the euphoria of having passed the course is over and people have had time to start forgetting the course − I assume here that the principal objective in taking MOOCs is to learn things for the longer term rather than get a certificate and quickly forget everything). So for each course, we have three outcomes: a “stickiness value” (do students stick to the course?), an immediate appreciation score, and a medium-term appreciation score. Using all that data, we can build classification regression models to find out what characteristics are predictors of our three outcomes.

(We could also try to cluster the students according to their preferences; and then tailor courses towards a number of student profiles, eventually.)

But then of course, I don't have a large dataset. I have, in fact, a piss-poor sample of just myself, with all the biases that entails and less data points than classification criteria. So let's forget about the scientific methods for now, and just review some of the defining characteristics of the MOOCs I've taken and from that sample, try to draw conclusions about what I like and what I don't like in a MOOC.

What's in a MOOC?

(That which we call a great course, by any other token would smell as sweet… er…)

Here are our classification criteria:

• Real course: is this course based off a “real” course at the institution in question? Levels: No, Partly, Fully, Actual course runs concurrently;
• Multi-part course (e.g. Statistics at Berkeley);
• Part of a sequence of courses;
• Length of course: in weeks (for multi-part, take the total)
• Lecture type: Classroom videos, Talking head, Talking head with infographics/animations, Slides with voice-over, “Slate” (or tablet-style). In case of a mix, we'll just go with the most common type. (In a thorough analysis we'd have to quantify the mix, e.g. 40% talking head, 60% slides with voice-over);
• Native English-speaking lecturer(s): Yes, No. (In a full study we'd have to switch this to native speaker of the course language, and add some data about subtitles in other languages);
• Speed of delivery: does the lecturer speak fast or slowly? To simplify, I'll cluster in the “fast” group the lecturers who significantly use hand/body language as accents;
• Tone: Conversational, Dry, Passionate (note that the latter is highly correlated with a fast delivery, so that might not be such a great criterium);
• Printable resources: None, Transcript, Lecture Slides, Outline/summary;
• Links to additional material: e.g. research papers
• Length of video segments (average): 0-5 minutes, 5-15 minutes, 15-25 minutes, > 25 minutes
• Length of video segments (maximum): 0-5 minutes, 5-15 minutes, 15-25 minutes, > 25 minutes
• Assignment difficulty: from 1 to 5: 1 is very easy (immediate answer), 5 requires multiple hours of work, roughly, so that's not so simple: quizzes are quick to answer but may be hard, interactive tools may take longer to use while the problem is simple overall. So it's a mostly subjective point.

The rest of the criteria are simple yes/no:

• Quick questions between video segments
• Ungraded practice problems or worked examples
• Homework: quizzes
• Homework: numeric / formula input
• Homework: essays
• Homework: code / programming / offline tool using
• Homework: online, interactive, custom tools
• Homework: multiple tries allowed
• Midterm(s)
• Final exam
• “Collaborative” focus on assignments: e.g. dedicated forums for each problem in a set, etc.
• Guided discussion on forums

Note that I count as one “course” the sum of all parts of a multi-part course. For instance, CS169 SaaS at Berkeley is only one course (it doesn't matter that students get two certificates). But self-contained courses that are part of a series count as distinct courses. There's a part of subjectivity here, but broadly, I count CS169 and Stat2X at Berkeley, or BIOC372x at Rice, as multi-part, while I have ANU's Astrophysics courses and Harvard's MCB80x courses, as multiple courses in a sequence.

My MOOCs

So that's a good handful of criteria! Let's see how (some of) my MOOCs fare. I've actually tabulated it all but Blogger doesn't seem to support uploading of random file types, so until I find someplace to host it, here's an unscientific look at the data:

The top-scoring courses (MIT's 7.00x, Berkeley's CS169x, Caltech's Ec1011x, and ANU's ASTRO1x[2]) are a mixed bunch. They all have fast speakers with either a passionate or conversational tone. Apart from the Astrophysics class, they are based on actual courses (Caltech's one was even run concurrently with the on-campus class). The first two had classroom videos, Caltech's was mostly slate / tablet, ANU's is about 2/3rds of the time lecturers in front of infographics and 1/3rd slate.

As far as homework is concerned, they all allowed multiple trials for most of the problems. The nature of the homework is varied: 7.00's shone through its use of custom interactive tools, CS169 was heavily about programming (well, that's the point of the course, innit?) and the other two are focused on numeric or formula input. Apart from ASTRO1, the difficulty or homework duration was very much on the high side; Ec1011 was clearly the most difficult course I took and I did spend many hours on the other two.
All courses put a focus on collaboration between students, by putting down links to the relevant forum sections on the appropriate pages.

Individually, the courses shine in different ways:

• 7.00's excellent lectures, additional videos, and immense wealth of tools make it by far the best course ever, be it a MOOC or an actual class, in my experience. To say I'm anticipating this summer's 7.QWB with some trepidation would be an understatement; I wish MIT's Biology department put up an XSeries.
• CS169 was good because of the subject matter, of the passion the lecturers put into the course, and because of the programming assignments. There were a couple of quizzes that were broadly speaking a let-down. I also appreciate that there is no exam: the homework is sufficient. The forums were pretty good too.
• Ec1011's big selling point is the homework's difficulty. You get to spend many hours on it every week (roughly speaking, I spent all Saturday mornings doing Ec1011 homework for the duration of the course, sometimes overflowing well into the afternoon); Prof Rangel's philosophy of “mastery teaching” is great: you get a large number of trials (10) for each problem and you're encouraged to discuss the problems on the forums, as long as you don't actually post the solutions. Overall, it means you get intimately familiar with the (albeit simple) models economists use.
• ASTRO1 is not so much about grading, as I wrote earlier, so the homework tends to be ridiculously easy. But the lectures are great (thanks to the lecturers' enthusiasm), the accompanying material (reference notes, worked examples, etc.) is very good, and the most brilliant idea is the weekly mystery that I've mentioned before; along with the accompanying forums it means we do some intriguing collaborative problem-solving that integrates everything we've learned in the class.

Rice's BIOC372.1x narrowly misses a top rating because of the quiz-based homework with only one try. That doesn't mean it's impossible to get a good grade (I did), but it makes doing the quizzes a chore more than a learning opportunity in its own right. I appreciate the nature of Immunology means it's more about memorizing things than acquiring problem-solving skills, but I'm sure there are ways to make the homework less annoying.

Harvard's SPU27x gets an honourable mention too, although I dropped the homework (wasn't interested enough) and downgraded to auditing the course rather than passing it. Basically, the course itself (teaching about science through the medium of cookery) is a great idea, and the demonstrations by guest chefs were great. Some of the labs were kind of interesting (molten chocolate cake has become a household classic) though I skipped through most (I wasn't particularly interested in measuring the elastic modulus of steak as it cooks, for instance).

The worst classes are the ones I dropped, i.e. Mount Sinai's Introduction to Systems Biology, Toronto's Bioinformatics Methods, and Harvard's PH525 Data Analysis for Genomics. Toronto's I won't go into too much detail about, basically the course wasn't what I expected or needed (it's better seen as a hands-on companion to a bioinformatics course; I guess I could try it again now I've finished Peking U's introduction to bioinformatics).
Mount Sinai's Introduction to Systems Biology suffered from long, purely slide-based lectures with a voice-over delivered in a sing-song voice, explaining (badly) ideas that are very complex in nature, making me feel completely out of my depth; I had to watch each segment two or three times to gain an understanding. The poor quality of the recordings (you could hear background noise such as police sirens driving by…) didn't help a bit. There wasn't really any homework besides a weekly quiz and a couple peer-graded questions. I clung on for two weeks then decided my sanity was worth more than that. It's a shame, as (in the absence of a MIT Biology XSeries…) Mount Sinai's Systems Biology 5-course Specialization looks like the best match for where I'd like to take my career (somewhere on the intersection of computer science, big data, and biology/life science).
Harvard's PH525 is a different kettle of fish. I just didn't have the mental bandwidth to commit the required effort to the course. The lack of actual homework for the first two weeks (just “understanding checks” in the form of quizzes) didn't help me getting involved. Also, Prof Irizarri had a tendency to sway from side to side in the lectures; since I mostly watch lectures on the bus or train, it made me seasick[3].

Conclusions and reflexions

Based on that skewed analysis of the MOOCs I did, can I draw conclusive, er, conclusions about what makes a good MOOC? Not really, but I can put forward a few points:

• Classroom videos are better, as they make me connect to the course more. Failing that, “slate” (tablet-style) or lecturers-with-infographics (ANU's ASTRO1 is a good example of that) does the trick.
• Shortish (about 10 minutes) video segments interspersed with quick questions, please.
• The speaking qualities of the lecturers are obviously of great importance.
• Downloadable or printable resources are very welcome. Pointers to additional materials are too, but somewhat less.
• Homework should be seen as a learning opportunity in its own right, so rather than focus on checking that students have learned the lesson, they should be more in a problem-solving format. Homework should be hard (or at least, long), but multiple tries should be allowed and collaboration between students on the homework should be encouraged.
• I'm frankly doubtful about the overall utility of final exams in the grand scheme of things. I think something like ASTRO1's “mystery” is the gold standard: a recurring problem with additional hints every week, that allows students to integrate every lesson's knowledge in order to bring about final understanding. Note that this approach is very well-suited to programming classes, too.
• I am broadly indifferent to a course being offered all in one go or split into two or three parts. I suppose splitting means people are more likely to register (it doesn't feel like committing to 10 or 15 weeks' worth of work). I don't care much, if anything I'd prefer everything in one go (no need to register twice, no risk to see the second half of a course rescheduled to the other half of the year.)

Now to take this further… Anybody knows of good public datasets about MOOCs? Were studies made to measure student's opinions of MOOCs multiple months after the courses have ended?

---

References

You can find links to most of the courses I mention in the Completed courses page. As for the others:

• Harvard's SPU27x Science and Cooking
• University of Toronto's Bioinformatics Methods
• Mount Sinai's Introduction to Systems Biology
• Harvard's PH525x Data Analysis for Genomics

Footnotes

[1] This is totally unsubstantiated, but I'd think there are two initial waves of dropping-outs: one the very first week and possibly even before that, when students realize this course isn't for them (wrong difficulty level, wrong appreciation of the subject, etc.); and one closely afterwards, when students basically throw their hands in the air and decide that although they are interested in the subject, the MOOC itself doesn't fit their requirements (it is a “bad MOOC” from their perspective). Here, I am concerned about this “second wave” − why makes people give up on a MOOC on a topic they are interested in?

Of course there are other drop-out causes: lack of time, interference from the real world or indeed other MOOCs, a late realization that the subject isn't so interesting after all, etc. But these would tend, I believe, to be more or less evenly spread out throughout the course program.

[2] Sometime I'll just drop the systematic “x” suffix in edX course labels.

[3] Hey, I didn't say I had only good reasons to drop a course!

Notes from the trenches: Diabetes, a Global Challenges (Copenhagen University)

The Diabetes course from Copenhagen University is drawing to an end; only one week to go. I'll do a full postmortem when it's over (next week, basically), but in the meantime, some unordered remarks/comments:

Unlike most courses I take, I didn't register to this one to acquire useful skills or knowledge, but out of curiosity. I have, obviously, a specific interest in the subject matter (my son has Type 1 diabetes, as has my brother − sadly, the course is mostly silent about T1D and focuses almost exclusively on the much more widespread Type 2 diabetes), but I didn't expect to learn anything useful in daily life with a diabetic child. In that respect, I wasn't disappointed: I didn't.

I'm not certain, in truth, of what I expected. Some sort of generic high-level description of what diabetes is, its variants, its epidemiology, and a quick overview of the status of diabetes research, I guess. Instead, what I got is a series of seven very detailed, sometimes very technical, lectures about seven different aspects of diabetes research and/or treatment. While the first lecture, about the epidemiology of diabetes, was quite what I expected in terms of content (the production is consistently of a much higher quality than what I expected), the following ones have been both more challenging to follow / understand and much more interesting, in the sense of detailed, than what I expected.

Of course, since we have a different lecturer every week, the delivery is not always consistent (some people are basically better public speakers than others − week 4 in particular was very difficult to follow, with long lectures given in a monotonous voice; but maybe I was just especially tired that week). What is consistent though is, as I said above, the excellent production as shown in the smooth integration of graphics, videos, etc. in the lecture slides. Even the basic graphical elements are clear and crisp. Obviously Copenhagen University have invested some time and money to buy a professional-level production.

In the end, we have a smooth but surprisingly information-packed course. I am not certain what I'll remember of it in several months' time, though, since there is very little interactivity (just a few in-video quick questions and a quiz each week). Sadly, it is not possible to download lecture outlines for future reference, but we do get lecture transcripts.

As a last word, a quick reference to the “M” in MOOC. I guess I'm not the only one to have expected a more basic course (not that I'm complaining though); generally the level of the discussion forums is very low. The professors encourage directed discussion by asking several open questions every week; I am quite sure that were I brave enough to read every post in detail I'd find a fair number of intelligent, thoughtful insights. As it is, they're completely drowned in repetitive platitudes, so I find it better to just avoid the discussion forums altogether.

(It took less than three messages for a discussion about genetic screening for diabetes-associated risk alleles to reach the Godwin point…)

It's strange how some courses get intelligent, focused discussion forums and while some don't. So far the best I've seen were Caltech's Principles of Microeconomics and ANU's Astrophysics ones. I guess that's because both courses put a focus on collaborative problem-solving, while open-ended questions just reap a lot of clichés. In any case, that's a topic for another blog post, I guess.

Notes from the trenches: MongoDB M202 Advanced deployment and operations

I'm only two weeks into M202, so my opinion still has ample time to change; but I am very favourably impressed with M202 so far. The course title isn't a lie: we are speaking of advanced topics, in a very hands-on, pragmatic fashion. Gone are the very high-level generalities of M102; this is not an introductory course but really a great resource for people who'll have to deal with big, complex MongoDB instances in real life (or even, not so complex instances). The video lectures are long and detailed; the only thing missing is the ability to download them as slides and/or download the transcript for future reference. I guess we're supposed to refer to the official documentation after the course, though.

The homework is still too short/easy, I feel. Probably MongoDB want to be “inclusive” − or maybe it's due to the course being more about professional training than academic training; culturally, professional training is rarely if ever graded: sitting the training sessions is enough to write on your CV you've “had training”. (Then again, in my experience − I've given a bit of pro training a while ago − people who take pro training are quite motivated and genuinely willing to acquire new skills, so graded homework is generally useless). There could be a little more hands-on material, overall.

Anyway; I'm pleasantly surprised. I hope it's going to stay this way for the remaining five weeks.

*Plug*… *Bzzzt*… I can do Astrophysics!

Okay, maybe saying I can do Astrophysics is a bit presumptuous. But I can do the kind of basic, back-of-an-envelope calculations that grounds a passing familiarity with the major concepts of the field, and I'm more or less up to date with some of the principal areas of active astronomical research in 2014.

Or at least, that's what the Australian National University thinks, since they'll sign a certificate to that effect once ASTRO1x − The Greatest Unsolved Mysteries of the Universe is closed. For… drum roll…

I have now progressed beyond a passing grade in ASTRO1x !

That's my 10th edX certificate secured, 12th overall.

Yeah, yeah, I know a couple of days ago I wrote I was “struggling” with this course, and now I post an almost-perfect progress report. Thing is, I was lagging somewhat and I was more than a bit miffed that I failed to answer an easy question about redshift − that's the dip you see in the Homework 5 assignment (that was a 2-point question and the homework assignments are very short in this course). To my defense, I also did write that I was “well on my way to get my 10th edX certificate with this course”, so there.

Anyway. I don't have much merit though, the course being very easy − in reality, it's not the grading and the passing that's important here, it's the communicating about the science. In other words, I doubt Paul Francis and Brian Schmidt care one bit about assessing students' capabilities; what they do care about is to get the ideas across to as many people as possible.

If you ask me, that's great. From where I'm sitting, it's hard to get excited about quasars and/or particle physics. When the LHC team at CERN said last year that they'd conclusively proved the existence of the Higgs boson, I was among those who went, “huh, okay, but what's the point?” Doing this MOOC won't change much my understanding of relativity, quantum mechanics, bosons, baryons and tachyons − that wasn't really the point; the point is that I know have a feel for how dynamic “hard physics” as a field is, how, contrary to the common idea, some of the big questions left unanswered are in fact fairly simple (the answers may be horrendously complicated, though). How short we are of having really figured out this universe we live in.

And that's bloody exciting.

Stuff to be done:

• watch the two remaining lectures
• do the two remaining homework
• sit the final
• wait for ASTRO2x - Exoplanets to begin!

I Learn At edX.org

At least, that's what's written on the T-shirt (er, on the back, where you can't see it on the photo). Indeed, from edX alone I have to my name 7 certificates, another two are in the bag and just waiting to arrive. Number 10 should be Stat2.2x (that you can see on the pic); as for #11, it'll probably be ASTRO1 from ANU (though that might be trumped by the intriguing MAS.S69x and its announced 1-week duration).

Anyway, the great people at edX have been nice enough to send me a T-shirt, so thanks edX!

Notes from the trenches: Greatest Unsolved Mysteries of the Universe from ANUx

The Australian National University in Canberra have an astrophysics course up on edX called “Greatest Unsolved Mysteries of the Universe”.

I have to admit I only registered when I figured that one of the lecturers − one Brian Schmidt − was a Nobel Prize laureate. I mean, I enjoyed A brief history of time and all, but truth be said, I'm not so much into astrophysics as all that, what with juggling between extremely large quantities (uh, how many megaparsecs did you say?) and extremely small quantities (like, what's an electronvolt?) and generally complex physics. You have to remember, I never really studied physics beyond Newton, Maxwell and Bernoulli. I have a nodding acquaintance with relativity through reading a lot of sci-fi, but in my vocabulary, “quantum” is a passable synonym for “magical”. Still, getting a course from a Nobel prize winner is an opportunity one never really should pass on, if only for bragging rights.

This guy is my teacher. And yeah, he's got a Nobel, sure. Doesn't yours have one too?

Even more so when it's not one course − but four. And not wimpy five-week courses either; no, full-fledged, deep-down, 10-week courses each. A grand total of 40 weeks of tutelage about the finer points of recent astrophysics research, from some of the best-regarded players in the field. So, yeah, I registered for ASTRO1 and ASTRO2 (Exoplanets, due this summer).

So, six weeks into the whole 10-week kaboodle, what to think about it?

First off, it would be unfair to say this is a course by Brian Schmidt. I mean, it is. But it's co-authored and co-hosted by one Paul Francis who should take at least as much credit as Schmidt for the course, and possibly more, if only for his awesome[1] participation on the course discussion forums and his award-worthy dress sense.

I want a waistcoat like this one.

Francis' fast, excitable delivery is also very engaging, but I guess that's subjective. Based on his personal page on ANU's website, he's been quite involved in outreach programmes, and won several awards for science teaching; basically, it shows.

So, what about the course itself? It's brilliant. As expected, I'm struggling a bit. Some notions I seem not to be able to commit to long-term memory. As often, I get the qualitative aspects of the science and find the quantitative bits less involving. Since I tend to put this course on the backburner, I tend to do the homeworks late at night, when I'm not at my most, shall we say, fast-thinking. In spite of this, the course is easy. It's introductory, we're spared the worst calculations (we don't get to do much worse than simple Newtonian point mechanics), all the calculations are (sometimes excessively) spelled out. So I guess if I'm having difficulty with the homework, it's only because I don't put in much more than minimal effort. But that's okay, that's part of the deal. (And anyway, I'm just in a huff because I got one question wrong in 5 whole homework assignments. Fear not; I'm still well on my way to get my 10th edX certificate with this course.)

Generally, each week is devoted to a particular unsolved problem, such as the expansion of the Universe, the nature of the first stars (and why we can't see them), quasars, and so on. Noticeably, the lessons make use of very recent data (some readings from last year only were mentioned in week 6), so it gives a good idea of what astrophysicists actually do nowadays.

One particularly brilliant idea is the weekly mystery, the premise of which is: we've been moved to an alternate, parallel universe, that is superficially like ours (the same laws of physics apply) but quite different in some respects (such as, the night sky is full of brightly-coloured bubbles). Each week we're given some information about the mystery universe; the final will, apparently, test how well we've figured it out. That's fun as it is, but the particular stroke of brilliance is that the information we get is that which we ask for on the discussion forums − that is to say, Paul Francis trawls the forums all week long to find out the most frequently asked-for piece of data, then builds the data set (often complete with figures, “photographs” of the bubbly sky, etc.) and gives us the result as the next week's mystery entry. It creates a great dynamic with the class.

It must also be a dreadful amount of work for the staff; I guess the course isn't going to run with this format every year. All the better reason to stick to ASTRO1.

[1] And that's not a word I use lightly.

Catching up on Stat 2.2x

As I wrote in an earlier post, I regret not having taken Stat2.2x from the start. I have since registered, and it's sort of an uphill battle to get back up to speed. I've viewed the videos up to about the halfpoint of week 2 now, and tried my hand at the problem sets (ungraded of course since the deadline is way past.) I didn't do too badly, but not nearly good enough to pass the overall course.

Let's do the calculations:

• there are five exercise sets, the lowest score of which is dropped, worth 25% of the grade together (meaning the 25% are split into 4 ES scores). So each ES is worth 6.25% of the overall grade. Three exercise sets are still available to me, so I can get up to 18.75%;
• last week's midterm was also worth 25% − but that's closed to me now;
• the final is worth 50% of the grade.

One needs 50% overall to pass the course. A total of 68.75% is still reachable; so a pass is far from impossible. However probabilities is something I never really developped a good intuition for; this takes time. Rushing through the course (doing in three weeks what's meant to be done in five) may not be the best way to do it.
And of course, it means I have to catch up to week 3 today since the corresponding exercise set is due at 1am tonight, Paris time.
But eh, it's a challenge, right?

About the course, then: Stat2.2x follows Stat2.1x, and everything is kind of the same: the lectures are very long and slow, Prof. Adhikari speaks very slowly (with a lovely accent though) and repeats herself quite a lot. That's deliberate: probabilities is one area in which one really should get an intuition for how to approach problems, and going through the material slowly and deliberately is a good way to build that intuition. Still, when you've understood something, it kind of grates to have it repeated three times over fifteen minutes.
The course logistics are a bit different from other MOOCs: all the lectures are released at once, but the exercise sets are released only at the start of the week under scrutiny. The midterm and final have a rather harsh timeframe, as there are only two days between release and due date, and they're on weekends. Since the final alone is worth half of the overall grade, one may have to choose between passing the course and that romantic getaway to Florence one had planned for months before.
In practice, it means one can quickly watch the whole course's material, then review the relevant material just before the exercise sets / midterm / final when they're released. That means going over the material twice, which helps integrating the knowledge, I find.
With a more forgiving schedule for the exams, it would be a perfect arrangement. But then again, left to my own devices, I'd do away with midterms and finals altogether − but that's a topic for another blog post.

Anyway, I'd better stop procrastinating, and learn stuff about hypergeometric probability distributions.

[EDIT: Yay! Made the 6.25% in time.]

Postmortem: Fundamentals of Immunology part 1 at Rice University

Rice University have put up a Fundamentals of Immunology on edX; part 1 is closing right now. I've been quite successful at it with an overall grade of 90%; it's been rather a lot of work to get to that level so I'm rather proud of it. It's also going to be my second Verified Certificate from edX.

Immunology?

Yeah, you know; the study of the immune system. White blood cells, lymphocytes, the CD4 receptors that VIH binds to, auto-immune reactions (including allergies), the lot.

I've actually got a vested interest in learning about all that, as a lot of people around me have autoimmune diseases; but it's a fascinating subject in its own right. The downside is that, as most medicine-oriented courses (it's based off a pre-med course at Rice) it's heavy on memorization, so be warned, if you take this course or indeed any other course on Immunology, you better be ready to spend hours revising.

What does the course cover?

This is the first part of a two-part course; I expect most of the really hairy stuff to be in the second part.

First off, the course focuses on the operating principles of the immune system in vertebrates, especially humans. Mice and even birds are mentioned at times, but really, it's all about people.

The course starts with an overview of the different types of pathogens (in ascending order of complexity: viruses, bacteria, single-cell eukaryotes such as Giardia, fungi, worms), then a 30'000-feet-high overview of the immune system(s) in higher organisms (plants, fungi, animals) and the distinction between the innate and adaptive immune systems (the latter being specific to vertebrates). Starting with lecture 2, we dive into the details, with an overview of hematopoiesis (how blood cells are made and how they differentiate) and a long list of white blood cell types (myelocytes, lymphocytes, neutrophils, basophils, dendritic cells, B-lymphocytes, T-lymphocytes, etc.) Lecture 3 is a quick run-down of innate immunity.

Most of the rest of the course focuses on B-lymphocytes, the immune system's antibody factories: how antibodies are structured, how B-cells differentiate, how and where they mature, etc. That takes three whole (and information-dense) lectures. The course finishes with a discussion of the complement system, that is to say the molecular process by which pathogens, once identified by antibodies, are neutralized and killed.

So, quite a lot to fit in 6 weeks of lessons. The estimate of 7-10 hours per week on the course presentation page may be a bit higher than what I actually did, but it's not very far off.

Who is the teaching team?

The lecturer is Dr Alma Moon Novotny. (Don't be fooled by her Russian-sounding name, she has a very strong American accent!) She obviously has a long experience of teaching the subject, and makes a lot of effort to make the “memory load”, as she says, lighter. She does it by way of models, cartoons, and analogies. It's a bit strange at first to have cartoonish characters in the slides for a college-level course, but as soon as you realize that you have to learn the essential characteristics of each of these cells by heart, you start thanking her for the fun way in which everything is presented.

In a similar vein, she is generally funny and jokey (for instance calling the stem part of an antibody the “Yoo-hoo! bit” since it's the one that summons other cells) and, well, just fun to listen to. This really helps in such a basically arid subject.

My name is Bond. James "B-cell" Bond.

What about logistics?

The course lasts for 6 weeks, plus two for wrapping up (review, final exam, grading). Each week is generally taken up by one lecture (three lectures are squeezed in the first two weeks), divided in shortish segments of about ten minutes each. Below each segment are one or two ungraded “fact check” questions to make sure you've understood it all.

The lectures are accompanied by two PDF documents: the lecture outline, and the slides themselves. Dr Novotny recommends using the outline to follow along with the lectures; I've been doing a mix of reviewing the outline before watching the lectures, and following along with the slides. The outlines and slides are a great help for reviewing; to prepare for the quizzes and final exam I eventually printed them all out and carried them around everywhere.

(Phew, revising lessons on the bus: hadn't happened to me in fifteen years!)

A quiz wraps up each week. Somewhat unusually for MOOCs, the quizzes are “closed-book”, which is to say you're not supposed to have the course material (or indeed anything else) at hand while taking them. There's no way to enforce the policy though, so it's all a matter of honour on the students' side. (To be perfectly honest, I hadn't understood they were closed-book until the third quiz. Note however that I didn't actually get much better grades on the first two, when I had the outlines etc. at hand, than on the other four or indeed the final exam, for which I did adhere to the closed-book policy).

The course is wrapped-up with a longish 60-question final exam covering the whole course, also closed-book.

One question per page… doesn't quite mesh with the edX navigation system

As usual for MOOCs, there is no textbook (which is why the outlines are so detailed). Dr Novotny does provide a handful of links to interesting resources on the Internet though; while revising, I found (as she mentions) that the relevant Wikipedia pages are actually very good.

My impressions

I enjoyed the course a lot. First because I learned a lot of stuff, then because Dr Novotny is simply a joy to listen to.

The less enjoyable parts were, of course, the quizzes. I doubt there's anybody on Earth who actually likes doing quizzes… It didn't help that some of them were mis-coded (this was obviously the first run of the course and the staff obviously had to get to grips with the edX platform; they were, however, very responsive whenever errors were flagged on the forums). We're evidently far from the very sophisticated 7.00x Introduction to Biology from MIT with its wealth of interactive tools instead of simple yes/no/maybe quizzes. However, it's also obvious the Rice team hardly had the same budget as the MIT one's for producing the course − and it would be unfair to decry the course for not being up to the very best course I've ever seen. This Immunology course is all that can be reasonably expected, and more.

Dr Novotny with an antibody

A quick note: as I said in the introduction, I paid for the Verified certificate. Not so much because I think the certificate will be helpful in my career (I don't see how it would) but because it's cheap (25 USD, about 20 euros), it's a way to indicate appreciation for the work being done, and it's an added motivator: having paid, I'm less likely to drop the course, even if it's hard work.

Overall, I'm eagerly anticipating part 2 (where we'll learn all about T lymphocytes). Do be warned though, if you want to take up this course: it's a lot of work, and a lot of it is unfortunately (but unavoidably) about memorizing stuff.

[Edit] And now the certificate's arrived!