edX turns 2

So edX will be exactly two years old on May 15th. I'll be celebrating my own way, I guess by catching up on Astrophysics or revising for the Statistics final.

Veni, vedi, t-shirti.

So, what's edX for me? Quite a lot.

I jumped on the MOOC bandwagon in September 2013; well under a year ago, though the story begins in the summer of 2013. Having read a couple of pop-econ books and sort of got into my mind that I'd like, one day, to get some sort of formalish background in the subject, I was made aware that MIT had this OpenCourseWare thing going − and hey, what did you know, they had an Introduction to Microeconomics up! So I'd loaded up my smartphone with lectures, and watched them on my daily commute, and loved every single one of them.

(Now you people of MIT and edX, if you can convince Jon Gruber to host an economics MOOC, that'd be wonderful − Caltech's Principles of Microeconomics with Calculus, being way more austere and technically oriented, cannot fill in the role of an Econ 101 for everybody.)

So when I was done with 14.01SC and found my commute rather empty, I browsed through OCW and found a Fundamentals of Biology class, and thought hey, I've always regretted choosing math/physics over bio when picking a higher ed path (mostly because I wasn't properly aware that yes, you can study biology and not end up either in med school or working for Nestlé), so I started rooting around, until the big banners did get through to me that hey, a more up-to-date version of this course, with added homework material and a certificate at the end, was available through something called “edX”.

Boy did I know what I was getting into.

7.00x was great, so great that I signed up for Harvard's SPU27x (a smashing course that changed the way I envision food, and y'know, I'm French and all that) and UT Austin's Age of Globalization (thinking I would get a 14.01SC-like mathematically-minded analysis of globalization; sadly that was not to be). Then Berkeley's CS169 came along, and well, I'd wanted to find out what this Ruby on Rails thing was like for years. I signed up for MCB80.1x because neuroscience is this kind of hip thing right now, isn't it?

When that batch was over, I was hooked. I signed up to Ec1011x out of nostalgia for the MIT introduction to microeconomics, and this is to date the course I've worked the hardest on. I registered on Coursera to take the Epigenetics course that's just started − because there weren't any courses on edX to follow up to 7.00x. The idea that I might reorient my career towards a mix of computer science and biology/medicine started to grow in my mind, so I grew more methodical in my choices: I knew I needed a thorough statistics refresher so I picked Berkeley's Stat 2.1x (and later, 2.2x). The R statistical environment seemed like a good thing to learn − MIT's Analytics Edge course, here I am. Bioinformatics looked like the thing, so (there being nothing on edX) I took Peking U's course over on Coursera. And so on. (I tend to favour edX courses over Coursera / Udacity ones because I prefer the site layout, and more importantly because I feel that as a non-profit, it's intrinsically better − there is much less chance of seeing my profile data resold, or free stuff becoming paying, to start with.)

Eight months after signing up to my first MOOC, I am doing six courses in parallel, half of which through edX. I have 8 edX certificates to my name, two more are already secured and at least another one should be mine by the end of May. My edX dashboard, mixing past, current and future courses as it does, is overflowing with 20 entries (I am seriously considering forking OpenEdX on Github, refactoring the dashboard so it works for people with > 10 courses, and doing a pull request.) I brought back my old laser printer from the cellar so I could print out Immunology outlines and revise them on the bus. I plan my holidays around course schedules. I have started a blog about MOOCs. I generally finish each week mentally exhausted.

Less than a year into this MOOCing game, edX has already had a bigger impact on my life than, I don't know, any Web organization besides Google (to give some perspective, I've dropped Facebook to make more time for MOOCs, and feel all the better for it.)

All this to say, happy birthday edX!

(I'm unlikely to make it for the livestream, unfortunately, though I'll give it a try.)

MOOC forums - why they suck and how they could suck less

In my recent Notes from the Trenches about the Copenhagen Diabetes MOOC, I digressed a bit about the rather irksome “guided forum discussion” (anti-)pattern one sees occasionally in classes; sometimes, it's even compulsory and participation (any participation) counts towards your grades, such as in UT's Age of Globalization course.

Why is this irksome? Because it plain doesn't work. Encouraging or forcing people to communicate is awkward at best; when the population count is in the tens of thousands, it means you end up with a lot of one-liners, “me too”s, clichés and platitudes.

Take the Diabetes MOOC. One of the discussion topics this week was “should we systematically sequence the genomes of newborn children to identify risks?” Within a couple of days, roughly two-thirds of the answers were along the lines of “no that's eugenist, you big Nazi you”. About half of the remaining posts were from one single “superposter” who apparently took it as his holy duty to answer as close as every single post as possible. The rest may have been thoughtful, intelligent responses but by that time my eyes had glazed over. I didn't even try to raise the level of the debate (yet, based only on the lectures, there was a lot to say about the − poor − predictive abilities of genome sequencing, false positives, etc.)

UT's Age of Globalization had mandatory forum participation. Well, that's a humanities course, I guess student interaction is pretty much an integral part of the point, so pointing towards the only interaction medium there is seems logical; however in the end we had the same behaviour: most people would just upvote an entry they liked, or respond with a one-line “me too!” or “no you're wrong”, and get the 1 point credit towards the certificate.

What it all leads to is a terrible signal/noise ratio, which is a powerful disincentive for quality participation. Thinking through an issue, weighing the pros and cons, imagining the various points of views takes time and effort; seeing that all this effort is systematically rewarded by either nothing or a short “thanks, that was very interesting” (or worse “that doesn't make me change my opinion”) that doesn't bring anything to the discussion, is disheartening.

The problem is compounded when the forum software itself is not so great. Take a look at edX's forum:

Some glaring issues:

• Very, very poor use of screen real estate. The screenshot above shows the totality of the forum's landing page for 15.071x, on my desktop computer which has a 27", 1920×1080 pixel display. Two million available pixels and edX only shows 8 posts − half of which are pinned staff announcements that once read, are of no interest whatsoever to students. So that leaves… yeah, four posts being displayed. Hurray.
• Actually when you click on a post, the height of the forum, including navigation pane, increases to the minimum of display height (minus some padding) and total post (with responses) height. Huh, what? That's really inconsistent from a UX point of view − the navigation pane's display should never be dependent on the detailed view. Makes the display jump around unpredictably.
• The scrolling behaviour between the two panes is a mess. It's such a mess that I don't even want to think it through.
• The forum structure (subsections) are hidden from view; by default the forum shows “all discussions”. It's far from obvious that there even are subsections, and that one should use them.
• There is no moderation worth of the name: basically the TAs just delete posts that violate the honour code (such as posting full homework solutions) but don't do the necessary housekeeping (such as moving threads to the correct subsection). Maybe there's moderation of flamewars, as I didn't see any, but I would pessimitically just think people can't be bothered to flame each other.
• The navigation pane's entries are: subject, number of upvotes, number of comments. And that's all. No mention of first post date, last comment date, first post author, last post author, whether staff have answered the post, etc.: all elements that can pique the interest of potential readers.
• Sorting is very basic. The most obvious missing options are “unanswered posts” and “most active discussions”.
• There is upvoting of posts, but no downvoting.

What this means is that the most active threads are the “Eeeee I'm from Wherever and really excited about this course!”, hello-introduce-yourself ones, and a couple of staff-originated course-related threads. Not so great for a community.

Some edX courses also suffer from their being split into multiple parts − meaning that the forum dynamic, if and when it has been established, has to restart from scratch every five weeks. Huh, a month is not enough for a community to assert itself, you know?

As usual, Coursera's software is a little better finished (either Coursera is much better funded − which is likely, the company has raised something like 85 million dollars according to Wikipedia; edX being a non-profit can't raise venture capital and relies on contributions from, as I gather, MIT, Harvard, UCBerkeley and Google) so most of these “glaring” issues are avoided, but there are still no “view unread” / “most active” filters; one has to navigate to the subforum one wants to check out the discussions there. Generally the end result is the same: slogging through the forums is a chore, and the signal-to-noise ratio is abysmal.

To be fair, the edX forums have one redeeming virtue, and that is the possibility to link subforums to courseware pages. What this means is that in practice, there are mini-forums embedded in the courseware pages, specific to that content − for instance, in ANU's ASTRO1x course, the weekly “mystery” page has its own subforum; in a limited way, that lets student skip the tedious forum navigation. (They also have good MathJax integration, letting people enter beautiful formulae).

Part of the problem is that the platforms just recreated (from scratch, with limited means… nuff said) forums following a dying model anyway. Online forums in the traditional sense have had their glory days from about the early 2000s (when they finally took over from newsgroups) to the early 2010s (when they've been displaced by Facebook, Google+/Hangouts, etc.) In 2014, for good or worse, they're not an appropriate medium anymore. Users are not used to the discipline of finding out the best subforum to post to, then slogging through the forum archives to find unanswered posts, watch out for replies, etc. They're used to the little red thingummy on Facebook telling them someone's replied to their posts.

Others have already written about how to solve the problem. I'm not certain that just waving the Web2.0 wand will magically make all troubles go away, but certainly, one can take a leaf off some social sites. Embedding live chat can be a good idea; it was tried for CS169 part 2, I think, unfortunately they picked a heavyweight JS IRC client that slowed down the pages tremendously and often made them crash altogether (it was kind of ironic to see such crappy, badly-tested software being put together for a course about software quality…), so after a few days a TA put up a note on the course forums giving instructions for how to disable the chat component completely. A worse admission of failure I have rarely seen; but the idea has some merit. A component integrated with the platform (rather than third-party) might work, especially if some thought is given to screen real estate usage (small font sizes, less whitespace, and you could fit a narrow chat column to the right of an edX courseware page even on a 1280px wide display − let's not even mention responsive design).

Having a look at how StackExchange works might yield some ideas − cleanly separate random chat from serious questions/answers. Make it easy for people to find if a question has already been asked, find unanswered questions, kill  (through downvoting for instance) duplicates and obviously worthless posts (like the individuals demanding special treatment). Have a look at how Discourse do things. Throw in some social media features (add specific posters to watch list, to make sure one never misses a post from JonPowles on the ASTRO1x forums; conversely, blacklist people who tend to be irritating) to be in the mood of the 2010s. Perhaps add direct messaging if you must.

Connect courses together. It's highly likely that students who took Introductory Biology will move on to more advanced biology courses; letting people follow through with their mini-social network already set up would be nice − and the possibility to refer to discussions that've already been had in other courses would be handy. I was quite surprised when taking an end-of-course survey, and one of the questions was “Do you intend to keep in touch with fellow participant?” − I realized that I couldn't actually name a single fellow participant. Also, why would I want to keep in touch? I don't know anything about them, in all likelihood we have nothing in common apart from a shared interest, at a specific point in time, in a specific course.

(I thought that segmenting the forums − basically create “bags” or “classes” of a few thousand students − might help too, but probably would do more harm than good: all of the dynamic online communities I've seen have had a core of a handful of stalwarts. I'm not sure the number of high-value individuals scale linearly with the size of the community, so segmenting would prevent these individuals from connecting with each other and create a good group dynamic. Besides, cutting up a 200,000-strong cohort into a hundred 2,000-strong classes would mean TAs have 2,000 times the moderation / participation workload.)

So, yeah, building decent forum/community software is a significant endeavour. It takes time, and effort, which the various providers may prefer spending on other things (e.g. fixing edX's dashboards, or the horribly slow math input fields; or indeed work on XBlocks so future courses may use more advanced tools). And I know, edX's open source, so why don't I just fork the code, fix it, and do a pull request? Well, apart from the fact that it is a significant endeavour and that even if I were to find the time and motivation to build such a thing (not to mention that my Python skills are inexistant), my PR would almost certainly not be merged (it is highly unusual that far-ranging changes that come out of the blue are accepted by open source projects, especially ones as active as edX − there were 31 merges on May 9th alone).

As a final note, I'd be interested to see how NovoEd does things. They're branding themselves as a “social environment for online courses” rather than as a “platform for creating and hosting MOOCs”; they will have given much thought to community-building. Unfortunately, the courses they're offering (mostly around finance, entrepreneurship, and startup creation) are of little to no interest to me, and I'm not quite ready to register to a boring course just to see how they're doing forums.

Random musings: On Platforms

MOOCs are here to stay; that is a certainty[1]. But we're still in the early stages of the MOOC “revolution”: a new ecological niche has opened up, we are in the middle of a diversity explosion, but sooner or later the number of players will go down and each will differentiate.

(To continue with the bad analogy, the dinosaurs have died out and we're in the diversification phase: Coursera's the birds, edX's the mammals, OpenEdX's the metatherians − closely related to placental mammals but different − Udacity's the crocodylians, FutureLearn's the bony fish, etc.)

Sooner or later, the number of platforms will go down as “the industry” converges to a handful, sort of like happened in the social network space: gone are the Orkuts, LiveJournals, and MySpace of the past; only Facebook and Twitter (and Baidu in China) remain, and the two are sufficiently different to not be frontal competitors.

Listing the MOOC platforms out there is something many people have done, so I won't replicate for replication's sake. Safe to say there are many. They are subtly different, though in reality, the variability of individual MOOCs is often greater than the variability in the platforms hosting them.

If we take a cursory look at the megafauna in this landscape, we have two major beasts competing frontally, a third player which is different enough to carve its own niche, and a slew of minor players. A full review isn't what I intend to do here (I do mean to do a writeup of edX vs Coursera at some point, though), I'll just give some highlights on the platform types out there.

Coursera and edX are the big beasts, very similar in terms of structure. Between them, they must hold down a massive proportion of the MOOC market (70%, 80%? I have no idea.) Universities create courses with a mix of video lectures, text resources, homework and exams; students enroll to specific sessions of a course, on pre-set dates, and if they pass a certain threshold, they get a certificate. For certain courses, students may opt for a “verified” certificate (for which the platform takes minimal steps to ascertain the student's identity, for instance by comparing webcam images with a picture ID like a passport). There are even a handful of “sequences” of courses leading up to an overall certification in a specific field (they are called “Specializations” in Courserese and “XSeries” in EdXian). The similarity is sufficient for there being cross-talk: Caltech's Principles of Economics with Calculus course ran once on Coursera then migrated to edX.
There are differences between the two (Coursera's more mature overall, edX's more flexible) but switching from one to the other is mostly transparent for students − it's the actual course that's important.

Udacity has a different model. First off, it's a for-profit business (Coursera's one too, but less obviously so as they're at the gain-market-share-lose-money stage of startup development; edX's a non-profit); the focus is much more clearly on paying customers. In terms of content, it means the courses tend to be more oriented towards building specialized professional skills than towards acquiring general knowledge of a field − Udacity teams up with businesses to offer classes on specific products, for instance. (There are also more general, less immediately “useful” courses, but they do seem a bit lost in there.) The classes themselves are self-paced: students may take them whenever they want, over as much time as they need. To access the “full learning experience” (which I understand means access to TAs and certification) one can pay a monthly subscription fee.

Then there are national endeavours such as the UK's FutureLearn, France Université Numérique (actually built on top of edX software), Australia's Open2Study, etc. I am sort of confused why these are actually needed, and indeed some universities hedge their bets (it looks like most Australian universities have courses up on either edX or Coursera). I guess the point is to integrate into the national higher-education system, so that a − say − French student can mix and match courses for her local university and FUN[2] and still gain credit for both? We're not there yet; if that is indeed the point then that will be an exciting development, but in the meantime the national platforms look like something very unnecessary indeed.

Saylor Academy deserves a special mention. They don't actually create course materials, but rather use the wealth of material that's out there (from OpenCourseWare to YouTube videos via freely-accessible textbooks) to package coherent courses and full curricula, going a full step further than Coursera/edX's “sequences”: by mixing and matching existing resources, you can actually get the full equivalent (content-wise) to a traditional US college 16-course major. Saylor does the packaging, exams, and certification.

Possibly what is missing (or which I haven't found) is a “MOOC navigator” site, something that will suggest pathways through MOOCs (e.g. for a biology education, “take edX's 7.00x for introductory biology, then hop on to Coursera's Useful Genetics, pick up a couple of biochemistry courses here and there, a stats course here…”) − that is to say, go above the level of individual courses and establish curricula, taking into account things such as starting dates, etc. Actually, a “skill tree” as in many MMORPGs might do it. Kind of a metaplatform, really. Hmm. That might be an interesting side-project for when the days finally decide to last 36 hours to give me time to actually do stuff.

[1] What I am not so certain about − and that's the exciting part − is the impact the MOOC “movement”, if you want to call it that, will have on both the educational system as a whole, and the workplace. Speaking from a French, occasional recruiter point of view, one of the first things I look at when looking at a resume is the school the applicant graduated from. It's not the only thing and it's not discriminatory, but it's a better-than-null-hypothesis proxy for the overall “quality” of the applicant, especially if they graduated less than five or ten years ago.

That said, the French system is intensely competitive, with a lot of engineering schools (204 are officially allowed to deliver a diplôme d'ingénieur, not counting universities), often quite small (most have classes in the ballpark of 50-100 students a year), admission to which is based on a handful of nation-wide competitive examinations. Specialized (and not-so-specialized) media maintain a nationwide ranking of these schools, updated on a yearly basis. By this system, the “better” schools (i.e. better-ranked) naturally attract the “better” students (i.e. those that are better than the rest at taking gruelling exams they have spent the previous two or three years preparing for).

All in all the French system is exclusive − only the very best go to ENS or Polytechnique, the next best go to Mines or Centrale, etc. It makes a decent guarantee that a graduate from a toppish-tier school is a very clever person with a great capacity for hard work. This is fine and dandy, but what it doesn't do is guarantee that graduates from a lower-ranked school or a mainstream university are not good and indeed many very apt students are excluded (in technical terms, the sensitivity is decent but the specificity is awful; there are few − though not zero − false positives but a great many false negatives.)

MOOCs by nature are inclusive. I tend to believe that people who are willing to stick to the sometimes heavy schedule of a MOOC − and even better, many MOOCs − show themselves to be highly-motivated individuals which may be of great interest to companies. But that's a gut feeling; only time will tell, if and when MOOCs go mainstream (I've only once seen a resume mentioning a MOOC).

[2] I wonder whether the “France Université Numérique” name was picked specifically to bring a little FUN in the university system?

Random musings: What makes a good MOOC?

All MOOCs are born equal in dignity, but not in much else

Having completed or being well on my way to complete about 15 MOOCs, and tried a handful more, it's evident to me that no two MOOCs are the same; which is right and proper, the organization of the course being, after all, down to the professor's preferences − and also of course subject to the specificities of the subject matter.

All MOOCs are not created equal, then, from a purely objective point of view. Subjectively, they are not all the same either. Some MOOCs make a strong impression, others are bound to just fade away, others still are fated to be dropped. It is therefore a valid question to ask what makes a good MOOC.

The ideal analytical method

Scientifically, the correct way to answer that question would be to describe each MOOC with a fair number of objective characteristics (we'll come to that), then ask students their impressions on the MOOC at least three times: once in the first two weeks (before the “second-wave drop-off”[1] kicks in), once around the end, and once a couple of months afterwards (once the euphoria of having passed the course is over and people have had time to start forgetting the course − I assume here that the principal objective in taking MOOCs is to learn things for the longer term rather than get a certificate and quickly forget everything). So for each course, we have three outcomes: a “stickiness value” (do students stick to the course?), an immediate appreciation score, and a medium-term appreciation score. Using all that data, we can build classification regression models to find out what characteristics are predictors of our three outcomes.

(We could also try to cluster the students according to their preferences; and then tailor courses towards a number of student profiles, eventually.)

But then of course, I don't have a large dataset. I have, in fact, a piss-poor sample of just myself, with all the biases that entails and less data points than classification criteria. So let's forget about the scientific methods for now, and just review some of the defining characteristics of the MOOCs I've taken and from that sample, try to draw conclusions about what I like and what I don't like in a MOOC.

What's in a MOOC?

(That which we call a great course, by any other token would smell as sweet… er…)

Here are our classification criteria:

• Real course: is this course based off a “real” course at the institution in question? Levels: No, Partly, Fully, Actual course runs concurrently;
• Multi-part course (e.g. Statistics at Berkeley);
• Part of a sequence of courses;
• Length of course: in weeks (for multi-part, take the total)
• Lecture type: Classroom videos, Talking head, Talking head with infographics/animations, Slides with voice-over, “Slate” (or tablet-style). In case of a mix, we'll just go with the most common type. (In a thorough analysis we'd have to quantify the mix, e.g. 40% talking head, 60% slides with voice-over);
• Native English-speaking lecturer(s): Yes, No. (In a full study we'd have to switch this to native speaker of the course language, and add some data about subtitles in other languages);
• Speed of delivery: does the lecturer speak fast or slowly? To simplify, I'll cluster in the “fast” group the lecturers who significantly use hand/body language as accents;
• Tone: Conversational, Dry, Passionate (note that the latter is highly correlated with a fast delivery, so that might not be such a great criterium);
• Printable resources: None, Transcript, Lecture Slides, Outline/summary;
• Links to additional material: e.g. research papers
• Length of video segments (average): 0-5 minutes, 5-15 minutes, 15-25 minutes, > 25 minutes
• Length of video segments (maximum): 0-5 minutes, 5-15 minutes, 15-25 minutes, > 25 minutes
• Assignment difficulty: from 1 to 5: 1 is very easy (immediate answer), 5 requires multiple hours of work, roughly, so that's not so simple: quizzes are quick to answer but may be hard, interactive tools may take longer to use while the problem is simple overall. So it's a mostly subjective point.

The rest of the criteria are simple yes/no:

• Quick questions between video segments
• Ungraded practice problems or worked examples
• Homework: quizzes
• Homework: numeric / formula input
• Homework: essays
• Homework: code / programming / offline tool using
• Homework: online, interactive, custom tools
• Homework: multiple tries allowed
• Midterm(s)
• Final exam
• “Collaborative” focus on assignments: e.g. dedicated forums for each problem in a set, etc.
• Guided discussion on forums

Note that I count as one “course” the sum of all parts of a multi-part course. For instance, CS169 SaaS at Berkeley is only one course (it doesn't matter that students get two certificates). But self-contained courses that are part of a series count as distinct courses. There's a part of subjectivity here, but broadly, I count CS169 and Stat2X at Berkeley, or BIOC372x at Rice, as multi-part, while I have ANU's Astrophysics courses and Harvard's MCB80x courses, as multiple courses in a sequence.

My MOOCs

So that's a good handful of criteria! Let's see how (some of) my MOOCs fare. I've actually tabulated it all but Blogger doesn't seem to support uploading of random file types, so until I find someplace to host it, here's an unscientific look at the data:

The top-scoring courses (MIT's 7.00x, Berkeley's CS169x, Caltech's Ec1011x, and ANU's ASTRO1x[2]) are a mixed bunch. They all have fast speakers with either a passionate or conversational tone. Apart from the Astrophysics class, they are based on actual courses (Caltech's one was even run concurrently with the on-campus class). The first two had classroom videos, Caltech's was mostly slate / tablet, ANU's is about 2/3rds of the time lecturers in front of infographics and 1/3rd slate.

As far as homework is concerned, they all allowed multiple trials for most of the problems. The nature of the homework is varied: 7.00's shone through its use of custom interactive tools, CS169 was heavily about programming (well, that's the point of the course, innit?) and the other two are focused on numeric or formula input. Apart from ASTRO1, the difficulty or homework duration was very much on the high side; Ec1011 was clearly the most difficult course I took and I did spend many hours on the other two.
All courses put a focus on collaboration between students, by putting down links to the relevant forum sections on the appropriate pages.

Individually, the courses shine in different ways:

• 7.00's excellent lectures, additional videos, and immense wealth of tools make it by far the best course ever, be it a MOOC or an actual class, in my experience. To say I'm anticipating this summer's 7.QWB with some trepidation would be an understatement; I wish MIT's Biology department put up an XSeries.
• CS169 was good because of the subject matter, of the passion the lecturers put into the course, and because of the programming assignments. There were a couple of quizzes that were broadly speaking a let-down. I also appreciate that there is no exam: the homework is sufficient. The forums were pretty good too.
• Ec1011's big selling point is the homework's difficulty. You get to spend many hours on it every week (roughly speaking, I spent all Saturday mornings doing Ec1011 homework for the duration of the course, sometimes overflowing well into the afternoon); Prof Rangel's philosophy of “mastery teaching” is great: you get a large number of trials (10) for each problem and you're encouraged to discuss the problems on the forums, as long as you don't actually post the solutions. Overall, it means you get intimately familiar with the (albeit simple) models economists use.
• ASTRO1 is not so much about grading, as I wrote earlier, so the homework tends to be ridiculously easy. But the lectures are great (thanks to the lecturers' enthusiasm), the accompanying material (reference notes, worked examples, etc.) is very good, and the most brilliant idea is the weekly mystery that I've mentioned before; along with the accompanying forums it means we do some intriguing collaborative problem-solving that integrates everything we've learned in the class.

Rice's BIOC372.1x narrowly misses a top rating because of the quiz-based homework with only one try. That doesn't mean it's impossible to get a good grade (I did), but it makes doing the quizzes a chore more than a learning opportunity in its own right. I appreciate the nature of Immunology means it's more about memorizing things than acquiring problem-solving skills, but I'm sure there are ways to make the homework less annoying.

Harvard's SPU27x gets an honourable mention too, although I dropped the homework (wasn't interested enough) and downgraded to auditing the course rather than passing it. Basically, the course itself (teaching about science through the medium of cookery) is a great idea, and the demonstrations by guest chefs were great. Some of the labs were kind of interesting (molten chocolate cake has become a household classic) though I skipped through most (I wasn't particularly interested in measuring the elastic modulus of steak as it cooks, for instance).

The worst classes are the ones I dropped, i.e. Mount Sinai's Introduction to Systems Biology, Toronto's Bioinformatics Methods, and Harvard's PH525 Data Analysis for Genomics. Toronto's I won't go into too much detail about, basically the course wasn't what I expected or needed (it's better seen as a hands-on companion to a bioinformatics course; I guess I could try it again now I've finished Peking U's introduction to bioinformatics).
Mount Sinai's Introduction to Systems Biology suffered from long, purely slide-based lectures with a voice-over delivered in a sing-song voice, explaining (badly) ideas that are very complex in nature, making me feel completely out of my depth; I had to watch each segment two or three times to gain an understanding. The poor quality of the recordings (you could hear background noise such as police sirens driving by…) didn't help a bit. There wasn't really any homework besides a weekly quiz and a couple peer-graded questions. I clung on for two weeks then decided my sanity was worth more than that. It's a shame, as (in the absence of a MIT Biology XSeries…) Mount Sinai's Systems Biology 5-course Specialization looks like the best match for where I'd like to take my career (somewhere on the intersection of computer science, big data, and biology/life science).
Harvard's PH525 is a different kettle of fish. I just didn't have the mental bandwidth to commit the required effort to the course. The lack of actual homework for the first two weeks (just “understanding checks” in the form of quizzes) didn't help me getting involved. Also, Prof Irizarri had a tendency to sway from side to side in the lectures; since I mostly watch lectures on the bus or train, it made me seasick[3].

Conclusions and reflexions

Based on that skewed analysis of the MOOCs I did, can I draw conclusive, er, conclusions about what makes a good MOOC? Not really, but I can put forward a few points:

• Classroom videos are better, as they make me connect to the course more. Failing that, “slate” (tablet-style) or lecturers-with-infographics (ANU's ASTRO1 is a good example of that) does the trick.
• Shortish (about 10 minutes) video segments interspersed with quick questions, please.
• The speaking qualities of the lecturers are obviously of great importance.
• Downloadable or printable resources are very welcome. Pointers to additional materials are too, but somewhat less.
• Homework should be seen as a learning opportunity in its own right, so rather than focus on checking that students have learned the lesson, they should be more in a problem-solving format. Homework should be hard (or at least, long), but multiple tries should be allowed and collaboration between students on the homework should be encouraged.
• I'm frankly doubtful about the overall utility of final exams in the grand scheme of things. I think something like ASTRO1's “mystery” is the gold standard: a recurring problem with additional hints every week, that allows students to integrate every lesson's knowledge in order to bring about final understanding. Note that this approach is very well-suited to programming classes, too.
• I am broadly indifferent to a course being offered all in one go or split into two or three parts. I suppose splitting means people are more likely to register (it doesn't feel like committing to 10 or 15 weeks' worth of work). I don't care much, if anything I'd prefer everything in one go (no need to register twice, no risk to see the second half of a course rescheduled to the other half of the year.)

Now to take this further… Anybody knows of good public datasets about MOOCs? Were studies made to measure student's opinions of MOOCs multiple months after the courses have ended?

---

References

You can find links to most of the courses I mention in the Completed courses page. As for the others:

• Harvard's SPU27x Science and Cooking
• University of Toronto's Bioinformatics Methods
• Mount Sinai's Introduction to Systems Biology
• Harvard's PH525x Data Analysis for Genomics

Footnotes

[1] This is totally unsubstantiated, but I'd think there are two initial waves of dropping-outs: one the very first week and possibly even before that, when students realize this course isn't for them (wrong difficulty level, wrong appreciation of the subject, etc.); and one closely afterwards, when students basically throw their hands in the air and decide that although they are interested in the subject, the MOOC itself doesn't fit their requirements (it is a “bad MOOC” from their perspective). Here, I am concerned about this “second wave” − why makes people give up on a MOOC on a topic they are interested in?

Of course there are other drop-out causes: lack of time, interference from the real world or indeed other MOOCs, a late realization that the subject isn't so interesting after all, etc. But these would tend, I believe, to be more or less evenly spread out throughout the course program.

[2] Sometime I'll just drop the systematic “x” suffix in edX course labels.

[3] Hey, I didn't say I had only good reasons to drop a course!

Notes from the trenches: Diabetes, a Global Challenges (Copenhagen University)

The Diabetes course from Copenhagen University is drawing to an end; only one week to go. I'll do a full postmortem when it's over (next week, basically), but in the meantime, some unordered remarks/comments:

Unlike most courses I take, I didn't register to this one to acquire useful skills or knowledge, but out of curiosity. I have, obviously, a specific interest in the subject matter (my son has Type 1 diabetes, as has my brother − sadly, the course is mostly silent about T1D and focuses almost exclusively on the much more widespread Type 2 diabetes), but I didn't expect to learn anything useful in daily life with a diabetic child. In that respect, I wasn't disappointed: I didn't.

I'm not certain, in truth, of what I expected. Some sort of generic high-level description of what diabetes is, its variants, its epidemiology, and a quick overview of the status of diabetes research, I guess. Instead, what I got is a series of seven very detailed, sometimes very technical, lectures about seven different aspects of diabetes research and/or treatment. While the first lecture, about the epidemiology of diabetes, was quite what I expected in terms of content (the production is consistently of a much higher quality than what I expected), the following ones have been both more challenging to follow / understand and much more interesting, in the sense of detailed, than what I expected.

Of course, since we have a different lecturer every week, the delivery is not always consistent (some people are basically better public speakers than others − week 4 in particular was very difficult to follow, with long lectures given in a monotonous voice; but maybe I was just especially tired that week). What is consistent though is, as I said above, the excellent production as shown in the smooth integration of graphics, videos, etc. in the lecture slides. Even the basic graphical elements are clear and crisp. Obviously Copenhagen University have invested some time and money to buy a professional-level production.

In the end, we have a smooth but surprisingly information-packed course. I am not certain what I'll remember of it in several months' time, though, since there is very little interactivity (just a few in-video quick questions and a quiz each week). Sadly, it is not possible to download lecture outlines for future reference, but we do get lecture transcripts.

As a last word, a quick reference to the “M” in MOOC. I guess I'm not the only one to have expected a more basic course (not that I'm complaining though); generally the level of the discussion forums is very low. The professors encourage directed discussion by asking several open questions every week; I am quite sure that were I brave enough to read every post in detail I'd find a fair number of intelligent, thoughtful insights. As it is, they're completely drowned in repetitive platitudes, so I find it better to just avoid the discussion forums altogether.

(It took less than three messages for a discussion about genetic screening for diabetes-associated risk alleles to reach the Godwin point…)

It's strange how some courses get intelligent, focused discussion forums and while some don't. So far the best I've seen were Caltech's Principles of Microeconomics and ANU's Astrophysics ones. I guess that's because both courses put a focus on collaborative problem-solving, while open-ended questions just reap a lot of clichés. In any case, that's a topic for another blog post, I guess.

Notes from the trenches: MongoDB M202 Advanced deployment and operations

I'm only two weeks into M202, so my opinion still has ample time to change; but I am very favourably impressed with M202 so far. The course title isn't a lie: we are speaking of advanced topics, in a very hands-on, pragmatic fashion. Gone are the very high-level generalities of M102; this is not an introductory course but really a great resource for people who'll have to deal with big, complex MongoDB instances in real life (or even, not so complex instances). The video lectures are long and detailed; the only thing missing is the ability to download them as slides and/or download the transcript for future reference. I guess we're supposed to refer to the official documentation after the course, though.

The homework is still too short/easy, I feel. Probably MongoDB want to be “inclusive” − or maybe it's due to the course being more about professional training than academic training; culturally, professional training is rarely if ever graded: sitting the training sessions is enough to write on your CV you've “had training”. (Then again, in my experience − I've given a bit of pro training a while ago − people who take pro training are quite motivated and genuinely willing to acquire new skills, so graded homework is generally useless). There could be a little more hands-on material, overall.

Anyway; I'm pleasantly surprised. I hope it's going to stay this way for the remaining five weeks.

*Plug*… *Bzzzt*… I can do Astrophysics!

Okay, maybe saying I can do Astrophysics is a bit presumptuous. But I can do the kind of basic, back-of-an-envelope calculations that grounds a passing familiarity with the major concepts of the field, and I'm more or less up to date with some of the principal areas of active astronomical research in 2014.

Or at least, that's what the Australian National University thinks, since they'll sign a certificate to that effect once ASTRO1x − The Greatest Unsolved Mysteries of the Universe is closed. For… drum roll…

I have now progressed beyond a passing grade in ASTRO1x !

That's my 10th edX certificate secured, 12th overall.

Yeah, yeah, I know a couple of days ago I wrote I was “struggling” with this course, and now I post an almost-perfect progress report. Thing is, I was lagging somewhat and I was more than a bit miffed that I failed to answer an easy question about redshift − that's the dip you see in the Homework 5 assignment (that was a 2-point question and the homework assignments are very short in this course). To my defense, I also did write that I was “well on my way to get my 10th edX certificate with this course”, so there.

Anyway. I don't have much merit though, the course being very easy − in reality, it's not the grading and the passing that's important here, it's the communicating about the science. In other words, I doubt Paul Francis and Brian Schmidt care one bit about assessing students' capabilities; what they do care about is to get the ideas across to as many people as possible.

If you ask me, that's great. From where I'm sitting, it's hard to get excited about quasars and/or particle physics. When the LHC team at CERN said last year that they'd conclusively proved the existence of the Higgs boson, I was among those who went, “huh, okay, but what's the point?” Doing this MOOC won't change much my understanding of relativity, quantum mechanics, bosons, baryons and tachyons − that wasn't really the point; the point is that I know have a feel for how dynamic “hard physics” as a field is, how, contrary to the common idea, some of the big questions left unanswered are in fact fairly simple (the answers may be horrendously complicated, though). How short we are of having really figured out this universe we live in.

And that's bloody exciting.

Stuff to be done:

• watch the two remaining lectures
• do the two remaining homework
• sit the final
• wait for ASTRO2x - Exoplanets to begin!