Implementing WebVTT in Firefox at Seneca

Fall Teaching

It's the start of a new semester, and once again I'm teaching the Open Source and Mozilla Development course at Seneca's Centre for Development of Open Technology.  I love teaching this course for many reasons: it's different every time I do it and never boring; the students and I get to work on real code together; a bunch of students get to have their first experience of what it's like to help push the web forward (hint: it's awesome); we get to work with some amazing people in the Mozilla community.

Evolving Thinking About Student Projects

The way I run the course has evolved over the years, and this year I'm going to try something new and a bit risky.  When I taught it for the first time, I can remember sitting in a room with Shaver at Seneca and coming up with about 15 or 20 projects for things Mozilla needed.  It was literally back-of-the-napkin planning, and it was equal parts amazing/useful projects and probably-not-worth-doing.  Mike and I had no idea what students were capable of, how big to make the projects, how much complexity, what background knowledge they could acquire in time.  As a result, we got some of it right and some wrong.  And we learned for next time.

Since then I've tried different approaches to the course project.  Sometimes we've sought out "wouldn't it be nice if we had..." type work, other times we've chosen to use existing bugs; sometimes we've done individual projects, other times had groups of 2 or 3.  There's no one right way to do it.

The Pointer Lock Experience

Last fall I assigned students individual bugs, and then spent the second half of the course implementing the Pointer Lock API instead of doing formal lectures.  Each class we'd go a bit further and discuss how things worked, what to try next, comb through the spec, etc.  It was a lot of fun, and a nice byproduct of the effort was that Firefox shipped Pointer Lock.

One thing I didn't like is that not all of the students got engaged with the code as much as they could have.  About half were eager to do it, and couldn't be stopped.  The rest saw a bunch of people already working, and decided to sit on the side lines.  I don't blame them: the code was complicated and we went fast in order to get it done.  Also, the tasks weren't highly parallelizable, since the spec was fairly constrained and the API small.  Another problem was that I still had each student working on other individual bugs, which meant they had to make a choice between getting their graded work done and getting involved in the class project.

This time I've decided to go all in on the larger class project approach.  What I liked about Pointer Lock was that it was real, mattered to the web and Mozilla, and was a strange mix of cutting-edge and non-critical tech at the same time.  I also liked that it allowed the students to form a community of knowledge together around a single spec, API, etc--when students can help one another, my experience is that they do.

When I first agreed to fix the Pointer Lock bug with my class, it wasn't really on that many peoples' radar.  By the end of the term I was getting regular emails from people wanting to know when it would land, and reading media reports about how Firefox was shipping it.  There's a sweet spot in there where students feel safe to learn as they go without too many critical eyes on them, but then also have enough eyes to help them find issues and solutions.  Getting the balance right is hard, in my experience.

Toward WebVTT

Earlier this summer I went searching for another "Pointer Lock"--I wanted a new spec I could implement with my class, one which mattered, but not so much we'd be blocking releases if we took a meandering path in order to get it done.  I emailed a bunch of Mozilla folks looking for ideas, and got some great suggestions.  A few of them were too complicated, too "you'd have to be deep in Mozilla lore to even understand why this matters", too big for the time we had, or too critical for upcoming releases.  But there was one that sounded possible, "We need someone to implement WebVTT" suggested Chris Pearce.

I've spent the past three years working on various projects connected to HTML5 audio and video, and I love the idea of staying in that domain for yet another project.  HTML5 audio and video continue to get more and more exciting, and I'm happy to do more to help it evolve.  Funnily enough I had another student write WebVTT support into Popcorn.js a few terms ago, so it's nice to circle back and do the native implementation now.  Also, as a member institution of the Inclusive Design Institute, I'm attracted to WebVTT's potential for improving accessibility for web media.

Working on WebVTT in a Mozilla Context

WebVTT (Web Video Text Track) is a file format that, when paired with the HTML5 track element, allows one to pair subtitle, caption, or other metadata about a media resource with an audio or video element.  Silvia Pfeiffer gave an excellent talk about it at Google in 2011.  Our plan is to start by writing a stand-alone implementation of the WebVTT parser and then to hook that into the track element in Firefox.  Mozilla media hacker Ralph Giles has already started work on both (parser is here, track element work is here).  There's a long road to finishing WebVTT and track in Firefox, and it's unclear to any of us how far we'll get.  Part of taking on a project like this is not allowing yourself to get overwhelmed by the end goal, and instead making small steps that add up over time.

My plan is to work with the students to plan and execute this project.  Our first deliverable, due next week (get busy guys!), is a set of conformance tests for the WebVTT file syntax and parser.  Once that's done I'll have people split out on different paths, some people figuring out UTF-8, getting the parser passing all the tests, dealing with edge cases, etc, and others working on DOM bits in Firefox, hooking the parser into the media code, etc.

Because this is real, and because this is open source, you're welcome to join us.  People often tell me they'd like to come and learn Mozilla at Seneca; well, you don't have to be physically present to do it.  We'll be working on the parser in github using Ralph's repo, and Bugzilla for the DOM/media work (I'll post bugs as they get filed).  The students will be in #seneca, #introduction, and #media mostly, so if you want to talk to them on irc, those are the places to do it.  Feel free to help us write code, test things, give advice, encouragement, or just say hello.  Working on something this complicated is daunting for a student, and part of the reason I know it will work is because of our community and the support available to them.

As usual, I'm somewhat terrified of the project, unsure if it's going to work, trying to figure out how to teach all the things the students need to know in order succeed.  At the same time, after living through it in a bunch of other projects with students, I've come to recognize that this terror is not really a bad thing: it motivates; it excites;  it humbles.  I want to give the students a realistic experience of being out in the wild, and I can't do that by playing it safe myself.  So here goes nothing...here comes WebVTT, implemented by a bunch of college students!