There has been some great discussion recently within Mozilla about contributor engagement. People like David Eaves and the Metrics team have been looking at this from a data-driven perspective, and engineers like Paul Bigger have been doing it at the level of our tools and processes. It's a topic very important to me, because I am routinely bringing new students (and increasingly, web, audio, and JavaScript developers) into Mozilla. This week there was a call on the topic, and various ideas where thrown out, looking at how we might improve.
In preparation for that call, and in order to help myself think about this issue with greater perspective, I decided to try an experiment: in order to understand what it's like to be a new developer coming to Mozilla, I would fix a bug in WebKit. Even though I work with new Mozilla developers every day, I have been working on the project for too long to really look at it objectively any more. If I have a problem, I know where to look to solve it (docs, code, bugs, web sites, irc, etc), or who to talk to in order to get answers.
The idea came earlier this week when Anna hit an issue with our Popcorn.js test suite. She was trying to fix our tests so the videos didn't make any sound--something that worked right away with Firefox, but seemed impossible in Chrome. After writing a little test case, it was clear that setting muted=true before calling play() was broken in Chrome.
I know how we implement this in Firefox, and had a pretty good guess why it might be broken in Chrome, so I decided this was a perfect chance to fling myself into the unknown. It didn't take long to get there.
Issue 1: Where do I file this?
Because I was seeing the bug in Chrome it's Chrome's bug, right? No. After filing bugs in both WebKit's bugzilla and Chrome's issue tracker, I found that it did indeed belong with WebKit. Having multiple bug trackers, and really multiple projects, made it hard for me as an outsider to know where I should be working. I expressed this in the bugs, but never really got an answer.
How does Mozilla do on this front? Well, we have just one bug tracker, so problem solved! Wait, which component would you file this bug in in our system? "Obviously it belongs in Core : Video/Audio!" Obviously. But isn't this an issue in Firefox? "Well actually..." Right. "Well actually..." is what I hit with my WebKit bug too, so I'm not sure that we're doing a whole lot better. How many bugs do we have in Firefox : General?
How do you solve this? It's hard, and especially hard for the first time bug filer or developer. I think that what has to happen here is that bugs have to get triaged into the right spot. In both the bugs I filed, that eventually did happen. It happens a lot in our bugzilla too; but we could probably do a better job, especially in making sure that we help people navigate the system vs. chastise them when they get it wrong. When someone confirms, moves, comments in, or otherwise engages with your bug, you feel like you're not alone.
Issue 2: Where do I get the code?
I knew roughly what I wanted to do, but I needed to get the code, build it, write a test, etc. So, still not knowing how the Chrome vs. WebKit divide worked, I decided that I was using Chrome, so I might as well get its source. I knew that I probably wanted Chromium, so I googled that and eventually found their developer documentation. Building meant getting the code via SVN, which I haven't used in a long time. It was also recommended that I get a tarball and then update that, which I did. Even after fixing my bug, I'm not clear if this was really what I was supposed to do, because the tarball I got had duplicate code and directories, and people on irc told me that it was odd I had certain files in my tree. I followed instructions, but had no way to judge if they were right.
One lesson I took away from this experience is that the build documentation should be done in such a way that it doesn't date itself. We have been talking about making downloadable Mercurial trees in Mozilla, too, since cloning is so expensive. If we go that route, we want to make sure that our main community still uses this method, so when it gets out of synch, we can update the docs. It's hard for new people to report build problems, because you have no baseline from which to measure your own errors vs. what used to work and is now broken.
I also found Chrome's build instructions more detailed than I wanted for what I was trying to do. There is quite a bit of good info on more advanced things you could do; however, I was less interested in having to make choices, and more interested in having one way to get it working the first time. I think Mozilla's Simple Firefox Build page is useful in this regard.
Issue 3: How to build?
Now that I had a tree, I was ready to build. The Chrome build on Mac was very different from Mozilla's, in that it used XCode. I don't know how to evaluate this, because I don't ever use XCode: for me it was harder than having to use the usual GNU tools on the command line, but that is quite likely my own ignorance. If you were a Mac developer and had the chance to use XCode, that might feel really familiar. I will say that even using their command-line build instructions, their build did a better job of using my machine's resources than our default build does. Unless you know how to set the right -j value for make, you aren't going to get the best build times.
Should we use native build systems? I'll leave that to the people who want to write those patches :) I will say that we should do more of this kind of thing, and have sane defaults, which don't require you to set options in config files when you're getting started. Let that be what advanced users do.
Issue 4: OK it's fixed, but I have a question, who can help me?
Because I knew the Mozilla code for the media elements, I was able to guess my way around the Chrome code and find what I was looking for fairly easily. It took some time for me to figure out which file I should be working in (third_party/WebKit/Source/WebCore/html/HTMLMediaElement.cpp or third_party/WebKit/WebCore/html/HTMLMediaElement.cpp??), but I got there by trial and error. A bit of debugging and I had my fix: a nice one-liner! Now I needed a test.
I had a much harder time figuring out where the tests lived and how to run them. I opted to ask on irc. The developer page mentioned a #chromium channel on freenode. I started there. I was working at night, and didn't get much response the first night. The second night I was told I was in the wrong channel: "You should try in #webkit."
Figuring out where and who to ask for help is hard. I think it's probably harder in Mozilla because our main developer channel isn't publicly listed. Also, all three channels (#chromium, #webkit, and #developers for Mozilla), are not really about development--they are more often than not about managing the tree (scheduling landings, backing things out, discussing failed tests, etc.).
One thing I would love to see Mozilla do is create a channel that is like what we've done for my students with #seneca, namely, a safe channel for asking Mozilla development related questions, and importantly, one that is populated with many core developers who don't mind lending a hand. A lot of people have been talking about how we can do a better job of mentoring new developers in Mozilla. I'm skeptical of this approach. Any one person is too busy to play this role. But, if we spread that load over a larger community of people, I think it can work. In fact, I've seen it work for the past 6 years in #seneca, with people like Mike Shaver, Ted Mielczarek, Mark Finkle, Jason Orendorff, Gavin Sharp, Joe Drew, etc. If some new developer or contributor arrives on irc, where are they supposed to go? And are the people they should be meeting there now? If not, we should fix that.
Issue 5: So you fixed your bug? Nice, now jump through these hoops.
Now I had a one-line patch and a test, and I was done from my perspective. Except now the fun really started. Much like Mozilla, I needed to get my work reviewed. I read the Chromium docs on how to make a patch. I did what they said, and was never able to make their gcl script work. Eventually I gave up and just made a diff on my own, and attached that to my bug.
Choosing a reviewer was dead easy: I had no choices! I simply set r? and that was it. I liked this better than our system, where you have to know who to ask for review. If you don't know anyone, how are you supposed to know who to ask? I think Mozilla should use this system, or provide a list of people who have reviewed in your module, or something. A blank textbox that expects a name is not ideal for the first time contributor.
Issue 6: r-. "Did you fill out Forms B, C, and D?"
As I expected, my first patch was rejected. My test needed a different name, and a small fix. But more, I needed to have a ChangeLog entry for my fix. This is something we don't have in Mozilla, so it was totally new to me. As far as I was concerned, it was yet more process I had to follow in order to get this one-line fix into the tree. But what was great in this case was that my reviewer didn't just fail the patch and say, "you're missing X," he also told me how to do X.
I've seen Mozilla reviewers do this too, and also the opposite. Expecting core developers to know what to do is one thing, but not new ones. I think it's easy to get caught up in your own process, and forget that you're basically asking a volunteer to go through a bunch of bureaucratic paper-work beyond the fix they have already done. It's unavoidable, for sure. But it doesn't have to be painful. It would be great if our bugzilla had some way to flag to reviewers that this is a new contributor, and it would be great if reviewers dropped some helpful links in the bug and an invitation to talk to them on irc.
Conclusion
I haven't linked to the bugs I discussed above in order to not centre out any particular people in the WebKit community. Overall, I found the experience mostly pleasant, and on par with what it would be like to fix the same bug in Firefox. But by removing my familiarity with the code, community, and process, I was made much more aware of them. Mozilla is natural to me now, because I've spent so much time struggling through the hard bits. The WebKit code and process were no more complicated, but were very strange because of their newness.
My major takeaways from this experiment are:
- Mozilla should avoid extra process for the sake of process, and where such processes are necessary, provide help for people struggling through them. We should probably reassess our processes from time to time, to make sure they still make sense.
- Mozilla should figure out where a new developer coming to the project is going to land, and then put knowledgeable people there to greet them. I think we need a new irc channel for this.
- It's always important to encourage new people, to thank them, and to honour their gift of time. We do this, but we could do it more. I could do it more.
I'd encourage other long-time Mozilla developers to venture out of the Mozilla code/community in order to have the shock of the unfamiliar. It's uncomfortable but helpful to be humbled. I know I learned a lot. I hope the present discussions around developer engagement continue.