This post is part of a series I’m writing about my work to implement the Mouse Lock API in Mozilla. I’m doing the work with students in my Mozilla Open Source course at Seneca College, and so theses posts are intentionally didactic and self reflective. The aim of the series is to show how a new feature gets implemented in Firefox. Please note that all code discussed is unreviewed and not part of a shipping Firefox at this point.
In last week's post we finished wiring up all the internal mouse movement and DOM code necessary to expose movementX and movementY on a MouseEvent. I said that our next task would be to figure out how to reposition the mouse cursor to the centre of the screen, in effect, undoing the user's mouse movement and allowing us to achieve the "infinite scrolling" of the spec. Our goal was to accomplish the following, in a cross-platform way:
When the mouse is locked, the cursor sits in the centre of the screen, and is hidden. The user moves the mouse to the right and down (shown above in red). The difference between the centre point and this new point (i.e., screenX and screenY) is movmenetX and movementY. After we record that movement delta, we need to force the mouse back to the centre, so that subsequent user movement doesn't cause the mouse to reach the edge of the screen (shown in grey).
Before I go into the technical details, let me cut to the ending and tell you that we solved it. I recorded a video of one of CJ's CubicVR.js WebGL demos that models a simple first-person-shooter. Jesse modified it to use the Mouse Lock API. In this demo, the first half uses the standard mouse events. Notice how I can only move so far to the left or right? It's impossible to go all the way around in a circle, for example. Then, half-way through, I switch to Full Screen mode and Mouse Lock, the mouse cursor disappears, and I can rotate the camera infinitely in the X and Y axes:
We also finished the success and failure callbacks for the lock() method, and dealt with the various cases of mouse lock being lost, through losing fullscreen (e.g., user initiated, loss of focus, etc.).
The code is working well enough now that I decided to create builds that others could try on their own. Mozilla has an amazing service for its developers called the Try Server. With the Try Server I can push a changeset and have it get built, and the tests run, as though it were a real commit. By being able to try my changes on all platforms, and to run the tests on many machines (it's ~128 hours of computing time to do it), I save a ton of time as an individual developer. The Try Server's dashboard let's me see what's happening. Here's what my current build looks like as I write this, with Green showing things working, Orange a failed test or tests, Red a crash or build failure of some kind:
The finished builds will go here (they'll only be kept for a few weeks, so if that link is broken, that's why). We've also written some Mouse Lock demos, which are posted here.
In terms of the technical bits of what we did, I want to focus on the mouse repositioning, and use it as way to discuss how to deal with uncertainty, failure, and community. A number of my students asked a version of the same question this week, "How do I know if I'm just wrong or don't know what I'm doing, or if I have a real question?" Behind this lies a larger question about self-confidence as a developer, and how to work on things in the open, where your inadequacies are so easily seen by others.
As I said above, we needed to be able to reposition the mouse pointer. To do this on Windows, you use SetCursorPos(); on OS X it's CGWarpMouseCursorPosition(); and on Linux (GTK) it's gdk_display_warp_pointer. Our first experiment was to simply hack the Windows API call directly into our code so we could prototype the behaviour. Having concluded that this would work, we went looking for some kind of Mozilla cross-platform way to call these indirectly. Luckily we found what we were after, the nsWindow::SynthesizeNativeMouseEvent function does exactly what we want, hiding the platform specific code behind a common method.
We rewrote our code to use this common method, and it worked on Windows, Mac, and...failed on Linux. It turns out that not all platforms implement this method (I'm looking at you, Linux), and so get the base implementation, which throws. "No problem, we'll add it!"
We tried an experiment to hack the call to gdk_display_warp_pointer directly into our other code without doing the full fix, just to make sure that things worked on Linux the same as Windows/Mac. According to the docs, we saw that we'd need to include gdk/gdk.h, which we did. However, the compiler complained that it couldn't find gdk.h. This seemed odd. We did a search and there were clearly lots of places that it was being included within existing code. But nothing we could do would change the compiler's mind.
At this point we had hit a wall. After doing everything we could to make sure we hadn't done something stupid (it's hard, but not impossible to misspell "gdk"), I went and asked on irc. The answer came quickly via hub and kinetik: "Did you add the CXXFLAGS for gtk in your Makefile?" We hadn't known we needed to, but now we had enough to go do some more searching. Obviously in places where gdk.h is used successfully they have added this, and a quick search through widget/gtk2/Makefile.in revealed the existence of $(MOZ_GTK2_CFLAGS).
So back to my students' question about when and how to ask for help. In this case we seemed to be asking a very basic question, "how do I include a header file?" But, before we asked we made sure to do everything we were able to do on our own. We didn't start by asking someone to do something for us. But when we reached the end of what we understood, it was appropriate to go looking for help.
Community development doesn't mean that everyone helps you with everything, nor that you can sit back and let someone else do your work for you. I think you should be cautious about asking every question you have. Show people that you're willing to dig on your own, and they'll be more likely to help you when you show up needing it. However, once you have done your work, and you're still stuck, you shouldn't be nervous to ask questions, even if they seem like stupid questions. No one made fun of me for not knowing how to get this header file to include--it was a valid question.
We're just over three weeks into this work, and we've now got the code working. What's left are the rest of the tests, to fix-up our code for review, and to deal with some edge cases. I'm hoping that by the end of this week we'll have something ready for review, which will be another interesting story.