Everything you see in the demo's 3D world is done using <canvas> with a WebGL context. The city and all other 3D structures, the models and animation, were created using Lightwave. We used CubicVR.js to work with the resulting 3D XML data, to build and power the scenes. We texture the buildings and other structures with a combination of images (regular jpg, png), videos (WebM), and 2D canvases. For example, we create various effects (interlacing on billboards, ticker-tape displays, audio visualizations) using Processing.js on 2D canvases, and then texture those onto the 3D objects.
We wanted to build a hybrid of what people are used to seeing in the regular web with the 3D capabilities of WebGL. We did this by pulling in live data from Twitter (see the ticker tape animations for #firefox4 tweets) and Flickr (see photo billboards). This makes the demo different for every run--we've had some interesting moments at conferences running this live and pulling things from Twitter.
Speaking of the music, we're using an Ogg Theora version of Shpongle's "Nothing is Something Worth Doing" (Copyright Twisted Records, UK, used with permission). Grab headphones or turn up your speakers to enjoy this awesome track. We listened to it hundreds of times building this, and it's never gotten old.
We built this demo in our spare time over a 4 week period with nightly builds of what is now becoming Firefox 4. We've been working with these technologies for a while now, and can tell you that the potential of Firefox 4, with all the new graphics and media features it's shipping, is amazing. The web platform is capable of an amazing amount, and it's exciting to think about where people will take it next. I know we're working on a new, larger demo already :)
FOTN was built by the same people who have been working on the Audio Data API and demos with me, and I'll end by naming them all. They are an amazing group of developers to work with: Charles Cliffe, Corban Brook, Al MacDonald, Andor Salga, Maciej Adwent, Boaz Sender.