Understanding the cost of arrays with canvas

by David Humphrey

Yesterday, Daniel and I spent a bunch of time optimizing his patch to add PImage to processing.js.  One of the many things it allows you to do is operate on the pixels of an image "off screen," then draw all or some of the resulting image.  To implement this on top of canvas we have to work with getImageData and putImageData.  It turns out that how you do this is really important for performance.

In one of his test cases, he had a series of large images that were being flipped through as a slideshow.  Processing provides a draw() function, which is basically a loop allowing for animation to take place: whatever you do in draw() happens at the specified frame rate.  Or, it tries to run at that rate.  When we ran the test with his initial code, we were getting ~6 fps which seemed...slow.  After a bunch of profiling and debugging, we got that to 80-100 fps.

I took a look at his test case under Shark (Benjamin has a great post on how to use it) and noticed that we were spending 9.8% of our time in js_CoerceArrayToCanvasImageData.  Reading through the code it started to make sense.  In Daniel's code, he had constructed his own ImageData object using a js array:

var imgData = {data: [], width: w, height: h};

Then, whenever he needed to alter pixels, he'd set imgData.data[n] as usual before updating the canvas:

putImageData(imgData, 0, 0);

On paper it looks good.  But it's not fast.  Every time you do this (and we have to do it a lot, and as fast as possible), the canvas context looks at the data array, sees that it's just a js array, and then coerces it into the proper type.  And that takes time.

I wrote a little test to demonstrate the issue.  This test picks 100 random gray scale colours and draws them one-by-one, pixel-by-pixel, to a 400 x400 canvas.  It does it first by using a custom ImageData object with a normal js array, then using a native ImageData object, returned via getImageData (NOTE: Opera doesn't support createImageData, so I've used getImageData so I could test this on various browsers).  Here's what I see on my MacBook Pro running OS X 10.6:

Browser JS Array Time/Run Native Array Time/Run
Firefox 3.6 48.9ms 48.46
Firefox Trunk 15.85ms 8.95ms
Chrome Unsupported 51.87ms
Safari Unsupported 9.3ms
Opera 1460.08ms 296.24ms
Right off the top, if you want this to work across all browsers supporting canvas, you shouldn't use custom objects with js arays--it won't work on WebKit.  But ignoring that, you shouldn't do this on those browsers that do support it because you pay a big price.  It's not surprising to see Firefox 3.6 showing no big difference between the two methods, since Vlad's WebGL arrays aren't in there.  But you can really see the effect in a trunk build that has them, which is faster than any other browser I tested (NOTE: I didn't test the equivalent trunk builds for the other browsers, just releases).

As with all performance tests, please don't take this as the final word, or generalize this as though I'm saying something about canvas performance in general.  I'm talking very specifically about the pixel array container you choose, and the numbers all lean toward using getImageData over [].  It's a 6 fps vs. 80-100 fps kind of choice in our case.  Not a hard decision.