High quality, high performance thumbnails in Flash

Ever need a thumbnail of an image in Flash? I do, and honestly speaking, the resampling that Flash does is less than ideal. Unless you only need to resize by half or bigger. But my thumbnails usually need to be smaller.

I searched for a solution and found on voq.com a promising library with some algorithms that worked quite nicely and a demo. The quality was nice, but the speed was slow. It also fiddled with the color a little bit. If I put the “easyScaling” parameter down from .5 to .25 I ended up with a nicer thumbnail but was slower to make and had more color disfiguration. You could tell what the original piece was better though.

After some more searching I found that Brooks Andrus played with some algorithms using Pixel Bender. The conclusion to that was he found he could do the same thing with the “smoothing” option in the BitmapData.draw() method. And this still leaves me with yucky thumbnails at small sizes.

I began thinking I would need to take the algorithms from the first source and port them into pixel bender like Brooks did for his ThumbGenie application.

I thought I’d try one more option first. I had played around in my mind with the idea of resizing a bitmap in half, then in half again, until I reached my destination size, since resizing by half still had good results. After discussing it with Tyler I tried it out and ended up with some great results. That last resize to get to the final thumbnail size wasn’t half because in an imperfect world your thumbnails aren’t always a power of two times smaller than your original. So I had an odd-man-out scale at the end I was applying. Tyler suggested I put that odd-man-out scale at the first instead. So instead of:

1000x750 * .5 resize * .5 resize * .5 resize * .8 = 100x75

It worked like this:

1000x750 * .8 resize * .5 resize * .5 resize * .5 = 100x75

Doing it that way landed me with even better results! It seems that Flash does it’s best work when resizing by exactly 1/2.

Thumbnail tests

The above image shows my test results as follows taking a snapshot of my homepage in an HTMLLoader in AIR.

  1. Regular BitmapData.draw() method without smoothing (0 miliseconds)
  2. Regular BitmapData.draw() method with smoothing (1 miliseconds)
  3. Using my own method that resizes by the odd scale first, then by halves (8 milliseconds)
  4. Using my method but after the odd scale, only scaling down by quarters (33 miliseconds)
  5. Using the voq Lanczos3 method with easyScaling at .25 and sharpening (694 miliseconds)
  6. Using the voq Lanczos3 method with easyScaling at .25 and no sharping (676 miliseconds)
  7. Using the voq Triangle method with easyScaling at .25 and sharpening (275 miliseconds)
  8. Using the voq Triangle method with easyScaling at .25 and no sharpening (266 miliseconds)

My personal opinion is that the best looking thumb is #3. I was surprised that #4 wouldn’t be as good. I knew the performance would be lower because I was doing more iterations of BitmapData.draw(), but I thought that scaling it to 75% (down a quarter) each time would end up nicer. Looks like 50% is the best scale to use.

The voq.com algos looked pretty decent, but obviously were slow. I’m quite happy with the solution I’ve found. I only had to write a little bit of code, it’s fast enough, and looks better than any other solution I’ve found (even Fireworks resizing IMO).

Here is the method I wrote for this. Note, when scaling up, it seemed to look better to just use smoothing and do it in one draw() and not iterations of 2.

private static const IDEAL_RESIZE_PERCENT:Number = .5;

public static function resizeImage(source:BitmapData, width:uint, height:uint, constrainProportions:Boolean = true):BitmapData
{
	var scaleX:Number = width/source.width;
	var scaleY:Number = height/source.height;
	if (constrainProportions) {
		if (scaleX > scaleY) scaleX = scaleY;
		else scaleY = scaleX;
	}

	var bitmapData:BitmapData = source;

	if (scaleX >= 1 && scaleY >= 1) {
		bitmapData = new BitmapData(Math.ceil(source.width*scaleX), Math.ceil(source.height*scaleY), true, 0);
		bitmapData.draw(source, new Matrix(scaleX, 0, 0, scaleY), null, null, null, true);
		return bitmapData;
	}

	// scale it by the IDEAL for best quality
	var nextScaleX:Number = scaleX;
	var nextScaleY:Number = scaleY;
	while (nextScaleX < 1) nextScaleX /= IDEAL_RESIZE_PERCENT;
	while (nextScaleY < 1) nextScaleY /= IDEAL_RESIZE_PERCENT;

	if (scaleX < IDEAL_RESIZE_PERCENT) nextScaleX *= IDEAL_RESIZE_PERCENT;
	if (scaleY < IDEAL_RESIZE_PERCENT) nextScaleY *= IDEAL_RESIZE_PERCENT;

	var temp:BitmapData = new BitmapData(bitmapData.width*nextScaleX, bitmapData.height*nextScaleY, true, 0);
	temp.draw(bitmapData, new Matrix(nextScaleX, 0, 0, nextScaleY), null, null, null, true);
	bitmapData = temp;

	nextScaleX *= IDEAL_RESIZE_PERCENT;
	nextScaleY *= IDEAL_RESIZE_PERCENT;

	while (nextScaleX >= scaleX || nextScaleY >= scaleY) {
		var actualScaleX:Number = nextScaleX >= scaleX ? IDEAL_RESIZE_PERCENT : 1;
		var actualScaleY:Number = nextScaleY >= scaleY ? IDEAL_RESIZE_PERCENT : 1;
		temp = new BitmapData(bitmapData.width*actualScaleX, bitmapData.height*actualScaleY, true, 0);
		temp.draw(bitmapData, new Matrix(actualScaleX, 0, 0, actualScaleY), null, null, null, true);
		bitmapData.dispose();
		nextScaleX *= IDEAL_RESIZE_PERCENT;
		nextScaleY *= IDEAL_RESIZE_PERCENT;
		bitmapData = temp;
	}

	return bitmapData;
}

Enjoy!

Update: I was getting unsatisfactory images when the resize scale was more than 50%. For example, the original code posted would size an image from 100×100 to 60×60 using one BitmapData.draw() step. And it didn’t look that great.

I found that if I sized the image up to a scale that allowed it to be size back down by exactly 50% that the results were much better. In the above example, the 100×100 would scale up by 120% to 120×120, then scale down 50% to 60×60. The final image looks much better this way. The code has been updated to work like this. It also had the option to turn constrain proportions off.

Update: I’ve posted my code library to Google Code. You can see my final implementation of bitmap resizing in the ImageUtils class.

Would it be bad to leave behind our Flash roots?

I am working with Tyler on Stealth, our high-performance component framework. After reading this article on performance by Arno Gourdol of Adobe I began wondering if we should leave behind our Flash roots of motion and timeline design by defaulting framerate to 0 in our Stealth-based applications.

Framerate makes great sense when doing games or timeline based animations, but in applications do we need it? We can update the screen on mouse moves, roll overs, etc. with the MouseEvent.updateAfterEvent instance method. And for transitions and tweening the class could use a Timer for the duration of the animation and again call Timer.updateAfterEvent. Then the screen would only refresh when it needs to. Performance would be greatly increased. Seems like it makes sense. Would this be something to add to Flex? Would it give us the performance we need/want for mobile applications and more responsive desktop applications?

Any foreseeable drawbacks? What do you think?

Update: I did some testing and it seems that the Timer class is directly influenced by the frameRate. With a frame rate of 0 a timer which should fire immediately (set to 0ms) doesn’t fire for 20 seconds! With the frame rate a 0.1 it happens at about 2 seconds, a frame rate of 1 is about 145 ms and a frame rate of anything over 4 seems to be around the same (10ms – 30ms probably depending on what the OS is currently doing).

MouseEvent updating and such happen as they should however, so as long as you start of with a frame rate of 4 so that the app can initialize visually, you could drop it down to 0 until a tween is needed and then bump it up to 4 for the duration of the tween. The rest of the visual changes can respond to mouse events (resize, click, rollover, etc). Or leaving it at 4 frames a second probably isn’t too bad on CPU, either way, you’d have to call updateAfterEvent when needed.

Overriding Flash’s Transform and Matrix

I’m building a a commercial library for scrap-book or other composition type applications and it would be useful to be able to override the flash.geom.Transform and flash.geom.Matrix classes with my own. So I played around with it for awhile and thought I’d share what I found.

First I tried setting my own matrix. I know that when you access displayObject.transform.matrix you get a clone of the object’s transform matrix, so I create a test class called MyMatrix.

public class MyMatrix extends Matrix
{
	public function MyMatrix(a:Number=1, b:Number=0, c:Number=0, d:Number=1, tx:Number=0, ty:Number=0)
	{
		super(a, b, c, d, tx, ty);
	}
	
	public override function clone():Matrix
	{
		trace("cloned");
		return new MyMatrix(a, b, c, d, tx, ty);
	}
}

Then tested it out:

var sprite:Sprite = new Sprite();
sprite.transform.matrix = new MyMatrix();
var m:Matrix = sprite.transform.matrix; // will I get a trace of "cloned"?
trace(m is MyMatrix); // true? or false?

Turns out I don’t get any trace statement when accessing matrix and “m is MyMatrix” gives me a false. Bummer. So I did the same thing with Transform, making a MyTransform class and setting it onto a sprite. Same results. Flash doesn’t really set and keep around my own transform or matrix classes. It seems to use their values and discard the objects. I’m sure it has to do with how the display list is implemented under the hood.

However, I did discover that to affect the display object your transform class doesn’t need to be assigned to it. Here is an example:

var sprite:Sprite = new Sprite()
var t:MyTransform = new MyTransform(sprite); // Transform takes a displayObject as it's one and only parameter
t.matrix = new Matrix(2, 0, 0, 2, 100, 100); // scale it up 2ce and set x/y to 100

trace(sprite.transform.matrix); // (2, 0, 0, 2, 100, 100)

I was able to set the matrix with my own transform class from outside the display object. Side note: speaking with Tyler about transform requiring a display object, he’s convinced this is the only reason you can’t serialize display objects to a bytearray. It throws an error when constructors require parameters, and Transform requires this one parameter. Might be useful to serialize display objects, maybe the Flash team could make the parameter optional! And have the display object set it when it is added as the transform.

So my two options are now:

  1. Use my own Transform class externally as explained above. I could keep a dictionary of transform objects to the data that I needed.
  2. Subclass Sprite, Shape, Bitmap, etc. and have it store the transform object of my choice.

I’ll probably go with option two. Maybe I’ll even be able to make a display object serializable with some trickery? Ooh, I gotta go try that! Catcha later.

Flight Framework Highlights

The Flight Framework is basically a collection of great utilities and independent tidbits that are organized to help us build our application. So I thought I’d make a call out to all the great standalone features in Flight, tell what they are, and leave it to another day to go more in depth on them.

  1. Fix for abstract classes
    The tiny static classes in the flight.error package allow you to ensure an abstract class is not instantiated and that abstract methods are overriden.
  2. Registry
    A global registry to store objects or data that anything in an application can have access too.
  3. Reflection
    The “Type” class handles all your reflection needs, including caching the expensive describeType calls and helping pull out needed metadata.
  4. Nice base for models
    The ValueObject class is a lazy IEventDispatcher that implements an equals() and clone() method which will work with all subclasses.
  5. ValueObjectEditor
    This nifty utility lets you “edit” a ValueObject and then either commit the changes or revert them. Great for dialogs that let ValueObjects be changed but can then either Cancel or Ok the change.
  6. AS3 ArrayContainer
    When working in an AS3 project, it would be nice to know when an array is changed, but Flex’s ArrayCollection brings in half the framework when referenced. This little guy is the little brother AS3 has been wishing for.
  7. Services
    Some common services (RPC services, e.g. remoting, HTTP, etc.) that can be used in Flex or AS3. Much smaller than Flex’s built in services.
  8. Configuration
    Config lets you define your application configurations in the app, an XML file, SharedObjects, and even the URL. Not only that, you can have multiple sources with some overriding others, and it is all accessible to your classes through a global config property. Super neat!
  9. Undo framework
    With CommandHistory and all the command interfaces available, you can build undo-redo into your application without using any of the rest of Flight. Way, way awesome.
  10. Weak-referenced binding
    Using the Bind class you can create faster, smaller, and weak-referenced binds. With two-way deep binds and being able to use in AS3-only projects without importing half the Flex framework, this is my favorite nugget.

None of the above pieces references any of the others. They can all be used independently in your own frameworks or applications. That’s my favorite part about Flight, is I can use it 100% if I want to, but if I can’t that doesn’t mean I have to give up my favorite features. I could use any/all of the above with any other framework, AS3 only, or Flex.

Undo/Redo for all your TextFields

I made the TextField undo component. Add this component to any Flash or Flex application and all your text components will have undo and redo. This has been open sourced and is on http://code.google.com/p/undo-textfields/.

Instead of making different Flex and Flash components to use in place of the available components, I made one that adds undo to existing text components. It should work in Flash and in Flex, though I just have a Flex example to show you today. You can see that it can be container specific, though you would probably just put it on the root of your application or the stage. Type in the text components in the top panel. Then do the same in the bottom. You should be able to press Ctrl+Z and Ctrl+Y (or Ctrl+Shift+Z) to undo and redo. If you’re on a Mac just substitute Ctrl with Cmd. I’ve only tested it in Safari and Firefox on a Mac. I hope the key command work on other machines without issue. But I also added a couple of menu items to the right-click context menu. Give it a whirl below, view the source, and see how easy it is to add text undo and redo to all your existing applications without any refactoring.

Update: The keyboard shortcuts don’t work in the ActiveX version of Flash (Internet Explorer on Windows).

Let me know if the key-shortcuts don’t work on your system. Maybe phase 2 can be some javascript magic to capture those better.

Undo in TextFields

I presented on creating your own undo/redo at 360 MAX unconference and a uFlash meeting. People seemed interested in knowing how to do it, but were more interested in having the code. It really should be part of Flex TextAreas and TextInputs. So I was going to code up a couple custom components that extended TextArea and TextInput (e.g. UndoableTextArea) and release them for use. However, as Tyler reminded me, that wouldn’t work in Flash apps, only Flex. So I have a proposal and would like to get your feedback.

If I created a component that you put on the root of your application (or any MXML component) which added undo/redo to any/all INPUT TextFields with in Flash or Flex would that be better? If you’re not doing any custom INPUT TextFields I don’t see any issues. But it could cause problems if you’re doing your own text components. Also, I would do it by listening to Event.ADDED events and whenever a TextField is added to the display list it would initialize it and add some right-click menu items to it. And I can even make sure when I have script access through ExternalInterface to capture the Ctrl+Z key combo.

Comment and let me know if this would be better than the Flex components. It would work in Flash apps too is the nice part and also you could add it to existing applications without any find-replace. The downsides is it could interfere with text manipulation code you may have in place for custom text components. I’ll decide this week and get to work on it next!

« Previous PageNext Page »