Testing visual components with FlexUnit

Introduction

Testing the behavior of visual components some may argue strays from the goal of unit testing. Part of the reason for this argument is that it is hard to isolate the class being tested to allow for controlled test conditions. The testing of components is complicated by the richness of the Flex framework in how it determines when certain methods such as measure() get called. The influence of styles and parent containers can also impact how a component behaves. As such the testing of visual components can be better thought of as automated functional testing. This recipe outlines techniques for using FlexUnit to test visual components.

In order to test the behavior of a visual component it needs to have gone through the various component life cycle steps. The Flex framework automatically handles this when a component is added to the display hierarchy. TestCases are not visual components which means that the component must be associated with an object external to the TestCase. This external association means that extra care must be taken to clean up after both failed and successful tests, otherwise stray components could inadvertently impact other tests.

This recipe extends How to handle events in a TestCase? and Create a FlexUnit TestCase.
 

Component Testing Pattern

The simplest way to get a reference to a display object that the component being tested can be added to is to use Application.application. Since the TestCase is running within a Flex application this singleton instance is available. The creation and activation of a visual component is not a synchronous activity which means that before it can be tested the TestCase needs to wait for the component to get into a known state. Waiting for the FlexEvent.CREATION_COMPLETE event by using addAsync() is the easiest way to reach that known state for a newly created component. To ensure that one TestCase method doesn't impact the running of another TestCase method the component created needs to be cleaned up and any external references to it removed. Using the tearDown() method and a class instance variable is the best way to accomplish these two tasks. The sample code below shows the creation, attachment, activation, and cleanup pattern for an instance of Tile:

package mx.containers
{
    import flexunit.framework.TestCase;
    
    import mx.core.Application;
    import mx.events.FlexEvent;

    public class TileTest extends TestCase
    {
        // class variable allows tearDown() to access the instance
        private var _tile:Tile;
        
        override public function tearDown():void
        {
            try
            {
                Application.application.removeChild(_tile);
            }
            catch (argumentError:ArgumentError)
            {
                // safe to ignore, just means component was never added    
            }
            _tile = null;
        }
        
        public function testTile():void
        {
            _tile = new Tile();
            _tile.addEventListener(FlexEvent.CREATION_COMPLETE, addAsync(verifyTile, 1000));
            Application.application.addChild(_tile);
        }
        
        private function verifyTile(flexEvent:FlexEvent):void
        {
            // component now ready for testing
            assertTrue(_tile.initialized);
        }
    }
}

The key points to note are that a class variable is defined to allow the tearDown() method to reference the instance that was created and added to Application.application. The addition of the component to Application.application may not have succeeded which is why the tearDown() method wraps the removeChild() call in a try catch block to prevent any erroneous errors from being reported. The test method uses addAsync() to wait for the component to be in a stable state before running tests against it.

Component Creation Testing

While it is possible to manually call the various Flex framework methods like measure() on a component instance, by having it be part of the display hierarchy the test better simulates the environment the object will run in. Unlike a unit test the environment external to the component isn't tightly controlled which means extra care must be taken to focus the testing on the component and not the surrounding environment. As an example the layout logic of the Tile container created above can be tested by adding children to it:

        public function testTileLayout():void
        {
            _tile = new Tile();
            var canvas:Canvas = new Canvas();
            canvas.width = 100;
            canvas.height = 100;
            _tile.addChild(canvas);
            canvas = new Canvas();
            canvas.width = 50;
            canvas.height = 50;
            _tile.addChild(canvas);
            canvas = new Canvas();
            canvas.width = 150;
            canvas.height = 50;
            _tile.addChild(canvas);
            _tile.addEventListener(FlexEvent.CREATION_COMPLETE, addAsync(verifyTileLayout, 1000));
            Application.application.addChild(_tile);
        }
        
        private function verifyTileLayout(flexEvent:FlexEvent):void
        {
            var horizontalGap:int = int(_tile.getStyle("horizontalGap"));
            var verticalGap:int = int(_tile.getStyle("verticalGap"));
            assertEquals(300 + horizontalGap, _tile.width);
            assertEquals(200 + verticalGap, _tile.height);
            assertEquals(3, _tile.numChildren);
            assertEquals(0, _tile.getChildAt(0).x);
            assertEquals(0, _tile.getChildAt(0).y);
            assertEquals(150 + horizontalGap, _tile.getChildAt(1).x);
            assertEquals(0, _tile.getChildAt(1).y);
            assertEquals(0, _tile.getChildAt(2).x);
            assertEquals(100 + verticalGap, _tile.getChildAt(2).y);
        }

In this example three children are added to the Tile with various sizes. Based on the Tile layout logic it should create a 2 x 2 grid and make each tile within the grid the maximum width and height found among the children. The verify method asserts that the default logic does in fact produce this result. It is important to note that the test is focusing only on the logic used by the component. This isn't testing if the layout looks good, just that its behavior matches the documentation. Another important point to note about testing components at this level is the effect that styles can have on a component. The dynamic lookup of the horizontalGap and verticalGap in the verify method is one way to make the test less brittle in case the default values change. This test method could have instead set the style values when it created the instance to ensure the values being used.

Post Creation Testing

Once the component is created additional changes made to it can be tricky to test. The generic FlexEvent.UPDATE_COMPLETE event is tempting to use but can fire multiple times as a result of a single change made to a component. While it possible to setup logic that correctly handles these multiple events the TestCase ends up inadvertently testing the Flex framework's event and UI update logic instead of the logic just within the component. As such designing the test to focus just on the component's logic becomes something of an art. This is another reason why most consider component testing at this level to be functional testing instead of unit testing.

Below is an example of adding another child to the Tile created above and detecting that the change has been made:

        // class variable to track the last addAsync() Function instance        
        private var _async:Function;

        public function testTileLayoutChangeAfterCreate():void
        {
            _tile = new Tile();
            var canvas:Canvas = new Canvas();
            canvas.width = 100;
            canvas.height = 100;
            _tile.addChild(canvas);
            canvas = new Canvas();
            canvas.width = 50;
            canvas.height = 50;
            _tile.addChild(canvas);
            canvas = new Canvas();
            canvas.width = 150;
            canvas.height = 50;
            _tile.addChild(canvas);
            _tile.addEventListener(FlexEvent.CREATION_COMPLETE, addAsync(verifyTileLayoutAfterCreate, 1000));
            Application.application.addChild(_tile);
        }
        
        private function verifyTileLayoutAfterCreate(flexEvent:FlexEvent):void
        {
            var horizontalGap:int = int(_tile.getStyle("horizontalGap"));
            var verticalGap:int = int(_tile.getStyle("verticalGap"));
            assertEquals(300 + horizontalGap, _tile.width);
            assertEquals(200 + verticalGap, _tile.height);
            assertEquals(3, _tile.numChildren);
            assertEquals(0, _tile.getChildAt(0).x);
            assertEquals(0, _tile.getChildAt(0).y);
            assertEquals(150 + horizontalGap, _tile.getChildAt(1).x);
            assertEquals(0, _tile.getChildAt(1).y);
            assertEquals(0, _tile.getChildAt(2).x);
            assertEquals(100 + verticalGap, _tile.getChildAt(2).y);

            var canvas:Canvas = new Canvas();
            canvas.width = 200;
            canvas.height = 100;
            _tile.addChild(canvas);
            _async = addAsync(verifyTileLayoutChanging, 1000);
            _tile.addEventListener(FlexEvent.UPDATE_COMPLETE, _async);
        }
        
        private function verifyTileLayoutChanging(flexEvent:FlexEvent):void
        {
            _tile.removeEventListener(FlexEvent.UPDATE_COMPLETE, _async);
            _tile.addEventListener(FlexEvent.UPDATE_COMPLETE, addAsync(verifyTileLayoutChangeAfterCreate, 1000));
        }
        
        private function verifyTileLayoutChangeAfterCreate(flexEvent:FlexEvent):void
        {
            var horizontalGap:int = int(_tile.getStyle("horizontalGap"));
            var verticalGap:int = int(_tile.getStyle("verticalGap"));
            assertEquals(400 + horizontalGap, _tile.width);
            assertEquals(200 + verticalGap, _tile.height);
            assertEquals(4, _tile.numChildren);
            assertEquals(0, _tile.getChildAt(0).x);
            assertEquals(0, _tile.getChildAt(0).y);

            assertEquals(200 + horizontalGap, _tile.getChildAt(1).x);
            assertEquals(0, _tile.getChildAt(1).y);

            assertEquals(0, _tile.getChildAt(2).x);
            assertEquals(100 + verticalGap, _tile.getChildAt(2).y);

            assertEquals(200 + horizontalGap, _tile.getChildAt(3).x);
            assertEquals(100 + verticalGap, _tile.getChildAt(3).y);
        }

The event handling logic now uses a class variable to track the last added asynchronous function created by addAsync() to allow it to be removed and a different listener added to handle the second time the same event type is fired. If an additional change was going to be made that would fire another FlexEvent.UPDATE_COMPLETE the verifyTileLayoutChanging() method would also have to store its addAsync() function to allow it to be removed. This chained event handling is brittle in that if the Flex framework logic changes how such events are fired the code might fail. The test doesn't care that two FlexEvent.UPDATE_COMPLETE events fire in order for the component to complete its task of laying out its children, it is an unintended effect of trying to capture component logic at this level. If the intermediate state captured in verifyTileLayoutChanging() is vital to the logic of the component then the assertions made in that method would have merit and a change in the number of events should warrant this test failing if the events are not fired correctly.

While a component may dispatch additional events such as Event.RESIZE the component state at the point the event is dispatched is usually unstable. In the case of a Tile when Event.RESIZE is dispatched the component's width has changed but the position of the children has not. Additionally there maybe actions queued via callLater() such that removing the component from the display hierarchy will cause errors when the queued actions attempt to execute. Some of these issues maybe avoided when testing a component whose update logic is synchronous, removing the need for any event handling. Alternatively the component being tested may dispatch an event that clearly defines when a change has been fully realized. Whichever method is chosen to handle such cases it is important to keep in mind how brittle these approaches are and how much they inadvertently test behavior outside of the component.

Testing with Timers

If many complex changes are being made to a component at once the number and order of events dispatched maybe too cumbersome to maintain. Instead of waiting for a specific event another approach is to wait for a period of time. This approach makes it easy to handle multiple objects that are being updated or a component that uses Effect instances that take a known amount of time to play. The primary drawback is that testing based on time can produce false positives if the speed or resources of the testing environment change. Waiting for a fixed amount of time also means that the runtime of the entire TestSuite increases more rapidly than just adding another synchronous or event driven test.

The Tile example from above can be written using timer based triggers as shown below:

        private function waitToTest(listener:Function, waitTime:int):void
        {
            var timer:Timer = new Timer(waitTime, 1);
            timer.addEventListener(TimerEvent.TIMER_COMPLETE, addAsync(listener, waitTime + 250));
            timer.start();    
        }
        
        public function testTileLayoutWithTimer():void
        {
            _tile = new Tile();
            var canvas:Canvas = new Canvas();
            canvas.width = 100;
            canvas.height = 100;
            _tile.addChild(canvas);
            canvas = new Canvas();
            canvas.width = 50;
            canvas.height = 50;
            _tile.addChild(canvas);
            canvas = new Canvas();
            canvas.width = 150;
            canvas.height = 50;
            _tile.addChild(canvas);
            Application.application.addChild(_tile);
            waitToTest(verifyTileLayoutCreateWithTimer, 500);
        }
        
        private function verifyTileLayoutCreateWithTimer(timerEvent:TimerEvent):void
        {
            var horizontalGap:int = int(_tile.getStyle("horizontalGap"));
            var verticalGap:int = int(_tile.getStyle("verticalGap"));
            assertEquals(300 + horizontalGap, _tile.width);
            assertEquals(200 + verticalGap, _tile.height);
            assertEquals(3, _tile.numChildren);
            assertEquals(0, _tile.getChildAt(0).x);
            assertEquals(0, _tile.getChildAt(0).y);
            assertEquals(150 + horizontalGap, _tile.getChildAt(1).x);
            assertEquals(0, _tile.getChildAt(1).y);
            assertEquals(0, _tile.getChildAt(2).x);
            assertEquals(100 + verticalGap, _tile.getChildAt(2).y);

            var canvas:Canvas = new Canvas();
            canvas.width = 200;
            canvas.height = 100;
            _tile.addChild(canvas);
            waitToTest(verifyTileLayoutChangeWithTimer, 500);
        }
        
        private function verifyTileLayoutChangeWithTimer(timerEvent:TimerEvent):void
        {
            var horizontalGap:int = int(_tile.getStyle("horizontalGap"));
            var verticalGap:int = int(_tile.getStyle("verticalGap"));
            assertEquals(400 + horizontalGap, _tile.width);
            assertEquals(200 + verticalGap, _tile.height);
            assertEquals(4, _tile.numChildren);
            assertEquals(0, _tile.getChildAt(0).x);
            assertEquals(0, _tile.getChildAt(0).y);
            assertEquals(200 + horizontalGap, _tile.getChildAt(1).x);
            assertEquals(0, _tile.getChildAt(1).y);
            assertEquals(0, _tile.getChildAt(2).x);
            assertEquals(100 + verticalGap, _tile.getChildAt(2).y);
            assertEquals(200 + horizontalGap, _tile.getChildAt(3).x);
            assertEquals(100 + verticalGap, _tile.getChildAt(3).y);
        }

Unlike the previous test examples that could complete as fast as the events fire this version of the test has a minimum run time of one second. The additional time added to the timer delay when calling addAsync() is to handle small variances in when the timer will fire. The intermediate method to swap FlexEvent.UPDATE_COMPLETE listeners from the example above is removed but otherwise the test code remains the same.

Programmatic Visual Assertions

The ability to capture the raw bitmap data of a rendered component can make it easy to programmatically verify certain visual aspects of a component. An example would be to test that changing the background and border styles of a component changes how it is drawn. After creating an instance of the component the bitmap data can be captured and examined. Below is a sample test to verify that adding a border to a Canvas produces the intended results:

package mx.containers
{
    import flash.display.BitmapData;
    
    import flexunit.framework.TestCase;
    
    import mx.core.Application;
    import mx.events.FlexEvent;

    public class CanvasTest extends TestCase
    {
        // class variable allows tearDown() to access the instance
        private var _canvas:Canvas;
        
        override public function tearDown():void
        {
            try
            {
                Application.application.removeChild(_canvas);
            }
            catch (argumentError:ArgumentError)
            {
                // safe to ignore, just means component was never added    
            }
            _canvas = null;
        }
        
        private function captureBitmapData():BitmapData
        {
            var bitmapData:BitmapData = new BitmapData(_canvas.width, _canvas.height);
            bitmapData.draw(_canvas);
            return bitmapData;            
        }
        
        public function testBackgroundColor():void
        {
            _canvas = new Canvas();
            _canvas.width = 10;
            _canvas.height = 10;
            _canvas.setStyle("backgroundColor", 0xFF0000);
            _canvas.addEventListener(FlexEvent.CREATION_COMPLETE, addAsync(verifyBackgroundColor, 1000));
            Application.application.addChild(_canvas);
        }
        
        private function verifyBackgroundColor(flexEvent:FlexEvent):void
        {
            var bitmapData:BitmapData = captureBitmapData();
            for (var x:int = 0; x < bitmapData.width; x++)
            {
                for (var y:int = 0; y < bitmapData.height; y++)
                {
                    assertEquals("Pixel (" + x + ", " + y + ")", 0xFF0000, bitmapData.getPixel(x, y)); 
                }
            }
        }

        public function testBorder():void
        {
            _canvas = new Canvas();
            _canvas.width = 10;
            _canvas.height = 10;
            _canvas.setStyle("backgroundColor", 0xFF0000);
            _canvas.setStyle("borderColor", 0x00FF00);
            _canvas.setStyle("borderStyle", "solid");
            _canvas.setStyle("borderThickness", 1);
            _canvas.addEventListener(FlexEvent.CREATION_COMPLETE, addAsync(verifyBorder, 1000));
            Application.application.addChild(_canvas);
        }
        
        private function verifyBorder(flexEvent:FlexEvent):void
        {
            var bitmapData:BitmapData = captureBitmapData();
            for (var x:int = 0; x < bitmapData.width; x++)
            {
                for (var y:int = 0; y < bitmapData.height; y++)
                {
                    if ((x == 0) || (y == 0) || (x == bitmapData.width - 1) || (y == bitmapData.height - 1))
                    {
                        assertEquals("Pixel (" + x + ", " + y + ")", 0x00FF00, bitmapData.getPixel(x, y));
                    }
                    else
                    {
                        assertEquals("Pixel (" + x + ", " + y + ")", 0xFF0000, bitmapData.getPixel(x, y));
                    } 
                }
            }
        }
    }
}

The testBackgroundColor() method verifies that all pixels in the Canvas are assigned the background color correctly. The testBorder() method verifies that when a border is added to the Canvas the outside pixels switch to the border color while all other pixels remain the background color. The capturing of the bitmap data is handled in the captureBitmapData() method and makes use of the ability to draw() any Flex component into a BitmapData instance. This is a powerful technique that can be used to verify programmatic skins and other visual components that may otherwise be hard to unit test.

For an alternative approach to testing the visual appearance of a component look at the Visual FlexUnit package available at http://code.google.com/p/visualflexunit/.

Hiding the Component Being Tested

One side effect of adding the component to be tested to Application.application is that it will be rendered. This can cause the FlexUnit testing harness to resize and reposition as the tests are running and components are being added and removed to the display hierarchy. To suppress this behavior the component being tested can be hidden by setting its visible and includeInLayout properties to false prior to being added to the display hierarchy. For example if the Canvas should be hidden while being tested in the above code the addition of it to the display hierarchy would be rewritten as such:

            _canvas.visible = false;
            _canvas.includeInLayout = false;
            Application.application.addChild(_canvas);