Automated TestSuite Generation

Introduction

In order to run a TestCase it must be included in a TestSuite. The process of creating a new TestCase and then adding it to a TestSuite can become habit, but if multiple developers are working on the same code base the chance that a TestCase gets missed can increase. Instead of manually adding each TestCase to the TestSuite it is possible to have the TestSuite automatically generated. The open source Antennae project includes a utility to automate the creation of a TestSuite by examining a source folder and adding tests contained within it.

This recipe extends the Create an application to run FlexUnit tests and Installing and Configuring Antennae recipes and requires a basic understanding of Ant.

Generating the TestSuite

It is possible to use the Antennae TestSuite generation tool without needing to setup and build the entire project with Antennae. The Antennae package includes a JAR file in the lib subdirectory called arc-flexunit.jar which contains the class  com.allurent.flexunit.framework.AllTestsFileGenerator. By running this class against a source directory it will find all classes named Test*.as or *Test.as and create a TestSuite with them included. The utility creates the TestSuite on standard output which can be redirected to any location. The generated TestSuite file is part of the root package and is called FlexUnitAllTests. A sample Ant file to call this utility would be:

<project default="generate">
    <target name="generate">
        <java classname="com.allurent.flexunit.framework.AllTestsFileGenerator"
              fork="true" failonerror="true" logError="true"
              output="src/FlexUnitAllTests.as">
            <classpath>
                <pathelement location="../lib/arc-flexunit.jar" />
            </classpath>
            <arg value="src/" />
        </java>
    </target>
</project>

This build file would be placed in the root directory of the project that the TestSuite should be generated for. In this case ../lib/arc-flexunit.jar is the location of the jar file, com.allurent.flexunit.framework.AllTestsFileGenerator is the name of the class to run, src/ is the root of the source tree that should be examined for files, and src/FlexUnitAllTests.as is the location of the generated file.

The format of the generated TestSuite file looks like this:

package
{
    import flexunit.framework.*;
    import mx.containers.CanvasTest;
    import mx.containers.TileTest;

    public class FlexUnitAllTests 
    {
        public static function suite() : TestSuite
        {
            var testSuite:TestSuite = new TestSuite();
            testSuite.addTestSuite(mx.containers.CanvasTest);
            testSuite.addTestSuite(mx.containers.TileTest);
            return testSuite;
        }
    }
}

The generation of the FlexUnitAllTests file can be automated such that it is always created before the FlexUnit application is compiled. See the Antennae documentation for additional details including the use of the AllTestsFileGenerator utility from within Flex Buidler.

Using the Generated TestSuite

Instead of manually constructing the TestSuite in the main application the FlexUnitAllTests class is specified as the TestSuite to run. Each time the FlexUnitAllTests class is regenerated all included tests will be compiled and run. The modified FlexUnit application which uses the FlexUnitAllTests would look like this:

<?xml version="1.0" encoding="utf-8"?>
<mx:Application xmlns:mx="http://www.adobe.com/2006/mxml" xmlns:flexui="flexunit.flexui.*" creationComplete="handleCreationComplete();">
    <mx:Script>
        <![CDATA[
            import flexunit.framework.TestSuite;
            
            private function handleCreationComplete():void
            {
                testRunner.test = FlexUnitAllTests.suite();
                testRunner.startTest();
            }
        ]]>
    </mx:Script>
    <flexui:TestRunnerBase id="testRunner" width="100%" height="100%"/>
</mx:Application>

Filtering Included TestCases

By convention all files that are named Test*.as or *Test.as are included in the generated TestSuite. This behavior can be overridden by creating a filters file that specifies through the use of regular expressions what files should be included. Each line of the filters file specifies a single regular expression that each filename will be matched against. If the filename matches any of the regular expressions specified it will be included. The regular expression is run against the complete path of the filename which allows for the inclusion of TestCases based on package or other criteria. A sample filters file would look like this:

/mx/containers/.*Test.as
RegExpTest.as

The first line includes all tests in any directory under /mx/containers/. The second line includes a specific test called RegExpTest.as where ever it occurs. It is important to note that the AllTestsFileGenerator utility operates at the file system level only. As such it is important to specify Test.as in any rule to avoid picking up non TestCase files.

If the above rules where stored in a file called filters.txt in the same directory as the Ant script from above, adding another <arg/> element to the Java call would specify the filters file to use:

            <arg value="src/" />
            <arg value="filters.txt" />

When a filters file is in use a test that always fails is automatically added to the TestSuite. This is done as a reminder that certain TestCases may have been excluded and the TestSuite doesn't represent the full set of Tests that could be run.