Problem
How to interpret the results of running one or more TestCase classes.
Solution
The red or green bar indicates the overall status while the error and failure counts indicate how many tests had problems.
Detailed explanation
This recipe extends the Create an application to run FlexUnit tests recipe and uses the TestCase classes from Create a FlexUnit TestCase and Run code before and/or after every test.
The primary indication of the result of running a TestSuite is the red or green status bar. This is a quick visual summary of the test results. Green means that all tests ran without any errors or failures. Red means that at least one test had an error or failure. Any example successful run looks like this:
The run count indicates the current status of the test run. The first number reports which test is currently running and the second number reports the total number of tests that will be run. These numbers are for the total number of tests, not the total number of TestCases. For example if a TestSuite contains only a single TestCase but that TestCase contains 5 different test methods, the total run count would be 5. In the above example only a single test was run.
The errors count indicates the number of unanticipated problems that occurred when running the tests. An unanticipated problem is code which caused the test to stop executing, but wasn't explicitly tested for in an assertion. A common error is accessing a null object reference. Anything which would cause code to stop executing as intended will be captured as an error. In the example below out of the 3 tests that were run, 1 resulted in an error:
The failures count tracks the number of assertions that didn't hold. A test method will report at most a single failure since the first assertion that fails will stop execution of the test method. In the example below out of the 3 tests that were run, 1 resulted in a failure:
The failures tab gives detailed information about tests which had an error or a failure. The list indicates the name of the test method that had a problem along with the TestCase that the method is part of. Selecting a failed test will populate the stack trace with information about how the test failed. In the error and failure screenshots above the first line of the output is the reason for the problem. The rest of the stack trace window includes a stack trace of the code at the point that the problem occurred.
The all tests tab is a list of every test that was run and whether it passed or failed. Clicking on a failed test in this list will display the stack trace as described above. In this example 2 tests failed and 1 passed:
1. Status Bar
The primary indication of the result of running a TestSuite is the red or green status bar. This is a quick visual summary of the test results. Green means that all tests ran without any errors or failures. Red means that at least one test had an error or failure. Any example successful run looks like this:
2. Run Count
The run count indicates the current status of the test run. The first number reports which test is currently running and the second number reports the total number of tests that will be run. These numbers are for the total number of tests, not the total number of TestCases. For example if a TestSuite contains only a single TestCase but that TestCase contains 5 different test methods, the total run count would be 5. In the above example only a single test was run.
3. Errors Count
The errors count indicates the number of unanticipated problems that occurred when running the tests. An unanticipated problem is code which caused the test to stop executing, but wasn't explicitly tested for in an assertion. A common error is accessing a null object reference. Anything which would cause code to stop executing as intended will be captured as an error. In the example below out of the 3 tests that were run, 1 resulted in an error:
4. Failures Count
The failures count tracks the number of assertions that didn't hold. A test method will report at most a single failure since the first assertion that fails will stop execution of the test method. In the example below out of the 3 tests that were run, 1 resulted in a failure:
5. Failures Tab and Stack Trace
The failures tab gives detailed information about tests which had an error or a failure. The list indicates the name of the test method that had a problem along with the TestCase that the method is part of. Selecting a failed test will populate the stack trace with information about how the test failed. In the error and failure screenshots above the first line of the output is the reason for the problem. The rest of the stack trace window includes a stack trace of the code at the point that the problem occurred.
6. All Tests Tab
The all tests tab is a list of every test that was run and whether it passed or failed. Clicking on a failed test in this list will display the stack trace as described above. In this example 2 tests failed and 1 passed:
Report abuse
