Pro-Test How To Guide
How do you get started using High Throughput Testing (HTT) to reduce the number of test cases? Below is a 5 step process to help you get started.
Step #1 - Identify the Factors
Identify the "factors" (sometimes called variables) for your project. The factors are typically items that can satisfy different conditions during different test cases. For example, if you wanted to ensure that your software would work under Windows NT, Windows 2000, and Windows XP, you would define one factor to be "Operating System". In this example, we will have four factors: OS (for Operating System), MB RAM (Megabytes of RAM), Access Type, and Browser.
Step #2 - Identify the Levels for Each Factor
For each factor, identify the levels of interest (sometimes called values). The levels are the different values the factor can take on between test cases. From Step #1, the factor "Operating System" will have three different levels: "Win NT", "Win 2000", and "Win XP".
Step #3 - Establish the Constraints
Identify the constraints for the relationships between factors. These constraints are sometimes referred to as infeasibilities. For example, you have the factor "Operating System" which has levels "Windows NT", "Windows 2000", and "Windows XP", and you have another factor "Amount of RAM" which has levels "16 MB", "32 MB", and "64 MB". Since Windows XP will not run on a computer with 16 MB of RAM, we need to establish a constraint for this relationship. This constraint would be expressed as
"If <Operating System> is at level <Windows XP> then <Amount of RAM> can't be <16 MB>".
In Pro-Test you can click on the icon labeled "Constraints" and enter this constraint.
Step #4 - Enter Any Previous Runs
In some cases you may have done some ad-hoc or exploratory testing, or in many cases you have a few specific test cases that you know you want to run. If you tell Pro-Test about these previous tests, then Pro-Test will use the combinations and may be able to reduce the total number of additional tests. Click on the tab "Previous Test" and enter these test cases.
Step #5 - Generate All Two Way Combinations
Generate the test cases and export to the desired format. To generate the desired test select "Test" - "Generate 2 Way" from the menu. Pro-Test will work to reduce the test cases to the smallest number possible. When done, you can export the results in a variety of formats. The resulting test cases will ensure total two way coverage of the test factors and levels.
At this point, you can export your test cases to an Excel, HTML, XML, Tab Delimited, or Comma Delimited file. In this example, with 6 addition tests you will be able to cover all two way combinations of test factors. In this case, all combinations would have required an addition 30 test cases.
While this example resulted in a relatively small savings in test cases, as the number of factors and levels increase, the benefits of HTT will be quite substantial. The table below expresses the savings for various scenarios between All Combinations and Pairwise Combinations.