Orthogonal Array and Combinatorial Software Testing

Orthogonal Array Testing (OATS) - Orthogonal Array testing and pairwise testing are very similar in many important respects. They are both well established methods of generating small sets of unusually powerful tests that will find a disproportionately high number of defects in relatively few tests. The main difference between the two approaches is that pairwise testing coverage only requires that every pair of parameter values appear together in at least one test in a generated set. Orthogonal Array-based test designs, in contrast, have an added requirement that there be a uniform distribution throughout the domain. This added requirement tends to result in significantly more tests for Orthogonal Array Testing solutions than are required for pairwise test sets.

Are the extra tests required by Orthogonal Array-based solutions (compared to pair-wise solutions) worth it? Probably not in software testing as our CEO, Justin Hunter, explains in software testing, you are not seeking some ideal point in a continuum; you're looking to see what two specific pieces of data will trigger a defect.

It is based on the principles behind design of experiments. Design of experiments and orthogonal array testing are great when you are looking at how important individual factors and interactions of factors are for achieving the highest productivity. But this isn't the problem software testing normally meant to solve. These approaches would be very good for example in evaluating different coding and architecture strategies to increase the speed of the software application for users.

Related terms: combinatorial software testing