Last week, we posted a poll on our LinkedIn page asking our followers which Hexawise feature they find to be the most impactful to their team’s success. The overwhelming response was that our users most appreciate Hexawise’s optimized test design, which received more than double the votes of the 2nd place option, test coverage visualization. 

The end goal of test design optimization is to provide more coverage in fewer tests. Hexawise achieves this by generating testing sets with minimal levels of repetition. 

In order to see how this works, let's first try to understand why repetitive testing sets are inefficient. Consider a generic system under test. When a user is interacting with a system, they are presented with a series of choices that they can respond to with a finite number of inputs. The system can therefore be modeled as a series of steps, with a number of discrete choices at each of these steps. Let’s model such a system diagramatically:

A test case models an individual user’s interaction with the system. At each of the above steps, this model user's choice is selected, creating a pathway through the system. Two possible test cases for this system are illustrated below.

The red test case would prompt a tester to input Choice 3 at the first step of the system, Choice 2 at the second step, and so on. In other words, both test cases instruct a tester on how to interact with the system at each step in order to model a user’s behavior. 

Let’s return to our initial point: why are repetitive test cases inefficient? First, we will model three repetitive test cases (illustrated in red, blue, and green below) in this diagrammatic system:

Note that out of the total 20 possible system choices, the three test cases cover only 7 of them. It is easy to see why: after the first test case is executed, the next two test cases each contain 4 choices that were already tested, with only a single new choice each.

Now, let’s look at the coverage that can be achieved with a more diverse set of three test cases: 

These three test cases are completely non-repetitive, as their paths do not overlap at any point. We can count that out of the 20 possible system choices, 15 of them are covered, as opposed to the 7 covered by the repetitive test cases. As one of the primary goals of testing should be to include as many system paths and choices as possible, the diverse testing set is clearly preferable. 

These two examples clearly illustrate our point that repetitive test cases are inefficient. If your test cases are repetitive, then your testers will spend time retesting elements of the system that have already been covered, when a less repetitive and more diverse testing set would have them using that time to test wholly new paths through the system. This allows for a greater level of coverage to be achieved in the time allotted for testing. 

Today’s discussion was meant to be a simple, illustrative look at the motivation behind reducing testing set repetition, one of the primary principles of Hexawise’s test design algorithms. The reality, however, is significantly more complicated. For instance, our discussion only tracked how many individual system choices were covered by the repetitive and non-repetitive testing sets; Hexawise, meanwhile, tracks how many pairings of system choices are included, which results in a more thorough level of coverage. This discussion of pairwise testing, however, is beyond the scope of this article.

Regardless of the exact theory and implementation, however, Hexawise’s principle of reducing repetition produces incredible results. Countless companies across a diverse array of industries have leveraged Hexawise to achieve better coverage in fewer tests, slashing testing costs, reducing testing time, and bolstering their bottom line.