By Charles Pitts · Feb 28, 2022

 

It’s a safe bet that everyone reading this has heard of Wordle. Some play it religiously, while others might be tired of hearing their friends complain about how hard today's word was. Regardless of your level of devotion to the game, the lessons it can teach us should not be overlooked. Beyond the values of patience, persistence, and anger management which it endows in all of us, the game’s hard mode provides an excellent illustration of some of the fundamental principles of combinatorial test design. The game describes its hard mode like so: 

 

 

What makes hard mode hard? In order to answer this question, let’s look at a recent word which confounded many, causing it to immediately start trending on Twitter: SWILL.

 

 

Let’s go through an example of someone failing this puzzle, guess by guess. To begin their attempt, the person guesses RAISE, which provides them with one green letter and one yellow letter. 

 

 

As they know the word has an S in it, they opt to guess a word which has two S’s in two other locations than the S in RAISE, in order to more quickly narrow down where the S is located. As they are in hard mode, this word must also contain an I in the 3rd spot. They input SOILS as their next guess, which brings them up to 3 green letters:

 

 

Their next guess, SKILL, brings them up to 4 green letters, leaving them with only a single blank space.

 

 

Here is where hard mode gets its name. They must use the 4 green letters in each of their remaining 3 guesses, allowing them to test only a single letter at a time. However, there are 4 possible letters which must be tested: T to make STILL, H to make SHILL, P to make SPILL, and W to make SWILL. It is now up to chance whether or not the person is able to correctly guess the word. If they are unlucky, they will run out of guesses with a board that looks like this: 

 

 

Let’s now consider how the person’s strategy may have differed if they were not playing on hard mode. Assuming that they play the same through the 3rd guess, they are then left in the same situation: with 3 guesses left and a single unknown letter remaining, testing the letters W, H, T, and P would reveal important information: if any one of those letters were present in the word, the solution would be immediately apparent.   

 

For anyone who cares about keeping their Wordle streak alive, entering PHOTO as their 4th guess is a brilliant move at this point even though it has a zero percent chance of being a correct guess because using the P, H, and T reveals as much useful information as possible with the guess. 

 

 

Entering PHOTO provides them with no yellow letters. Is this a failed or “bad” guess then? Not at all. It is highly valuable to know that P, H, and T are not contained in the answer, which leads directly to the conclusion that W must be the final letter, as no other unused letter could make valid words when combined with S_ILL. Their 5th entry, SWILL, solves the puzzle.

 

 

Studying these two games, we can see that hard mode is difficult because as the game progresses, the number of letters that the player can test per guess decreases. Because the player is limited to 6 guesses, it is in their best interest to try and gain as much new information as possible with each new guess, and reusing letters, which hard mode forces the player to do, limits their ability to reveal new information efficiently.

Implications for Software Testers?

1) Focus on maximizing net new information with each test (which means you need to be clear about what is already “known” before each new test)
Now, let’s think about each Wordle guess as if they were a test case, with each letter of the alphabet being a test condition (or parameter value) of the system under test. STILL as guess number 4 would be a horrible guess if hard mode had been disabled. The least possible amount of net new information was produced with that guess. It would be impossible to design a test or guess that would reveal less information. Additionally, if a test case retraces an identical combination of system inputs that have already been reused in test cases that came before it (instead of exploring new combinations of system inputs), that test will be like the “STILL” guess; it will produce much less new information about the system under test than it could.

2) To maximize the amount of new learning with a Wordle guess or software test, you need to maximize the amount of learning achieved by every single letter in a word (or every single test condition in a test).

Now consider just how efficient it was to guess “PHOTO.” The letters P, H, and T had not appeared in any of the guesses so far: the word could therefore be called highly variable in relation to the rest of the guesses. As a result of this variability, we gain new information about 3 different letters, allowing us to solve the puzzle with certainty in the next guess. Within software testing, a highly variable test case, which maximizes the difference between it and the rest of the testing set and includes wholly new pathways through the system under test, will have the best chance of producing a large amount of new and useful information. Tests that produce large amounts of new information are inherently more efficient than those that do not, and are therefore a much better use of testing time and resources.

3) “Don’t test on hard mode”.

Our conclusion? Don’t test on hard mode. Adding a new test with only one or two new test conditions changed in your new test may be an easy and simple way to design test cases, but the lack of new information revealed means that this method is highly inefficient. Upgrade your testing (and improve your Wordle scores) by maximizing the variability of each new test case with Hexawise’s automatic test design tool.

Generating your tests with Hexawise is all about maximizing the amount of new information that you learn, not only with each test, but with each test condition you use in every additional test you execute.