Testing sucks by James Whittaker

Bet that got your attention. It's true, but let me qualify it: Running test cases over and over in the hope that bugs will manifest sucks. It’s boring, uncreative work and since half the world thinks that is all testing is about, it is no great wonder few people covet testing positions. Testing is either too tedious and repetitive or it’s downright too hard. Either way, who would want to stay in such a position? ... For the hard parts of the testing process like deciding what to test and determining test completeness, user scenarios and so forth we have another creative and interesting task. Testers who spend time categorizing tests and developing strategy (the interesting part) are more focused on testing and thus spend less time running tests (the boring part). ... So all the managers out there need to ask themselves what they've done lately to make their testers more creative. If you don't have an answer, then testing isn't the only thing that sucks.

One of the great benefits of Hexawise is that it takes care of figuring out the best test plan to provide the coverage is needed. The software test planer needs to use their knowledge, experience and creativity to determine what factors and parameters are critical to test. Then Hexawise generates a test plan that provides maximum coverage with the fewest possible tests. If people try to manually create test plans to address interaction between factors to be tested it is not only extremely time consuming, not very fun and it is essentially impossible to do well.

Some things are just so complex or so effectively handled with well designed software people cannot compete. Designing software test plan coverage is one of those areas.

Hexawise also lets the software tester easily tune the test coverage based on what is most important. Certain factors can be emphasized, others can be de-emphasised. Knowledge is needed to decide what factors are most important, but after that designing a test plan based on that knowledge shouldn't take up staff time, good software can take care of that time consuming and difficult task.

Another nice feature included with Hexawise is automated detailed tester instructions are generated. And you can easily provide customized text to assure the test instructions and the expected outcomes are clear and complete.

Hexawise greatly reduces the number of test that need to be run by creating powerful test plans that provide more coverage with fewer tests. This again, frees up tester time to focus on value added activities.

Allowing testers to focus on adding value is a key aim of ours. We strive to automate what we can and allow testers to apply their knowledge, experience and creativity to helping create great software. Hexawise grew out of the work of George Box, William Hunter (the founders father) and W. Edwards Deming, they sought to use statistical tools to free people to focus on creative tasks. For example read: Managing Our Way to Economic Success, Two Untapped Resources by William G. Hunter - "Two resources, largely untapped in American organizations, are potential information and employee creativity."

Hexawise includes sample test plans that let you see the benefits above in action. Sign up today to try it out (free trial).

 

Related: Cem Kaner: Testing Checklists = Good / Testing Scripts = Bad? - A Faster Way to Enter Test Inputs – the “Bulk Add” Option - Practical Ways to Respect People

By: John Hunter on Nov 11, 2012

Categories: Context-Driven Testing, Testing Strategies

I have started focusing on the Hexawise blog recently. For a good part of this year we have not had much activity on the blog, but we plan to make this blog more active going forward. As part of that effort we are starting a Software Testing Blog Carnival to highlight some post on software testing that I find interesting and think others will too. Enjoy,

 

 

water buffalo south africa

Water buffalo, Timbavati Game Preserve, South Africa by Justin Hunter

 

  • Bugs Spread Disease by Elisabeth Hendrickson - "It wasn’t the bugs that killed us directly. Rather, the bugs became a pervasive infection. They hampered us, slowing down our productivity. Eventually we were paralyzed, unable to make even tiny changes safely or quickly."

  • A Sticky Situation by Michael Bolton - on using agile, kanban style, system for managing the software testing workload: "The whiteboard was the center of an active discussion between programmers and project managers about the project status. After the meeting, the whiteboard and the notes on it remained as a kind of information radiator. 'I suddenly realized that if they could do that, I could too,' Paul said. He began by dividing the whiteboard into three columns: To Be Done, Work in Progress, and Done."

  • A Quick Testing Challenge by Alan Page and a response, Angry Weasel Challenge, a presentation by to Figure out why TheApp.exe won't load and cause it to load by solving the problem.

  • 3 Strategies to Maximize Effectiveness of Your Tests by John Hunter - "Use the MECE principle. The MECE principles states you should define your values in a way that makes each of them “Mutually Exclusive” from the others in the list (no subsets should represent any other subsets, no overlaps) and “Collectively Exhaustive” as a group (the set of all subsets, taken together, should fully encompass all items, no gaps)."

  • Musings on Test Design by Alan Page - "The point is, and this is worth repeating so often, is that thinking of test automation and “human” testing as separate activities is one of the biggest sins I see in software testing today. It’s not only ineffective, but I think this sort of thinking is a detriment to the advancement of software testing."

  • Creating a Shared Understanding of Testing Culture on a Social Coding Site by Leif Singer - "those learning testing in this environment write tests to make sure that a program behaves a certain way, and forget that they might also need to test how it should not behave (e.g. on invalid input)."

  • Testing in Scrum with a Waterfall Interaction by Davide Noaro - "Testing each user story separately is, for me, the basis of the Agile process, even in an interaction with a Waterfall-at-end scenario like the one described. Integrating testing into the process itself is something we should do for any software development process, not only in Agile or Scrum."

By: John Hunter on Oct 31, 2012

Categories: Software Testing

Hexawise includes an array of sample plans when a new user account is created. These provide concrete examples of how to categorize items when creating a combinatorial test plans (also called pairwise test plans, orthogonal array test plans, etc.). Once you [sign in to your Hexawise account](http://hexawise.com/ (or setup a new, free, account) looking at this [sample test plan](https://app.hexawise.com/share/HT3UG7M8 (which is similar to the situation raised in the question that follows), might be useful.

Within your Hexawise account you can copy the sample test plans that you are provided with and then make adjustments to them. This lets you quickly see what effects changes you make have on real test plans. And it also lets you see how easy it is to adjust as changes in priorities are made, or gaps are found in the existing test plan.

 

A Hexawise user sent us the following question.

What is the recommended approach to configuring parameter with one or more values?

I have two parameters which are related.

If Parameter 1 = Yes, Parameter 2 allows the user to select one or more values out of a list of 25 - most of which are not equivalent.

For Parameter 2, is the recommended approach to handle this to create separate parameters each with a yes/no value? i.e. create one parameter for each non-equivalent value, and one parameter for the equivalent values. Then link each of these as a married pair to Parameter 1.

I'm open to suggestions as to alternatives.

Here's the screen in question. Parameter 1 = "Pilot", Parameter 2 = checkboxes for types of plans.

aviation question inline

Great question.

I would recommend that you use different parameters for each option (e.g., "Scheduled Commercial" as a parameter with "Selected, Not Selected" as your Values associated with it).

Also, I'd recommend following these 3 strategies to maximize the effectiveness of your tests.

First, consider using adjusted weightings. You may find it useful to weight certain values multiple times, e.g., have 4 values such as "Select, Do Not Select, Do Not Select, Do Not Select" to create 3 times as many tests with "Do Not Select" as "Select."

Second, use the MECE principle. The MECE principles states you should define your Values in a way that makes each of them "Mutually Exclusive" from the others in the list (no subsets should represent any other subsets, no overlaps) and "Collectively Exhaustive" as a group (the set of all subsets, taken together, should fully encompass all items, no gaps)

Third, avoid "ands" in your value names. As a general rule it is unwise to define values like "Old and Male" or "Young and Female", etc. A better strategy is to break those ideas into two separate Parameters, like so:

First Parameter = "Age" --- Values for "Age" = Old / Young

Second Parameter = "Gender" --- Values for "Gender" = "Male / Female"

 

Related: Efficient and Effective Test Design - Context-Driven Usability Considerations, and Wireframing - Why isn't Software Testing Performed as Efficiently and Effecively as it could be?

By: John Hunter on Oct 25, 2012

Categories: Efficiency, Hexawise tips, Pairwise Software Testing, Software Testing, Software Testing Efficiency, Testing Strategies

When we met with Hexawise users in India, we noticed that the page load response times for Hexawise users there were sometimes significantly slower than in the United States due to sporadic internet connectivity issues. One area that troubled us in particular was the extra few seconds it could take to enter test inputs into plans.

We are constantly looking for ways to make our software test case generating tool better and came up with a solution.

Hexawise Bulk Add Instructions 1 inline

Hexawise Bulk Add Instructions 2 inline

 

Even with a great internet connection this feature lets you be more productive, so if you have not tried out the bulk add feature, give it a try today.

Hexawise: More Coverage. Fewer Tests

 

Related: Hexawise Tip: Using Value Expansions and Value Pairs to Handle Dependent Values - Maximize Test Coverage Efficiency And Minimize the Number of Tests Needed - 13 Great Questions To Ask Software Testing Tool Vendors

By: John Hunter on Oct 23, 2012

Categories: Hexawise test case generating tool, Hexawise tips, Software Testing

We have thousands of users and we're growing by hundreds of new users a month. We don't have a way to accurately breakdown our users into Agile vs Waterfall users but I know from conversations with Hexawise users that many users are using Hexawise with great results on Agile projects.

Agile projects involve short sprints in which new features are added to existing functionality. These sprints are often a couple weeks long. In such situations, it is important to be able to create tests quickly. Hexawise is extremely good at accomplishing exactly that.

Let me explain by way of an example. A Hexawise user named Sandeep asked how Hexawise could be used to create tests for an insurance ratings engine. I was able to create a sample set of 37 2-way tests within 20 minutes.

Now lets imagine that a sprint creates 2 new features that we'd like to test in combination with the existing test inputs in the existing plan. Let's assume "Feature 1" is a new ability of the application to handle input coming in from three sources: (1) web, (2) agent, and (3) call center. Let's assume "Feature 2" is a new ability of the application to handle Group Discounts. The Group Discount parameter will have 3 categories of values as well: (N/A) for people who aren't able to claim them, "Private company," and "Government / Public."

Added the new features, and created another, completely different set of tests that achieve 100% 2-way coverage. Each value in the new feature is tested in at least one test with every other value in the plan. It took less than 4 minutes to create this new set of tests incorporating the new features. It is an extremely attractive benefit of using Hexawise that is excellent for both Waterfall projects (which inevitably have late-breaking requirements changes) as well as Agile projects (where late-breaking feature additions are expected to happen as part of the process).

You want to keep testing documentation light, you want to minimize the amount of time you spend in selecting and documenting tests, and it is helpful to achieve as much coverage in as little time as possible and minimize the risk that you'll accidentally forget to test things that you meant to test (which can be easier to do if you have light test cases with enough detail in them to maximize your odds of testing the important stuff without accidentally omitting tests). Hexawise helps you achieve all of thise objectives. The example shows, as clearly and concisely as I'm able to explain it, how you can use Hexawise to create a small set of powerful tests within four minutes (from 1.7 trillion possible tests) that are well suited to test new features developed in an Agile project iteration.

 

Related: What is Agile? What is not Agile? - Why isn't Software Testing Performed as Efficiently and Effecively as it could be? - Cem Kaner: Testing Checklists = Good / Testing Scripts = Bad?

By: Justin Hunter on Oct 19, 2012

Categories: Agile, Hexawise tips, Pairwise Software Testing, Software Testing

I came across Elisabeth Hendrickson's "Test Heuristics Cheat Sheet" yesterday and developed some pairwise testing (AKA 2-way combinatorial) test cases using many of the good ideas contained in it. I would highly recommend it, I'd recommend you send it (or email a link to this blog post) to everyone on your QA team.

As an indication that the Hendrikson's Test Heuristics Cheat Sheet works well to uncover defects,

  • I wanted to create a set of pairwise tests that could be broadly applicable to test thousands of different applications, I incorporated many ideas from the Test Heuristics Cheat Sheet.

  • I intend to use those inputs to test our test design tool, Hexawise.

  • The way Hexawise works is that users enter "things they want to test" into Hexawise on the first of three screens, the "Define Inputs" screen, then click on "Create Tests." Hexawise then uses a scientific approach to maximizing coverage of the combinations of all the "stuff to be tested" in the fewest possible number of tests. This scientific approach is based on the >40 years of Design of Experiments lessons and includes both pairwise / AllPairs methods as well as more thorough 3-way, 4-way, 5-way and 6-way tests (as well

  • Ironically, even before starting to execute the test conditions suggested by Hexawise, I discovered that the special characters that I had input into the "Define Inputs" screen (which I took from the Test Heuristics Cheat Sheet) triggered a previously unidentified defect in Hexawise itself.

  • The fact that it was triggered so quickly in an application that has been live for a year and used thousands of times is a strong indication that using checklists and cheat sheets can be a great way to efficiently find defects.

Why is using checklists to guide your testing often such an efficient and effective way of finding defects? Here's my top ten list:

  1. The "bad ideas" have already been weeded out.
    1. The ideas on the list have found enough defects to make the author of the checklist think there is value in testing the particular idea.
    2. If you've got a checklist or "cheat sheet" put together by someone as thoughtful and experienced as the Bachs, Bolton, and Hendrickson, you're getting a highly-condensed executive summary version of many of their valuable insights.
    3. All testers go through many, many, "I wonder what would happen if we did this or considered that?" scenarios.
    4. The checklists referenced above represent expertise culled from thousands of testing projects.

  2. Checklists are directly actionable. You can apply them in almost no time at all.
  3. They work well. See Cem Kaner's slides on the Value of Checklists (11 Mb pdf file).
  4. They can easily evolve into some of your most powerful test artifacts.
    • Start with the lists above. See if each of the ideas for tests trigger defects in your Systems Under Test.
    • Find a lot of defects from certain test ideas? Create your own checklist of ideas that worked and iterate them over time.. Consider expanding upon the checklist items and concepts that do bear fruit.
    • Don't ever find defects from certain of the test ideas? Consider dropping those items from the checklists if they don't bear fruit for you (or put tests for those ideas at the back of your lists and only include tests for them if you have extra time).

  5. Checklists include useful, defect-triggering ideas that you may not have thought of on your own.
  6. They're free.
    • No software or books to buy.
    • No courses or conferences to attend.

  7. Using checklists mitigate the risk that you will forget to test for things that you know you should be testing for (but could well forget to test for in any specific instance).
    • As humans, we're naturally forgetful as a species despite our best efforts.
    • Checklists are widely used with good results by doctors, lawyers, pilots, software testers, and people going to grocery stores to minimize the effects of these shortcomings.

  8. Software testing checklists are an efficient way to communicate actionable information.
  9. Software testing checklists are widely applicable to all kinds of software testing.
    • Checklists can be used in creating Unit Tests, Assembly Tests, Product Tests, System Tests, Functional Tests, Load Tests, Performance Tests, User Acceptance Tests, etc.
    • Checklists can be used by Exploratory Testers and "script-everything-in-advance" test-case-centric testers.
    • Checklists can be used in Agile projects as well as Waterfall projects.

  10. Software testing checklists can be easily used in pairwise and combinatorial testing.
    • Using elements from the checklists in a pairwise test will have the added benefit that not only will you test for every one of the testing ideas on the checklist (e.g., XXX) but also, you can easily test for every idea on the checklist **in combination with** every other test idea on the checklist in at least one test case.

    By: Justin Hunter on May 3, 2012

    Categories: Checklists, Software Testing, Testing Checklists, Uncategorized

It's common to have a test plan where the possible values of one parameter depend on the value of another parameter. There are many options for how you can represent this scenario in Hexawise, some options that involve using value expansions (when there is equivalence) and other options that do not use value expansions (when there is not equivalence).

Using Value Expansions in Hexawise

The general rule of thumb for value expansions is that they are for setting up equivalence classes. The key there being the equivalence. The expectations of the system should be the same for every value listed in that particular value expansion.

Let's consider a real world example involving a classification parameter with a value that is dependent on the value of a role parameter:

Inputs
Role: Student, Staff
Classification: Freshman, Sophomore, Junior, Senior, Adjunct, Assistant, Professor, Administrator

So if the Role parameter has a value of Student, then the Classification parameter must have a value of Freshman, Sophomore, Junior or Senior, but if the Role parameter has a value of Staff, then the Classification parameter must have a value of Adjunct, Assistant, Professor or Administrator.

Using value expansions in this case might be a good option. You could setup your inputs, value expansions and value pairs this way:

Inputs
Role: Student, Staff
Classification: Student Classification, Staff Classification

Value Expansion
Student Classification: Freshman, Sophomore, Junior, Senior
Staff Classification: Adjunct, Assistant, Professor, Administrator

Value Pairs
When Role=Student Always Classification=Student Classification
When Role=Staff Always Classification=Staff Classification

You would use this approach if there were no important differences in the business logic or expected behavior of the system when the different expansions of the value were used. If Freshman versus Sophomore is an important label for the users to be able to enter and see, but the system under test doesn't change its behavior based on which value is selected, then those expansions of the value are equivalent and don't need to be tested individually for how they might interact with other parts of the system and create bugs. If this equivalence scenario is true, then you will greatly simplify things for yourself and create fewer tests that are just as powerful by using value expansions.

In the scenario that would support using value expansions, the system might have different behavior for a Junior versus an Adjunct Professor, but not for a Freshman versus a Senior. A Freshman and a Senior are always equivalent in the system, so they can be combined in a value expansion.

However, if the expectations are not the same, then a value expansion should not be used. For example, let's suppose this hypothetical system has business logic giving priority class scheduling to Seniors and only last available scheduling priority to Administrators. In this case, using value expansions as described above would probably be a mistake. Why? Because a Sophomore and a Senior aren't treated the same way by the system, yet Hexawise considers all the expansions of the Student Classification value as equivalent. As long as you've got a test that has paired a value expansion of the Student Classification value with the Overbooked value of the Class Status parameter, then Hexawise won't insist on pairing all the other value expansions for the Student Classification value with Class Status = Overbooked in other tests. You could therefore miss a bug that only occurs when a Senior signs up for an overbooked class.

"One to many" or "multi-valued" married pair model

If the system under test does not consider the values to be equivalent and has requirements and business logic to behave differently, then by using value expansions to signal equivalency to Hexawise when there isn't equivalency is probably a mistake.

So what would you do in that case?

We've decided that it might be nice to be able to set up your inputs and value pairs like this:

Inputs
Role: Student, Staff
Classification: Freshman, Sophomore, Junior, Senior, Adjunct, Assistant, Professor, Administrator

Value Pairs
When Role=Student Always Classification=Freshman, Sophomore, Junior, or Senior When Role=Staff Always Classification=Adjunct, Assistant, Professor, or Administrator

Unfortunately, this kind of a "one to many" or "multi-valued" value pair is something we've only recently realized would be very helpful, and is something we have on the drawing board for Hexawise in the intermediate future, but is not a feature of Hexawise today. In the meantime, you could model it with three parameters:

Inputs
Role: Student, Staff
Student Classification: Freshman, Sophomore, Junior, Senior, N/A Staff Classification: Adjunct, Assistant, Professor, Administrator, N/A

Value Pairs
When Role=Student Always Staff Classification=N/A
When Role=Staff Always Student Classification=N/A

Another modeling option to consider, if there is only special logic for Administrator and for Seniors, but the rest of the values we've been discussing are equivalent, is to use value expansions for just the equivalent values:

Inputs
Role: Student, Staff
Classification: Underclassman, Senior, Professor, Administrator

Value Expansions
Underclassman: Freshman, Sophomore, Junior Professor: Adjunct, Assistant, Full

Value Pairs
When Role=Student Never Classification=Professor When Role=Student Never Classification=Administrator When Role=Staff Never Classification=Underclassman When Role=Staff Never Classification=Senior

I hope this helps you understand the role of value expansions in Hexawise, when to use them (in cases of equivalency), and when to avoid them, and how value pairs and value expansions can be used together to handle cases of dependent parameter values. Value Expansions are a powerful tool to help you decrease the number of tests you need to execute, so take advantage of them, and if you have any questions, just let us know!

By: John Hunter on Apr 26, 2012

Categories: Combinatorial Software Testing, Combinatorial Testing, Hexawise tips, Pairwise Testing, Software Testing, Testing Strategies

Request for ideas: What are good examples of good interactive online training (including exercises & quizzes) for how to use web apps?

We continue to look for ways to improve the ability of our users to add value to their organizations. We are always in the process of improving the software tools we provide.

The value proposition for using pairwise and combinatorial tools are huge. The biggest roadblocks we see in adoption are not being aware of the advantages and not being sure how to proceed once the advantages are seen.

We designed Hexawise to be very easy to use. But even so the concepts behind combinatorial testing take some people a bit of time to operationalize. To help with this we offer on site and off site personal training.

We are looking to create some online training to ease the transition into using Hexawise's combinatorial software testing tools most effectively. We would love to know what good examples of interactive online training people have found useful.

 

Related: Why isn't Software Testing Performed as Efficiently and Effecively as it could be? - In Praise of Data-Driven Management (AKA "Why You Should be Skeptical of HiPPO's")

By: John Hunter on Apr 24, 2012

Categories: Uncategorized

84 percent coverage in 20 tests

Hexawise test coverage graph showing 83.9% coverage in just 20 tests

 

Among the many benefits Hexawise provides is creating a test plan that maximizes test coverage with each new scenario tested. The graph above shows that after just 20 test 83.9% of the test combinations have been tested. Read more about this in our case study of a mortgage application software test plan. Just 48 test combinations are needed to test for every valid pair (3.7 million possible tests combinations exist in this case). If you are lost now, this video may help.

The coverage achieved by the first few tests in the plan will be quite high (and the graph line will point up sharply) then the slope will decrease in the middle of the plan (because each new test will tend to test fewer net new pairs of values for the first time) and then at the end of the plan the line will flatten out quite a lot (because by the end, relatively few pairs of values will be tested together for the first time).

One of the benefits Hexawise provides is making that slope as steep as possible. The steeper the slope the more efficient your test plan is. If you repeat the same tests of pairs and triples and... while not taking advantage of the chance to test, untested pairs and triples you will have to create and run far more test than if you intelligently create a test plan. With many interactions to test it is far too complex to manually derive an intelligent test plan. A combinatorial testing tool, like Hexawise, that maximizes test plan efficiency is needed.

For any set of test inputs, there is a finite number of pairs of values that could be tested together (that can be quite a large number). The coverage chart answers, after each tests, what percentage of the total number of pairs (or triples, etc.) that could be tested together have been tested together so far?

The Hexawise algorithms achieve the following objectives that help testers find as many defects as possible in as few tests as possible. In each and every step of each and every test case, the algorithm chooses a test condition that will maximize the number of pairs that can be covered for the first time in the test case. (Or, the maximum number of triplets or quadruplets, etc. based on the thoroughness setting defined by the user). Allpairs (AKA pairwise) is a well known and easy to understand test design strategy. Hexawise lets users create pairwise sets of tests that will test not only every pair but it also allows test designers to generate far more thorough sets of tests (3-way to 6-way coverage). This allows users to "turn up the coverage dial" and generate tests that cover every single possible triplet of test inputs together at least once (or every 4-way combination or 5-way combination or 6-way combination).

Note that the coverage ratio Hexawise shows is based on the factors entered as items to be tested: not a code coverage percentage. Hexawise sorts the test plan to front load the coverage of the tuple pairs, not the coverage of the code paths. Coverage of code paths ultimately depends on how good a job the test designer did at extracting the relevant parameters and values of the system under test. You would expect there to be some loose correlation between coverage of identified tuple pairs and coverage of code paths in most typical systems.

If you want to learn more about these concepts, I would recommend Scott's Scott Sehlhorst articles on pairwise and combinatorial test design. They are some of the clearest introductory articles about pairwise and combinatorial testing that I have seen. They also contain some interesting data points related to the correlation between 2-way / allpairs / pairwise / n-way coverage (in Hexawise) and the white box metrics of branch coverage, block coverage and code coverage (not measurable by Hexawise).

In Software testing series: Pairwise testing, for example, Scott includes these data points:

 

  • We measured the coverage of combinatorial design test sets for 10 Unix commands: basename, cb, comm, crypt, sleep, sort, touch, tty, uniq, and wc... The pairwise tests gave over 90 percent block coverage.

 

  • Our initial trial of this was on a subset Nortel’s internal e-mail system where we able cover 97% of branches with less than 100 valid and invalid testcases, as opposed to 27 trillion exhaustive testcases.

 

  • A set of 29 pair-wise... tests gave 90% block coverage for the UNIX sort command. We also compared pair-wise testing with random input testing and found that pair-wise testing gave better coverage.

 

Related: Why isn't Software Testing Performed as Efficiently and Effecively as it could be? - Video Highlight Reel of Hexawise – a pairwise testing tool and combinatorial testing tool - Combinatorial Testing, The Quadrant of Massive Efficiency Gains

Specific guidance on how to view the percentage of coverage graph for the test plan in Hexawise:

 

When working on your test plan in Hexawise, to get the checklist to be visible, click on the two downward arrow keys located shown in the image:

How-To Progress Checklists-2 inline

Then you'll want to open up the "Advanced" list. So you might need to click here:

Advanced How-To Progress Checklist inline

Then the detailed explanation will begin when you click on "Analyze Tests"

Decreasing Marginal Returns inline

 

This post is adapted (and some new content added) from comments posted by Justin Hunter and Sean Johnson.

By: John Hunter on Feb 3, 2012

Categories: Combinatorial Software Testing, Combinatorial Testing, Efficiency, Multi-variate Testing, Pairwise Software Testing, Pairwise Testing, Scripted Software Testing, Software Testing, Software Testing Efficiency

jobgraph

 

This chart shows that the number of job listings posted for testers has remained essentially flat for testers over the last six years. During this same period, the number of job listings for developers has increased almost 50%. If your company's hiring has mirrored this chart's trends, and your software testers are increasingly feeling that there's not enough time in the day to test everything that's being thrown at them, your testers might have a point.

By: Justin Hunter on Feb 8, 2011

Categories: Software Testing