This interview with Matt Heusser is part of the Hexawise “Testing Smarter with…” software testing interview series. Our goal with these interviews is to highlight insights and experiences as told by many of the software testing field’s leading thinkers.

Matt Heusser
Matt Heusser

Matt Heusser is a software craftsman with a deep background in software delivery and testing.

In 2014, Matt received the Most Influential Agile-Test Professional Award at Agile Testing Days in Potsdam, Germany.

Personal Background

Hexawise: If you could write a letter and send it back in time to yourself when you were first getting into software testing, what advice would you include in it?

Matt: My advice would be to trust your instincts and experiences more than what you read in books or online. Often the advice I read in books and online seemed vapid (shallow), or simplistic, or I felt it "just wouldn't work here." Eventually I realized that a lot of it (this was the testing advice of the 1990's) wasn't working well most places.

Today we have better advice. The time from research to publish is short, and there is a lot less "loss" in the system. Still, what works for Google and Microsoft might not work for your 20 person company, and what works for that cool, 100% physically distributed, 40 person software company might not work for your 2,000 employee insurance company. Take it with a grain of salt, trust your instincts - but always keep exploring and experimenting.

Hexawise: Which person or people have had the greatest influence on your understanding and practice of software testing?

Matt: It's hard to come up with a list of influences, but Cem Kaner, James Bach, Ken Pier, Brian Marick, Johanna Rothman, Lee Copeland, Jerry Weinberg, Kent Beck, and Ron Jeffries all come to mind.

Hexawise: What one or two software testing-related experiences have you found to be most personally satisfying in your career?

Matt: I remember once we wanted to get a insurance data extract out on a friday, but it had to be right. Serious students of testing will tell me you can never know that it is right - but non-serious student customers don't know that. I had about a half a day. The change was to one field, and we had the results in a database table. The table-to-file and the file transfer we had confidence in; the change, not so much. So I wrote my own computer program to loop through every current member in the insurance plan, over four hundred thousand, calculate the expected result, and compare them.

We found a small bug in the requirements; an edge case that was ambiguous. The programmer and I had interpreted the requirements in different ways that were both arguably correct. The "differ" program I wrote found the case, the customer explained that either was acceptance - and we shipped!

Views on Software Testing

Hexawise: What do you wish more developers, business analysts, and project managers understood about software testing?

Matt: A decade ago, when I went to the Google Test Automation Conference, one of the speakers said that automation was better because it was "repeatable." I almost stood up and asked aloud "so what?"

Every build of the software different. If the software is different, if the risk picture has changed, if we have some idea of what tests we ran before and how valid they might be on this build - why would we ever test the exact same way?

Software isn't an assembly line. Every build is different. The way we test it can be different, but I certainly don't see a ton of value in spending extra money to make sure we test things the exact same every time.

Turns out you don't need to remove the friction from handoffs. Often, you're better served to make communication cheaper and getting good at collaboration.

Hexawise: Can you describe a view or opinion about software testing that you have changed your mind about in the last few years? What caused you to change your mind?

Matt: It's going back a bit, but in graduate school, I wrote a paper "On the optimization of physically distributed requirements, development, test, operations, and management development groups" The body of the paper was considerably shorter than the title - the body was "You're Screwed."

At the time, the literature suggested that communication costs were high, so we needed get everything right prior to "the handoff." We needed to get every document, every bit of code, everything complete, consistent, correct, unambiguous, before "the handoff."

I knew that trick never worked, and thought the fix was co-location.

Then I learned about agile/XP, which was still focused on co-location - still, part of XP was making communication, collaboration, and change cheaper. Then I worked at Socialtext, where my co-workers were all over the world - Ingy took a skiing vacation in France, skiing during the day and working core hours at night. And it was far more productive than any other job I had ever had.

Turns out you don't need to remove the friction from handoffs. Often, you're better served to make communication cheaper and getting good at collaboration.

Hexawise: You have written about the benefits of lean thinking in software testing. What advantages do organizations gain when they adopt a lean thinking view of software testing?

Matt: You know that thing that happens, where you can't do your job because you filed a ticket and it will take the DBA's a week to add a column to a table, so you can't do your job, for a week?

Or whatever else it is? Right now I've got a contractor billing on my team with no laptop. He'll have it nine days after he started ... if we're lucky.

Typically, when a company goes to lean thinking, that kind of stuff stops happening.

Industry Observations / Industry Trends

Hexawise: Do you have specific suggestions for testers working within an organization using agile or lean software development methods?

Matt: It's hard to come up with examples without context, but generally, I'd start by looking at the delays we have in our job and the amount of multitasking. Be sure to include failure demand - things that should be reasonably expected to work the first time, but took a round of fixes. Often you'll find what should take an hour is taking you a week.

Hexawise: What do you see as the most powerful trends in the software testing field over the last 5 to 10 years? What trends do you believe will be the most powerful over the next 5 to 10 years?

Matt: The past ten years? Test-Driven Development, Continuous Integration, REST APIs that can be tested at the integration level, Virtualization, Lightweight Virtualization.

The next five? The promise of continuous delivery might just be realized. I realize that sounds like a lot of technical stuff, and I'm a process and people guy. The challenges of the next few years will be all skill, people and process.

Hexawise: There is still a widespread belief in fairly mechanistic software testing (checking) by some of those using software testing. Lean thinking, exploratory testing, etc. encourage engaging the minds of software testers. Are you optimistic about the prospects for tapping more of the potential software testers have going forward?

Matt: Oh yes. That's what I hope the next five or ten years are all about.

You know that thing that happens, where you can't do your job because you filed a ticket and it will take the DBA's a week to add a column to a table, so you can't do your job, for a week? ... Typically, when a company goes to lean thinking, that kind of stuff stops happening.

Hexawise: Have you seen a particularly effective process where the software testing team was integrated into the feedback from a deployed software application (getting feedback from users on problems, exploring issues the software noted as possible bugs...)? What was so effective about that instance?

Matt: My preference is for cross-functional delivery teams, so I get a little down on the term "test team", but yes, I have seen delivery teams where customer feedback was part of the process. One team that Justin Rhorman and I worked with managed a fortune 500 retail website; they had a process that periodically popped-up requests for feedback. The testers worked through these in a rotation, summarizing them, reporting them to management and working with management.

I think that worked well because of the rotation - no single bottleneck. If multiple testers observed the same feedback independently week over week, it was harder to dismiss.

Staying Current / Learning

Hexawise: You have spoken at many conferences. What advice do you have for people attending software conferences so that they can get more out of the experience?

Matt: Try to come up with three things to do on Monday that justify the investment. These things should be entirely within your power to do. They should not require training, a team-level change in process, particular learning time, purchase of tools or hiring of consultants. Things you can just do - that you will not ask permission for. (Your boss probably doesn't know what you do anyway.)

Then do it and tell the team about it. If you want to write a trip report, it should be a one-pager, and describe what you are doing, not what you heard.

Hexawise: What software testing-related books would you recommend should be on a tester’s bookshelf? What blogs would you recommend should be included in a software tester's RSS feed reader?

Matt: I don't read blogs like I used to, instead I follow twitter and click on interesting links. A few weeks ago I wrote an article on 28 testers to follow on twitter that might be a good place to start.

As for books, I'd say How To Reduce the Cost of Software Testing, Lee Copeland's A Practitioner's Guide to Software Test Design, Lessons Learned in Software Testing and Jez Humble's Continuous Delivery book (the writing is a bit tough but it's worth it). I'd also suggest some non-testing books, like Out Of The Crisis by Deming, Peter Drucker on Management, and Taleb's Black Swan book.

Hexawise: We share an interest in seeing lean thinking concepts be adopted by software testers. How would you suggest someone interested in learning more about lean thinking in software testing do so?

Matt: You could try a google search for "heusser lean" and see what it returns, seriously I've written a lot. Here are two older articles that I think cover the start of it Applying lean concepts to software testing and The secrets of successful Lean software testing.

Profile

Matt Heusser is a software craftsman with a deep background in software delivery and testing. After earning his undergraduate degree in Math with a concentration in computer science in 1997, Matt began his career as a programmer, writing code in C, perl, PL/SQL, and Visual C++. Along the way, Matt was the initial organizer of Grand Rapids Perl User's Group ("Perlmongers"), earned a master's degree in CIS from Grand Valley State University, taught IS part time at night at Calvin College, and served as the initial lead organizer of the Great Lakes Software Excellence Conference.

After leaving Priority Health, a Health Insurance Company, in 2008, Matt went on to become a member of the technical staff at Socialtext, the world's first wiki company, where he worked with Audrey Tang, Ingy DotNet, and Dan Bricklin, the creator of the Spreadsheet, to help build a web-based spreadsheet/wiki that predated google docs. The test framework Matt worked on at Socialtext, WikiQTests, is documented as a case study (chapter 16) of O'Reilly's book "Beautiful Testing."

Matt left Socialtext in 2011 to become a full-time consultant. Since that time he reviewed Robert C. Martin's books "Clean Code", "The Clean Coder" (for which he wrote the preface) and "Clean Architecture." In 2014, Matt received the Most Influential Agile-Test Professional Award at Agile Testing Days in Potsdam, Germany.

Links

Blog: Creative Chaos

Twitter: @mheusser

Previous Testing Smarter with... interviews: Testing Smarter with Rikard Edgren - Testing Smarter with James Bach - Testing Smarter with Michael Bolton

Subscribe to the RSS feed for our blog.

By: John Hunter on May 23, 2017

Categories: Testing Smarter with..., Software Testing, Lean, Agile

I read a great post by Aleksis Tulonen Brainstorming Test Ideas With Developers and wanted to share it with you.

The ideas in the post provide a very efficient way to:

  1. increase the robustness and thoroughness of testing,
  2. prioritize among different testing ideas (both from a relative importance standpoint and from a relative timing perspective)
  3. surface new testing ideas that would not otherwise be considered
  4. provide some useful risk mitigation protection against the reality that "all models are wrong; some models are useful" problem

I'd suggest also scheduling a debrief with the same testers and developers a few weeks or months after the code was deployed.

At that time take a look at photos of the pre-testing Post It notes and a list of defects that slipped past testing and ask

  1. "what extra tests would we need to have added to have found these defects?"
  2. "what specific inputs did we forget to include?",
  3. would techniques such as pairwise testing have been beneficial?
  4. what areas did we spent a lot of effort on that did not result in learning much?
  5. what lessons learned should we incorporate for next time?

The idea of delibrately examining your software development and testing practices will be familar to those using agile retrospectives. The power of continually improving the development practices used withing the organization is hard to appreciate but it is immense. The gains compound over time so the initial benefits are only a glimpse of what can be achieve by continuing to iterate and improve.

Related: Pairwise and Combinatorial Software Testing in Agile Projects - Applying Toyota Kata to Agile Retrospectives - Create a Risk-based Testing Plan With Extra Coverage on Higher Priority Areas

By: John Hunter and Justin Hunter on Dec 15, 2016

Categories: Agile, Software Testing, Testing Strategies

At Hexawise we aim to improve the way software is tested. Achieving that aim requires not only providing our clients with a wonderful software tool (which our customers say we’re succeeding at) but also a commitment from the users of our tool to adopt new ways of thinking about software testing.

We have written previously about our focus on the importance of the values Bill Hunter (our founder's father) to Hexawise. That has led us to constantly focus on how maximize the benefits our customers gain using Hexawise. This focus has led us to realize that our customers that take advantage of the high-touch training services and ongoing expert test design support on demand that we offer often realize unusually large benefits and roll out usage of Hexawise more quickly and broadly than our customers who acquire licenses to Hexawise and try to “get the tool and make it available to the team.”

We are now looking for someone to take on the challenge of helping our clients succeed. The principles behind our decision to put so much focus on helping our customers succeed are obvious to those that understand the thinking of Bill Hunter, W. Edwards Deming, Russel Ackoff etc. but they may seem a bit odd to others. The focus of this senior-level position really is to help our customers improve their software testing results. It isn't just a happy sounding title that has no bearing on what the job actually entails.

The person holding this position will report to the CEO and work with other executives at Hexawise who all share a commitment to delighting our customers and improving the practice of software testing.

Hexawise is an innovative SaaS firm focused on helping large companies use smarter approaches to test their enterprise software systems. Teams using Hexawise get to market faster with higher quality products. We are the world’s leading firm in our niche market and have a growing client base of highly satisfied customers. Since we launched in 2009, we have grown both revenues and profits every year. Hexawise is changing the way that large companies test software. More than 100 Fortune 500 companies and hundreds of other smaller firms use our industry leading software.

Join our journey to transform how companies test their software systems.

Hexawise office

Description: VP of Customer Success

In the Weeks Prior to a Sale Closing

  • Partner with sales representatives to conduct virtual technical presentations and demonstrations of our Hexawise test design solution.

  • Clearly explain the benefits and limitations of combinatorial test design to potential customers using language and concepts relevant to their context by drawing upon your own “been there, done that” experiences of having successfully introduced combinatorial test design methods in multiple similar situations.

  • Identify and assess business and technical requirements, and position Hexawise solutions accordingly.

Immediately Upon a New Sale Closing

  • Assess a new client’s existing testing-related processes, tools, and methods (as well as their organizational structure) in order to provide the client with customized, actionable recommendations about how they can best incorporate Hexawise.

  • Collaborate with client stakeholders to proactively identify potential barriers to successful adoption and put plans in place to mitigate / overcome such barriers.

  • Provide remote, instructor-led training sessions via webinars.

  • Provide multi-day onsite instructor-led training sessions that: cover basic software test design concepts (such as Equivalence Class Partitioning, the definition of Pairwise-Testing coverage, etc.) as well as how to use the specific features of Hexawise.

  • Include industry-specific and customer-specific customized training modules and hands-on test design exercises to help make the sessions relevant to the testers and BA’s who attend the training sessions.

  • Collaborate with new users and help them iterate, improve, and finalize their first few sets of Hexawise-generated software tests.

  • Set rollout and adoption success criteria with clients and put plans in place to help them achieve their goals.

Months After a New Sale Closing

  • Continue to engage with customers onsite and virtually to understand their needs, answer their test design questions, and help them achieve large benefits from test optimization.

  • Monitor usage statistics of Hexawise clients and proactively reach out to clients, as appropriate, to provide proactive assistance at the first sign that they might be facing any potential adoption/rollout challenges.

  • Collaborate with stakeholders and end users at our clients to identify opportunities to improve the features and capabilities of Hexawise and then collaborate with our development team to share that feedback and implement improvements.

Required Skills and Experience

We are looking for a highly-experienced combinatorial test design expert with outstanding analytical and communication skills to provide these high touch on-boarding services and partner with our sales team with prospective clients.

Education and Experience

  • Bachelor’s or technical university degree.

  • Deep experience successfully introducing combinatorial test design methods on multiple different kinds of projects to several different groups of testers.

  • Set rollout and adoption success criteria with multiple teams and put plans in place to achieve them.

  • Minimum 5 years in software testing, preferably at a IT consulting firm or large financial services firm.

Knowledge and Skills

  • Ability to present and demonstrate capabilities of the Hexawise tool, and the additional services we provides to our clients beyond our tool.
  • Exhibit excellent communication and presentation skills, including questioning techniques.
  • Demonstrate passion regarding consulting with customers.
  • Understand how IT and enterprise software is used to address the business and technical needs of customers.
  • Demonstrate hands-on level skills with relevant and/or related software technology domains.
  • Communicate the value of products and solutions in terms of financial return and impact on customer business goals.
  • Possess a solid level of industry acumen; keeping current with software testing trends and able to converse with customers at a detailed level on pertinent issues and challenges.
  • Represents Hexawise knowledgeably, based on a solid understanding of Hexawise’s business direction, portfolio and capabilities
  • Understand the competitive landscape for Hexawise and position Hexawise effectively.
  • A cover letter that describes who you are, what you've done, and why you want to join Hexawise.
  • Ability to work and learn independently and as part of a team
  • Desire to work in a fast-paced, challenging start-up environment

Why join Hexawise?

salary + bonus; medical and dental, 401(k) plans; free parking and very slick Chapel Hill office! Opportunity to experience work with a fast-growing, innovative technology company that is changing the way software is tested.

Key Benefits:

Salary: Negotiable, but minimum of $100,000 + Commissions based upon client license renewals Benefits: Health, dental included, 401k plan Travel: Average of no more than 2-3 days onsite per week Location: Chapel Hill, NC*

*Working from our offices would be highly preferable. We might consider remote working arrangements for an exceptional candidate based in the US.

Apply for the VP of Customer Success position at Hexawise.

By: John Hunter on May 12, 2016

Categories: Hexawise, Career, Software Testing, Lean, Customer Success, Agile

We have thousands of users and we're growing by hundreds of new users a month. We don't have a way to accurately breakdown our users into Agile vs Waterfall users but I know from conversations with Hexawise users that many users are using Hexawise with great results on Agile projects.

Agile projects involve short sprints in which new features are added to existing functionality. These sprints are often a couple weeks long. In such situations, it is important to be able to create tests quickly. Hexawise is extremely good at accomplishing exactly that.

Let me explain by way of an example. A Hexawise user named Sandeep asked how Hexawise could be used to create tests for an insurance ratings engine. I was able to create a sample set of 37 2-way tests within 20 minutes.

Now lets imagine that a sprint creates 2 new features that we'd like to test in combination with the existing test inputs in the existing plan. Let's assume "Feature 1" is a new ability of the application to handle input coming in from three sources: (1) web, (2) agent, and (3) call center. Let's assume "Feature 2" is a new ability of the application to handle Group Discounts. The Group Discount parameter will have 3 categories of values as well: (N/A) for people who aren't able to claim them, "Private company," and "Government / Public."

Added the new features, and created another, completely different set of tests that achieve 100% 2-way coverage. Each value in the new feature is tested in at least one test with every other value in the plan. It took less than 4 minutes to create this new set of tests incorporating the new features. It is an extremely attractive benefit of using Hexawise that is excellent for both Waterfall projects (which inevitably have late-breaking requirements changes) as well as Agile projects (where late-breaking feature additions are expected to happen as part of the process).

You want to keep testing documentation light, you want to minimize the amount of time you spend in selecting and documenting tests, and it is helpful to achieve as much coverage in as little time as possible and minimize the risk that you'll accidentally forget to test things that you meant to test (which can be easier to do if you have light test cases with enough detail in them to maximize your odds of testing the important stuff without accidentally omitting tests). Hexawise helps you achieve all of thise objectives. The example shows, as clearly and concisely as I'm able to explain it, how you can use Hexawise to create a small set of powerful tests within four minutes (from 1.7 trillion possible tests) that are well suited to test new features developed in an Agile project iteration.

 

Related: What is Agile? What is not Agile? - Why isn't Software Testing Performed as Efficiently and Effecively as it could be? - Cem Kaner: Testing Checklists = Good / Testing Scripts = Bad?

By: Justin Hunter on Oct 19, 2012

Categories: Agile, Hexawise tips, Pairwise Software Testing, Software Testing

An unusually hectic work-schedule has been keeping me hopping lately. I returned this weekend from a great two-week trip to the UK in which I visited with 5 testing teams using our Hexawise tool to design test cases for applications being used in two banks, a consulting and systems integration firm, a grocery store chain, and a telecoms company.

Every product manager worth his or her salt will tell you it is a good idea to go meet with customers, listen to them, and watch them as they use your application. Even though everyone I know agrees with this, I find it difficult to make happen as regularly as I would like to. This trip provided me with a reminder of how valuable in-depth customer interactions can be. The two weeks of on-site visits with testing teams proved to be great way to: (a) reconnect with customers, (b) get actionable input about what users like / don't like about our tool, (c) identify new ways we can continue to refine our tool, and even (d) understand a couple unexpected ways teams are using it.

Bret Petticord's tweets on "What is Agile?" / "not Agile?" prompted me to write this quick post. I like them a lot.

 

20100225-dumbhsb9bpbaf5pu921kf9xim9

 

When we first created our Hexawise tool, we followed the 4 steps Bret lays out in his description of "What is Agile?" My experience in the UK over the last two weeks was the start of one of many "Repeat" cycles.

I admire people who can succinctly summarize wisdom into bite-sized quips like Bret achieved with his two tweets. Another guy who excels at creating sound-bites is James Carville. Love him or hate him, he has that skill in spades. When I watched the movie "War Room," I felt like I was watching the "master of the sound-bite" in his element. Me? I'm more of a rambling, meandering, verbose communicator. I've just taken 332 words and a screen shot with Bret's tweets when all I set out to do in starting to write this post was to share Bret's 32 words with you.

By: Justin Hunter on Feb 25, 2010

Categories: Agile, Combinatorial Testing, Hexawise test case generating tool