Hitting the right QA Notes with SD Times' Feinman

Recently I had the chance to catch up SD Times' Jeff Feinman for an interview about the latest trends we're seeing in software QA. In the resulting article, "Hitting the Right Notes in QA", Jeff surveyed practically every software testing vendor in the field, from unit-and-code coverage firms to the functional UI types of tool providers.

 The article talks about the importance of metrics and measurement tools that support testing - comparing software testing in today's complex environments to attempting to play a symphony, saying it would be "like a symphony orchestra getting up on a stage without a single sheet of music and trying to play Bach or Beethoven."

The golden rule of software testing - the earlier you find bugs, the less costly they are to fix - was mentioned as the driving force for measuring the value of testing. Cost-per-bug found in development is indeed much cheaper.

Agreed - however there is also the aspect of automation closer to integration time which must happen in these complex environments that must happen - as there are simply too many possible combinations and points of failure in today's distributed and "loosely coupled" environments like SOA. How many aspects of enterprise software are really built from scratch, and how many are reused or re-purposed from existing components.

I wager that going forward in QA, the big challenge in testing is no longer at the code level, or even the user interface -- that isn't where the big chaos is happening, and there are already a number of processes and tools that do that successfully now.

All in all appreciated the perspectives in this industry-wide QA survey.

Comments

[Manoj]

Agree to the automation need for SOA testing...
An SOA orchestration throws in hundreds of different possible combinations for the services and also data combinations. Manual testing will not work for SOA based applications. Trying to hit a moving target with static fragile manual test will not yield results.

[Jason]

Thanks for your comment, Manoj. Yes we are finding just tying a few services and components together, you can have far too many permutations for the human eye to reliably test. Risk-based testing is one helpful approach, but if it is largely manual and not touching the integration points, that is still only testing for the things we THINK can go wrong. We need more awareness of this, and the leadership of advanced testers to get out of the mode of running manual or surface-level test suites.