Here's another great story Ken and I gathered after a recent successful deployment. Technology is always rushing forward with more customizable development and test lab solutions for every enterprise. Even at iTKO we continue to extend leading-edge product functionality for all the unique scenarios that customers bring forward.
Much of the time this type of customization is focused on making automated testing or performance testing more efficient. But in reality, there are still tens of thousands of manual testers out there, and save for some slight process improvement, they have been largely ignored over the years when it comes to automating their complete environment.
An iTKO telco customer recently showed some examples of how they enabled their manual testers to be much more efficient through the use of Service Virtualization. After sending transactions through their Front-End CRM UI, they log into their Back-End UI which shows them the detail of the Virtual Services captured behind their System Under Test. This Back-End UI is something they developed on top of LISA Virtualize, and it exposes the current data inside LISA, such as the status of various Customer Orders, previously executed transactions, etc.
That’s mildly interesting, so testers can at least see if the transactions are properly reaching the Back-End. However, what’s really clever is that they let the manual testers go in and modify the Back-End Virtual Service directly. Testers can automatically skip a number of steps to put an order into the completion phase (which would typically take a very long time to enter in the Front-End UI), or they can setup negative scenarios such as an invalid order, error condition, etc.
This lets users decide in real-time exactly how they want the system to behave. They are able to test a larger number of scenarios simply by being able to create conditions that the test lab normally does not allow. And manual testers are empowered to setup their own scenarios without involving the DBAs and higher-level Test Data Management architects. So by being able to quickly “prepare” the virtual environment for a specific test, they do not have to spend so much time setting and resetting test data.
Fortunately for them this has come with a number of positive results as well:
- Functional coverage after the first project stands at 80%, and they plan to reach 90% by the next release.
- Testers are 40-50% more efficient without needing access to specific test data, or getting a DBA to setup a specific scenario
- Testing across the board has started earlier in the software lifecycle, and they are finding defects much earlier in the SDLC
- Oh, and on top of all that they’re already saving over $200K annually, simply by not sending transactions to validate their system against the third-party vendor's test lab system (which was seldom working reliably and had unstable data anyway).
We hope you find these real-world stories useful in thinking about how to improve your own dev and test processes. Let us know what you think, and if there are any software challenges you would like us to cover in future articles!
Comments