One of the most difficult obstacles to attaining enterprise-ready SOA is the sheer scale of the systems and data that need to be managed. To test the actual results of an application, we need a very realistic set of data – both positive and negative – to input, and then get out of the environment under test.
True, we can map much of our interaction with other Services according to the metadata we set forth during architecture and design processes. But when you get past that ideal model of connecting the endpoints, you still have the mainframe, or an SAP or Oracle Financials enterprise system, and the administrative owners of that system, to contend with. The data and business logic embedded at these layers has been added to and customized over the course of several years.
So why can't we just have developers and testers work against the live system?
Well, those system administrators might be reluctant to provide access to key business systems in deployment. Beyond that challenge, getting a bed of realistic test data in place can be more than difficult – and hardware doesn’t scale to replicate the terabytes of data such an implementation requires. Implementing a complete mirror image copy of the system to test requires another enterprise license and implementation team – far too costly in scope.
In addition, managing data in order to do successful service development, integration and testing can truly be a moving target. It's tough to maintain that context of an actual user moving through the system, without actually having access to every implemented layer.
The best practices for overcoming the data crunch isn't by any means an easy road, but it has to be done. It still starts with good architecture - mapping out realistic business, and the relationships that define them. Next, we need to capture as much of the data as we need to provide a realistic test environment. There isn't a way to replicate all of it, but we need to obtain enough to encompass most of the virtualization of test beds, and the behaviors of apps as Virtual Services, can help you get to the point of reaching the 80/20 rule for the data you need most often.
Finally, we need a strong Governance approach to staging, promoting and deploying the application, which includes continuous testing and validation of the expected behaviors, and underlying data. No amount of simulation in development can account for all of the unforeseen consequences of changes in the deployed system.
Part of our approach is to automate that process of data capture and modeling with Virtual Services. By monitoring a given Service and all of the live and test traffic that is going into, and out of it with LISA, you can get a pretty rich data set. Often people tell me that it must represent only a scenario where the data is not very complex. Not so. Though admittedly, the more complex your data set is, the more elaboration and checking you need to do on that data set. And what if there are data errors within your Virtual Services? Well, in a sense that is what you are looking to uncover, right?
However, it is important to note that there is no shortcut for Continuous Testing & Validation.
When you move into staging and deployment, you need to move from that virtual data model to actually doing continuous integration testing. The point is - to get 80% of the testing you need done at those early stages accomplished by using a Virtual Service data model, then you can have a more dedicated testing effort with less conflict against that live data in deployment.
Obviously, there is much more to this process than I can cover in a post, but I hope you will seek out leaders and solution providers that can help you accomplish all three of the above goals.
