The ITKO Blog: Service Virtualization and Application Lifecycle Optimization - CA Technologies

How to be Fast and Agile with Increasing Services Complexity Without Falling on your Face

Agile development is spreading in the SOA world to rapidly decrease development times, but if testing is not done properly, you might lose all the efficiency gains through having to conduct a massive cleanup effort afterwards. Like building bridges, fix it as you go is a much better approach than fix it later with massive traffic flowing over the structure. InfoWorld's Eric Knorr writes about this in his recent Modernizing IT article, "How to be agile without falling on your face."

Eric reports on a conversation with iTKO’s Derick Townsend. Derick said that companies within an M&A cyclone -- particularly in times of high volatility like we are seeing today in financial services markets -- are desperate to accelerate application development projects.

"They get 80 percent through a project, and they think the end is in sight, but the last 20 percent really stymies them, and that has to do with the test and delivery part. They're dealing with so much complexity. Telcos are experiencing this too. The components and underlying systems are changing constantly underneath," Townsend said.

Eric goes on to write about how iTKO provides what we call Service Virtualization, which is different than traditional hardware virtualization. Instead, iTKO LISA absorbs the behavior of all relevant components, services and data in an extended IT environment and simulates them, This capability allows developers to determine how an application based on those services will behave under various circumstances in the real world. Developers can become more agile using Virtualization practices to support increases in complexity, without a corresponding increase in testing and development risks and cost.  

Pros and Cons of Application Testing in the Cloud

Well it seems we have a series of posts on Cloud Computing and Test environments to cover, as we've been seeing a lot of good background education in our research on the topic.

Jan Stafford recently posted on a conversation with Bernard Golden - author of IT bestseller, Virtualization for Dummies. Bernard is a strong advocate for virtualization and cloud computing technologies, but warned that it requires some new skills and has to be done right. In a brief video he said that application development is one of the areas were the cloud makes a lot of sense.

Bernard provided this example. A firm wanted to test their application and need 100 browser instances. In the old days it would have required 100 machines -- that would be a massive undertaking. Even with hardware virtualization, you would need 5 to 10 machines, and there would be some complex configuration issues. However, by putting it all in the cloud, they were able to sync up 100 virtual instances of the browsers and take them down over a weekend at a dramatic cost reduction.

However, that's just a web UI testing exercise. Bernard added that few development labs actually replicate the exact server and software environment as the target data center that will run the application in production. The cost of that set-up would be astronomical by conventional means, but that is starting to change. With Service Virtualization in a Cloud Environment, you can basically eliminate all of those downstream dependencies from the equation quite inexpensively.

In addition, development in the cloud allows you to scale up and scale down the application for testing of various load sizes without incurring hardware costs. The cloud can support more difficult testing requirements like load testing and scaling that many labs can’t scale to support. Finally, since many applications will be delivered through the cloud, testing them first this way makes sense.

To reap all these benefits, Bernard cautioned that there are several important decision such as finding the right cloud services provider. Software testers will have to acquire some new skills, and development teams have to be careful about integration with internal applications. They will also have to manage application lifecycle in the cloud. Many IT service providers have also made significant strides in delivering improvements in these areas.

Green SOA is Good

We have covered SOA’s role in green IT before on this blog, (see Making IT More Green and Saving Money at the Same Time Through SOA). Loraine Lawson provides an useful addition to the conversation with her post, SOA's Green ROI.

Lorraine sets the stage with the fact that today, over 70 percent of the SOA adoption rationale is business driven, versus only 30 percent in 2006. She adds that technology may never again be the driving force in the corporate decision process, but in SOA, it will always be the implementation mechanism. We have written that with SOA and many other tech efforts, going green and getting a better ROI are interconnected.

Lorraine goes on to cover the opinion of Deborah Grove, a green IT specialist, who has long been of the opinion that SOA is key to the long-term definition of Green IT. “Right now, we are picking up the 'low hanging fruit' by addressing the PUE (power usage effectiveness) of data centers’ power consumption.  However, looking at the entire IT food chain, ensuring that ecosystems of suppliers, partners, and customers are moving to an online collaborative style with less paper and less independent development of customized applications through the use of SOA means that fewer hardware pieces will be needed for dev and app servers.”

I certainly agree and automated testing and virtualization can play a significant role in reducing waste from the IT food chain. Lorraine points out that Verizon Wireless used SOA to green its IT infrastructure, reducing its hardware footprint by 95 percent. The company measured its ROI by reduced tonnage of hardware in the data center. However, the green of virtualization is not limited to hardware, as we have discussed here (see Guide to Service Virtualization in Enterprise Application Development – Part One of Six.) It is nice to see an increasing awareness of the green potential of SOA.

 

Now is the Time to Get Your SOA House in Order

Joe McKendrick recently wrote about Using Downtime to Boost SOA. He refered to Kleber Bacili, CTO and founder of Sensedia, says "times like these may be the perfect times to get your SOA house in order.”" We certainly agree.

As Kleber said, the "recent 'SOA is Dead' debate has actually rejuvenated interest in service oriented architecture," He adds that if your budget has been tightened, this is a great time to clean up those foundering projects that never quite seem to get traction.

Kleber writes that this clean up can best be done implementing SOA governance initiatives with a well-planned and incremental approach. This allows you to streamline wasteful and unruly SOA projects, eliminating duplicated development through governance practices.

Indeed we can turn the tables on mere cost cutting and generate value here. There is plenty of opportunity to increase customer capture and retention, and deliver increased value by delivering the most critical business functionality faster (and with the quality to call it "delivered").

To do so, you need to validate that each release does indeed meet requirements, or you end up with a mark in the cost column instead. Virtualization of services can also eliminate costs while increasing agility, by eliminating the dependencies that can keep your development teams on the bench waiting for access to live services and incomplete systems from other parties.

Of course, if you do all of the above, your teams won't have any excuse for downtime anymore -- they'll be too busy delivering!

SOA's Practical Entry Points

SOA continues to move forward, and judging by the headcount at the latest IBM Impact SOA event we attended, it shows no sign of waning in popularity.

Eric Roch recently discussed a TechTarget survey in his post, SOA Survey Points to Practical Approaches. He wrote that the survey results indicate that the top SOA benefits organizations aim for are improved data integration (32%), enable legacy application integration (32%) and integrated disparate department applications (23%). These are very practical approaches to more cost effective data management and IT operations. Integration plays to SOA’s strengths, but also ups the stakes for automated testing, as we have discussed here a bit. You have more heterogeneity, more integration points and more change than ever - and we are far beyond the point where manual or UI testing approaches can succeed. These dependencies are also dynamic, so you need constant validation of business outcomes.  

BPM is the next most common SOA application, with an indication that its use will grow, as 38% of the subjects are planning to use it in the future. The survey also provides another data source to suggest that SOA is not dead as 49% of the respondents said their organization has one or more SOA projects under way. In addition, it is becoming more strategic as 60% characterize their current or future SOA projects as enterprise level as opposed to operating simply at the departmental/divisional level (21%), or single, isolated projects (19%).

We have repeatedly seen SOA used for cost reduction efforts around the integration of distributed systems and improvement of business processes. Eric calls this "SOA blocking and tackling," and says a great deal of it remains to be done. Automated testing and validation provides the means ensure that SOA meets business goals, while virtualization eliminates service constraints in the environment, so testing and validation can continue. 

Realizing the Benefits of Enterprise Integration and SOA at Lower Costs

We have now made available the complete podcast of a recent webinar in a compact video: Realizing the Benefits of Enterprise Integration and SOA at Lower Costs, featuring iTKO’s John Michelsen and Perficient's Eric Roch who cover how to achieve shorter release cycles and lower costs within SOA implementations. Here's an overview of some of the topics covered in this well-attended session on iDevNews.

Eric begins with key challenges to successful EAI and SOA implementations and some case study examples. Perficient is involved in technology implementation with over 1500 consultants. Eric leads the Business Integration Practice that includes BPM and SOA. He covered integration testing that includes a combination of services, components, and systems. Eric said that the scope of most projects causes test environment complexity issues and service availability issues as we have often discussed here. Often quality assurance is pushed to the end, and companies end up with infrastructures that do not perform. There are also complex dependencies between development groups.

Tools are also often dated holdovers from client server days. So companies must either build or buy new test tools. Most companies start off in a build mode but they end up with tools or custom "test harnesses" that cannot be used in a repeatable way. The test harness building option can become more time consuming than building the application itself. Eric encourages companies to make a well-considered test tool decision to ensure there is sufficient agility. Eric then went into great detail on the specific test requirements in SOA and provides a case study from an electric utility company where they employed multi-tier testing with LISA, and used the Virtual Service Environment capability to simulate "test reponsive" endpoints on the ESB, Services and data layers, rather than hand-coding their own responders. See the attached diagram for an example:

iTKO founder and "Chief Geek" John Michelsen then drilled down into a bit more detail on testing, validation, and virtualization within SOA. He focused on areas where you can drive out costs and provide greater efficiencies. John mentioned that traditional testing methods cannot automate testing in the complex world of SOA and this limitation produces significant inefficiencies. We can now build applications faster than we can test them using the traditional methods, and this adds unintended costs to SOA implementations. The power of SOA to integrate services raises the complexity to test this integration.

John covered how iTKO LISA automates testing, provides continuous validation of the integrated services, and offers virtualization to get around availability constraints on testing and validation. Virtualization gives almost infinite ability to virtualize constraining services and also strips out up to 90% of traction costs for these services. He mentioned an example from a cable company where a simple code change caused significant ramifications to the services that were affected. Continuous validation can pick this up.

Then a Q&A session allowed for further exploration and a great deal of interaction. The first question asked if the John and Eric had noticed project failures when firms attempted to integrate several ESBs, often as the result of mergers or acquisitions. John gave a number of examples and how the issues were addressed with iTKO after the problems occurred. Eric offered other examples and how they handled the issues.

The next question asked how to use virtual environments for integration testing. When can you use service virtualization versus requiring a live validation? John aid that this is a great question as it gets to the heart of the iTKO virtualization approach. LISA does not virtualize you, it virtualizes the things you are dependent on. In about 80% of your development cycles, that fact that your downstream service is live is a problem that service virtualization can address. Now in the final stages of integration testing, then you want to test against live data. So you virtualize your dependencies to take costs out during development and then switch to live data at the end. Eric offered some more examples.

The next question asked if John and Eric had experience with the virtual validation of SOA governance and how this worked? John first replied. LISA provides the infrastructure to do this and experts at firms like Perficient can use LISA and integrate it as the validation engine. Eric added that governance is about people, processes, and policies. He likes to fit the governance model to align with the culture of the organization. Then you measure it and change it over time. You cannot get too heavy handed about governance without creating adoption problems. Eric went on to offer more details on how they do this.

The next question asked how a delivery partner like Perficient can keep its own costs down in a complex implementation to stay competitive. Eric said they bring expertise and they have a set of methodology to increase project velocity. He often sees that SOA is too focused on architecture and services but not covering the business issues and the quality concerns. It is about the process and not the technology.

Then the moderator asked why testing is such a big chunk of SOA implementation costs when there is now a modular architecture. John said that at the granular level testing is easier. The problem comes with the overall number of dynamic parts and the resulting complexity. Eric said the problem of dependencies in the lifecycle has existed for a while, even with mainframe systems. You can get efficiencies if you can eliminate dependencies.

We encourage you to hear the complete podcast. It is available at our iTKO Resources site.

Cloud Insanity - Stop Repeating Mistakes of the Past

I enjoyed Mike Kavis’s post, Cloud Insanity - Focusing on all the wrong things, again, especially his definition: "Cloud Insanity - To repeat the same behaviors that caused SOA to fail and expect a different result." How true. We have written a good bit on this blog about how SOA can succeed if you pay attention to the right stuff, like the governance issues, the business issues over the tech issues, and change management (see for example, Avoiding SOA Disillusionment - eBizQ Panel Recap).

These are some of the issues that Mike also lists as things to address with both SOA and the cloud. He goes on to say that we should focus the cloud debate on how to "tackle security and compliance, how to manage hybrid solutions, how to provide reliability and scalability, how to prevent latency, and many other important topics. But enough of the Cloud Insanity."

As with any hot technology, we have many vendors and consultants repackaging their expertise as a "miracle tonic" for Cloud Computing, without regard for the drivers that determine where it makes the most sense for business. We should apply the lessons learned from successful SOA efforts to help ensure that cloud efforts are successful (see Avoiding the Hidden Costs of Cloud Computing with Lessons Learned from SOA).

One of the additional lessons similar to service-oriented development is the need for systematic testing that makes use of automation and virtualization as much as possible, so we can validate changing cloud-based services as part of a continuous business process. In return, Cloud computing is a boon for test labs and virtualized service environments, adding a lot of flexibility and options in how these assets are deployed for the business.

Comparing the State of Consumer Web and Enterprise Applications

Here is an interesting post by Jeremy Chone in Bits and Buzz on Enterprise Web vs Consumer Web [2.0]: Top Six Differences. I was struck by this comment on one differences and thought of similar things we've seen in the field. Especially the fifth factor.  Here it is in Jeremy’s words.

"5) Integration (Loose vs Strict) ... In the consumer space, application integration happens very organically. Application providers (e.g., Twitter) or application container providers (i.e., Facebook) define  interfaces with which everybody can integrate. The result is that thousands if not hundreds of thousands of applications will potentially integrate with some of these interfaces. Consequently, interfaces tend to be simpler, lighter (e.g., REST), and often even client-centric (e.g., OpenSocial and iGoogle). In other words, consumer application integration is about breadth rather than depth. In the enterprise space, on the other hand, application integration has to be thoughtfully designed and managed and often has an impact in all layers of the application (i.e., presentation, logic, and data). Heavy and expensive technology, such as ETL, SOA, JMS, and Portal, are often an effective way to establish strong integration between applications."

Many of the current enterprise application efforts have been influenced by the consumer web (and you can call it 3rd party services, Cloud, SaaS, what have you).  However, success depends on recognizing the challenges and differences, as much as the opportunities and similarities. Much of the consumer web involves preparing for the unknown such as traffic volumes, as well as integration from multiple and unanticipated applications that you may not have control over, or be able to enforce an expected service level on.

While more is often known about the architecture in the enterprise world, the level complexity can go way up due to much deeper layers of system-to-system interaction.  There is also less tolerance for lapses in performance and data integrity.  This is where the edge between Consumer Web and enterprise applications gets particularly jagged, and where we are seeing the most innovation.

For instance, the need for software testing to be truly stateful and asynchronous as you cross over the edge between the enterprise and consumer web is a necessary thing (and quite a mouthful) -- if we are going to consume the best of both in our distributed apps.

How do you validate a transaction, if the response may move around among several third parties and decision points, and maintain its state when (and if) an appropriate response occurs? This kind of leap is going to take a lot of automation and innovation to allow the two to achieve scale.

Meeting the Business User’s Expectations with Cloud computing, SaaS

Neil Ward-Dutton recently posted a comprehensive piece on Cloud computing, SaaS and SOA - the universal service network. He wrote that "Regardless of where services come from (and indeed because they will come from multiple places, creating cross-enterprise service networks), it's increasingly the case that in order to deliver effective IT capabilities in the 21st Century, you need to understand SOA principles and build technology and management structures that really support the principles of service orientation."

He then goes on to discuss two sets of business user expectations. First they expect flexibility. Neil writes, "when business teams that are using SaaS-based offerings learn about the infrastructure side of the story - how easy they can be to customise, extend, and integrate with - and ask why internally-developed systems don't exhibit the same qualities? Another slab of SOA pressure, that's what." The business will demand a high level of flexibility and control in how service-based offerings are composed and consumed.

Later Neil brings up the second expectation. He refers to Todd Biske, on ITIL and SOA, and states how it's "crucial to consider the full service lifecycle when you do SOA, and drive governance to ensure that you can deliver "real service" - not just code wrapped in XML-based interfaces." Then he offers some examples where the provider's idea of service doesn't match the consumer's expectation.

In addition to flexibility, businesses want applications that work. These two points go to the heart of the need for higher levels of test automation and agility throughout the design and development process, as well as the benefits that virtualization can bring to both flexibility and getting good, ready data.

Without rapid, repeatable test cycles, you cannot achieve flexibility as development cycles get bogged down. This was the fate of one financial service institution before they moved from manual to automated testing. Service Virtualization can further accelerate the development process by eliminating the bottlenecks of constrained or unavailable systems as one European telco discovered.

Neil concludes that the "rise of cloud computing and SaaS should entice us to revisit our development-centred assumptions about SOA and search for a 'bigger picture' that focuses on consumer expectation and value first." One way to do this is to simplify the development process and enable faster iterations. This enables us to focus less on the integration challenges of the technology, and more on the business challenges the technology is designed to meet.

Avoiding the Hidden Costs of Cloud Computing with Lessons Learned from SOA

George Lawton recently provided a post describing how "as with SOA, some development costs obscured by cloud computing." He extensively quotes iTKO’s John Michelsen and this is greatly appreciated.

George writes that while cloud computing holds great promise, it will be filled with surprises, and some of these will reflect surprises encountered in the early going of SOA. The more loosely coupled and distributed the cloud based system becomes, the more moving parts you don't have control over -- and the more hidden development costs lurking within this loose coupling.

Much of cloud functionality is created and hosted outside the organization, which creates ideal "pay as you go, leave when you want" cost and agility advantages, but it still requires testing, perhaps more than ever. To compound the problem "after the application is deployed, the service provider can make changes that could adversely affect the application without notice."

These needs call for automated testing and continuous validation. Then new two needs emerge as George notes. "After an application has been tested and deployed, paying per-unit of service (storage, CPU time, and bandwidth) allows a new app to get deployed with minimal cost. But these fixed costs can take a hidden toll when a company has to do any sort of testing, and particularly load and performance testing of a new app."

For instance, take a firm we know, whose operations team needed to run a performance test to verify required Service Level Agreements (SLAs) to verify that customer response times would be within bounds. One routine test cost about $10K in access fees from the service provider - an unforeseen bill that wouldn't have appeared if the team was banging on their own servers. That inhibits the team from testing and adjusting their service appropriately.

In addition, there is lack of control and ready access over many of these external services, especially if they are still being developed. This is where service virtualization comes in. Now you can bypass costs and areas where you cannot easily set up test situations on dependent services. As George notes, the methodologies and tools that were developed for SOA testing and validation will come in handy to handle cloud applications.

Of course, none of these challenges mean Cloud-based development isn't a great thing. We love it for all the reasons we loved SOA, then SaaS-based models, because it gives the enterprise incredible flexibility and lower cost of entry for needed infrastructure and applications. We are also seeing a new level of quality by pushing Virtual Services to provisioned Cloud environments for testing and performance labs, in order to keep unexpected third-party costs under control and allow unfettered access to test and development teams.

Expect to hear much more on this front as Cloud Computing gets tapped for serious, distributed enterprise applications that need validation and virtualization.

James Governor on Combining Cost Reduction with Innovation Through SOA

James Governor, co-founder of RedMonk, recently spoke at IBM Virtual Global SOA Forum. Joe McKendrick provides a transcript of his talk on ebizQ, "Work Smarter, Take Out Costs In a Tough Economy." He begins with saying that he "wants to talk about culture, because service oriented architecture clearly sounds like something technical, but really if you want to succeed, its going to be all about the people."

He goes on to say that SOA is the language of change. SOA was started, in part, to enable businesses to become more flexible. "Enterprise customers were concerned that their cost of integration was very high. Once they had built a system, effectively, they poured concrete onto it." This probably affected the thinking about business process and the ability to innovate, and not simply the technology components.

In today's economy, change is needed more than ever. It now takes on two components, the ability to quickly adapt to changing market conditions, and the ability to do this in a cost effective manner. Though SOA was once an overhyped term, there is a "little soa" in the way companies are now quietly using service-oriented approaches to create real value.

"Little soa" encompasses everything else lumped under flexibly leveraging technology to meed business needs - from BPM to integration platforms, rich Internet apps and REST-based approaches that have business services under the covers. "Little soa" can address both of these components and hopefully changes the way people think about innovation and the capacity to engage in it and implement it. This is where the people side of technology comes in.

James goes on to say, “We're going along this road, and it’s very important that we need frankly, a fast car that will keep us on the road. And SOA is a language that the business can talk to IT and systems, in order to stay on that road. In order to have alignment, so that we can move forward, and actually deal with change, and deal with these curves that we're being thrown."

He later mentions virtualization. This is one way to keep the fast car moving forward and stay on the road at lower cost. We have been covering this in our series on Field Reports on SOA Success. See for example, Part Five: Major Airlines. Here the airline was able to virtualize the kinds of services that are normally prohibitively costly and complex to replicate using conventional means, and keep their SOA development and testing on track.

(Concrete Photo by Brenda Anderson and Car Lights Photo by miss blackbutterfly both from Flickr.)

Q&A with Butler Group's Rob Hailstone

We recently had an extensive briefing and demonstration process with UK-based analyst firm Butler Group's Rob Hailstone - Practice Director, Software Infrastructure, and his team, the result of which is their independent Technology Audit report covering iTKO. We paid for download rights to this analyst document for limited time from our site, and we'll cover the report in a future post.

So after much scrutiny of iTKO's LISA product on their part, I decided to turn the microphone around, and had some Q&A questions for Rob, which I think provide some insight into his perspective on the enterprise software space.

Q: With economic pressure to optimize IT greater than ever, what issues are currently "top of mind" for your practice right now?

RH: My own practice area focuses on SOA. The most immediate impact of the global economic downturn is that organisations that don’t have a really concrete plan for how SOA is going to benefit the business are suspending or abandoning those plans. This is a good thing. Business benefits don’t automatically follow the deployment of SOA, they have to be targeted and worked towards. It is now more important than ever to understand exactly what has to be delivered in order to justify the cost of SOA deployment.

Q: What are your thoughts on the rate of innovation going on today in fields such as virtualization and cloud computing in the enterprise?

RH: Virtualisation has been central to IT ever since the first multi-tasking machines, but the apparent commoditisation of IT hardware caused us to lose focus. We are now much more aware of all the peripheral costs, such as management, power, floor space, etc., and it is obvious that the requirement for virtualisation has never really gone away.

Cloud has a different set of considerations. Some CIOs that will happily embrace virtualisation will think long and hard before relinquishing their physical resources. Cloud provides a great way for packaged application vendors to deliver the application as a service without having to make the heavy investment in scalable, fault-tolerant infrastructure, and it also makes a lot of sense for smaller organisations who need cost-effective off-site backup or a full application platform. However, organisations with an established IT infrastructure that might look to the cloud as a top-up computing capability will probably find the cost of effectively managing a hybrid environment too high in the immediate future.

Q: You've conducted a number of Technology Audits with Butler. How do customers best leverage these documents?

RH: The obvious and most frequent use of Technology Audits is to provide enough information so that a subscriber can decide whether to include a particular product on an evaluation shortlist. They are also used as a reality check on a particular type of strategy to establish whether there is sufficient supporting technology in place to be worth persevering with a more detailed investigation. We try to ensure that we point out where a particular product is not likely to be appropriate as well as showing its strengths.

Q. Any other thoughts in terms of this report?

RH: SOA is all about enabling a higher rate of change, but just putting the architecture in place without the control mechanisms will ultimately lead to chaos. Implementing a change management process backed up with a sound set of testing technologies is an important part of building for long-term success with SOA.

Thanks for letting me ask some questions, Rob, and we had fun digging deep into LISA for the audit.

Eric Roch on Starting SOA Right, Again

There has been a lot of discussion on the success, or lack thereof, of SOA. A number of people have pointed out that the issues are more likely implementation shortcomings than problems with SOA itself (for example, More on SOA Decline and Avoiding SOA Disillusionment).

Since we have an upcoming webinar next week with SOA expert/blogger and Perficient chief technologist Eric Roch, I decided to review one of his recent takes on the topic in his "The SOA Blog." Eric provides some useful guidelines for those who are taking a second look at their SOA efforts in his post, Getting Started with SOA - Again? Eric advises us to start simple, tackle a business problem, and pick your first project carefully. He works off some advice from IBM. He suggests that process and connectivity efforts are two good ways to get restarted.

Eric adds, “The connectivity entry point is often driven by the purchase of new software packages that require integration. The process entry point is an opportunity to work with the business to improve business processes. Both of these entry points offer quick successes and ROI.” Eric adds to begin with BPM, and let that be a driver of SOA. So far, I agree.

As I have said before, getting the business teams to think in terms of optimizing, integrating and reusing processes, makes it easier for the IT side of the house to architect and manage their SOA around these vertical and horizontal processes. When interactions between groups, decision points and constraints are modeled successfully, it makes the case for technology that can support it. Eric then provides three more entry points in addition to process and connectivity. Supporting enhanced collaboration, better information access, or asset reuse are other likely starting points for SOA success. These are great suggestions that all can show the value of beginning together multiple services. With all of these, proper testing and validation is a prerequisite to SOA success. 

We hope the above advice gives you some starting points for, uh... "un-re-disillusionizing" your SOA initiatives and getting some initial wins. We also hope you can join us March 31 (or afterward) for the webinar with me and Eric.

Monsters of SOA Webinar 3/31: Reducing SOA Costs with QA

We're looking forward to what I am informally dubbing a "Monsters of SOA" online discussion and presentation titled "Realizing the Benefits of Enterprise Integration and SOA at Lower Costs" coming up on IDevNews March 31, 1pm Eastern with iTKOs Chief Geek & Founder, John Michelsen (left), and Perficient's Chief Technologist Eric Roch (right). iDevNews' editor Vance McCarthy will moderate.

We've had the pleasure of collaborating with Eric's consulting teams on  the ground at  several large-scale SOA migration and implementation customer sites, and we have found there are many points when our best advice and tools for testing and validation merge with Perficient's best practices and delivery process - so much so that they have developed a specialized QA program for SOA - which according to real project timelines can consume up to 30% of a typical SOA initiative's budget.

The fact is, the more services-based functionality you add, and the more agility you introduce, the faster your rate of change is. And we aren't throwing anything away by adding SOA, so that means more you need to test, and more demand for stronger validation throughout the project lifecycle.

Don't get me wrong of course, agility and the distributed approaches that enable it are great things - and QA and validation are great things, but while we deliver that, we need to deliver it in more automated, smarter ways so it drives agility and cost reduction instead of expanding in cost with the complexity of the architecture.

Eric also puts out a SOA blog we frequently read on Toolbox. In this talk we have two IT leaders with hundreds of SOA-style project engagement experiences between them, so stick around for the insights and Q&A.

Anyway, hope you can tune in March 31 or afterward to watch the session archive, signup is free here: https://secure.soatube.com/registration/w_iTKO2/?code=itko. A preview copy of the slides can also be downloaded.

Perfecting IT Strategies in a Down Economy

In today's tough economy, you can't even read an article that doesn't start out: "In today's tough economy..." This one is no different. Fortunately when it comes to IT, the message does not need to be accompanied with knee-jerk cost-cutting measures.

Last month Todd Biske offered some solid advice on what to do in today's down economy in his post, "Don’t Go On an IT Diet, Change Your Behavior". Needless to say, his post remains relevant. I especially liked his comments:

“The whole point of Enterprise Architecture, SOA, and many of these other strategic IT initiatives is to allow IT to be more agile- to respond more quickly to changes in the business objectives. Guess what? We’re in the middle of a big unprecedented change in our lifetime. My guess is that the best survivors of this meltdown will be organizations that don’t go on a starvation diet, but instead simply recognize that their priorities and goals have changed and execute without significant disruption to the way they utilize IT.”

Todd also wrote that cost cutting efforts to respond to the economy should not mean that organizations drop their SOA initiatives, or other long-term initiatives, especially when most of these efforts are trying to reduce costs cost by eliminating redundancy in the enterprise. It would be ironic as these efforts are exactly what are needed today.

SOA efforts should be more closely managed and held to their ROI objectives in any economy - otherwise, they will not have a chance to create the value you expected. It may be that closer review will actually improve the number of SOA efforts that are successful, as people will be more highly motivated to get them right. There is no time for poor execution now but there is a lot of room for innovation. As Jonathan Schwartz, the Sun COO, wrote in "Innovation Loves a Crisis," now is the time to make changes that will make you most adaptive to the new economy and best positioned to come out of the downturn on top.

McKinsey also supported this strategic thinking in their report, "Leading through uncertainty": “The task of business leaders must be to overcome the paralysis that dooms any organization, and to begin shaping the future.” Transforming to SOA - not because it's "cool" but because we have real value expectations, is one way to become more flexible and cost effective at the same time.

Accompanying that, we are also sure to mention Service Virtualization as a key means of increasing the availability of SOA assets in the lifecycle, at a small fraction of the environmental cost. It is money well spent not just in terms of cost savings, but more so in coming out of a lean time with a more agile and efficiently delivering dev and test organization.

Minimize IT Outsourcing Risk with Collaborative Quality - Part Two

This is the second of a four part series that summarizes the key points in Minimize IT Outsourcing Risk with Collaborative Quality. You can click on the title to get the complete white paper.

In the last post, we looked at the risks that can come from outsourcing IT development. Solving for these risks, not just through contractual negotiations, but in every day collaboration, can unlock the benefits of outsourced IT and address larger distributed development issues. In fact, if the appropriate IT infrastructure is available to better manage accountability and quality across distributed teams, while eliminating dependencies and constraints that restrict the agility and productivity of these teams, then it follows that these benefits could apply not just to outsourcing situations, but to any distributed team environment.

The first step in meeting the challenges of distributed teams is to gain a commitment across all teams to deliver components against well-defined requirements, in an auditable form, with the ability to accommodate the highly interdependent nature of today’s applications. This requires more than the right attitude, it requires the right tools.  Key testing, validation and virtualization capabilities will help firms mitigate these risks, and foster more collaborative quality across interdependent teams. These capabilities include:

1. A collaborative testing framework that directly verifies all technology layers with a high level of automation, so the team can better meet defined requirements at lower cost.
2. Continuous Validation so both parties can assure that each software component supports all upstream and downstream dependencies, in an auditable form.
3. A Virtual Service Environment (or VSE) which simulates the behavior of the highly interdependent and critical systems the teams must build and test against, even when they may not have access to live or completed components.

With these capabilities, the consumer of services should not just receive code assets, but also the deliverables they need to rapidly validate them - collaborative testing is matched to functional requirements. This is important, since resolving a misunderstanding early in the requirements phase can be orders of magnitude less expensive than fixing it later in deployment. So how can we move testing earlier into the lifecycle, to save costs and improve quality?

If the offshore partner’s testing is not tied to functional requirements, there is no way to know if there is any value to the test. The key to achieving collaboration is to demand that each component a team delivers is also comes with a suite of working tests that prove its functional requirements have been met. That way the next team, whether local or offshore, can iterate on testing from there.

For instance LISA tests are saved as .XML documents, then the business team can augment these processes by combining them with traditional requirements docs as executable test assets in ALM and Test Management tools. From this foundation, the tests become part of a continuous build and validation process, where regressions are proven to match defined requirements in later testing cycles. In our next post, we look at how you can handle the changing components and dependencies and achieve this kind of validation on an ongoing basis.

Field Reports on SOA Success - Part Six: Major Enterprise Software Firm

In life, there are some good surprises (birthdays, hearing a great new song, meeting that special someone), but in enterprise software, not so much. This sixth example in our stories from the field is about a major enterprise software firm.

This firm decided to move much of the sales of their software upgrades and maintenance contracts to a web portal, where clients could just buy and download the updates they needed. Previously the sales force would not have to get involved with contacting each client and conducting manual orders on routine stuff, which wasted time for both parties. Many clients had open POs so they could just add on these incremental purchases. This was to be a win for both sides.

To set this up, they needed to access the customer data and their software inventory. They could then make suggestions about what additions might be needed, offer the ability to instantly purchase, and download. They did manual testing of the system UI, and thorough unit testing of the software. Everything seemed okay for rollout.

However, as they discovered three months after the launch, there was a problem with the all-important middle step. The system was supposed to push the order into the enterprise invoicing system, SAP in this case. Then the client would be billed.

Ninety days after launch, the guy responsible for the SAP transaction system asked if they had started this new system since no orders had entered into the system of record. To their embarrassment, the clients were getting their software downloads, but their invoices were not getting recorded.

It seems another group implemented a policy so that each account had a password change every 90 days. These changes occurred between the application test time and the launch time. So fulfilled upgrade orders were getting logged as bad data, and then flushed on a regular basis. The clients were not complaining because they were getting the software, and likely did not mind that the invoicing appeared slow.

The firm brought us in to do end-to-end business process testing to make sure that all steps in the process worked on sample cases. We tested not just at the web ordering UI layer, but followed that transaction through the app server, ESBs, CORBA and all the way to confirming that it made it into the system of record. We call this practice “invoke and verify.”

After software was downloaded, LISA made sure the invoice was processed for that specific download, and the team quickly rooted out the “changed password” culprit that was causing orders to get routed incorrectly from within the ESB layer. This validation was also done on a continuous basis to make sure any new changes did not disrupt the flow. Now all steps in the process work, and the original goals of better customer service at lower cost -- without the unwelcome surprises -- were achieved.

You can find the complete case details here.

Field Reports on SOA Success - Part Five: Major Airlines

We’ve talked a lot about how ensuring quality is a big challenge for complex SOA and enterprise software environments. But there is a bigger-picture risk that can drive any project past deadline and over budget: constrained and costly access to necessary systems and data throughout the development and testing lifecycle. This fifth example is about a major airline.

Like many companies, they faced constraints in their testing environment. In the airline industry, this is magnified, as all carriers have to make use of third party reservation systems. The billing for this access runs just like cell phone billing (but on a much bigger pricing scale!), with a set limit, followed by overage charges. Anything over the limit can get very expensive. In addition, the access itself is not cheap to begin with.

This airline wanted to expand their testing lab environments from four to nine. However, given the transaction fees I just described, and the associated hardware cots, this change would run into millions of dollars of extra costs on an ongoing annual basis. In addition to the extra transaction costs, the airline would have to purchase more hardware. Since you cannot just deal with airline and customer data over the public web for very real security purposes, they needed to provision special gateways and data circuits to interact with the third party source.

However, this was just the beginning of the problems they faced. Travel data is very complex and changing. There are weather delays, fare changes, rewards programs, partner systems, flight operation systems, and so on. To build applications, you need access to stable customer and flight data scenarios for testing and validation.

The airline found that when they ran regression tests, about 10% of the test cases failed. The failures were not mostly due to bad software, but data volatility. They had to find a way overcome a huge number of false negatives due to bad test data. Teams would often waste half their day chasing down these errors. Increasing the number of test labs by over 100% would only compound these data troubles.

Introducing a strategy that covered both automated testing and Service Virtualization addressed these issues, so we brought LISA to bear and demonstrated how to virtualize these kinds of services that are prohibitively costly and complex to replicate using conventional means.

First, they eliminated the need to incur almost all the transaction costs to the third party system by accurately simulating the required data as virtual services for development and test purposes.

This also took the additional hardware costs for gateways and data circuits out of the equation by reducing infrastructure needs and virtualizing any new infrastructure requirements. These hard dollar cost savings more than covered their investment in automated testing.

There was more, as the cycle of false negatives due to bad test data was broken, as stable test data could be provided. This may prove to be the largest source of saving overall. Now application development could proceed at a much faster clip, helping the airline generate customer benefits and revenue from the new application much quicker, and reducing development costs.

They are also planning to make use of the automated testing system and the data it generates to streamline their training programs and make them accessible on a local basis, reducing the travel expenses associated with getting their employees flying on the new systems they develop.

You can find the complete case study here.

SOA Success Stories Part Four: Global High-Tech Manufacturer

Here's another tale in our series of field stories to show successful cases of ensuring quality and agility for dynamic enterprise apps. This case covers how a global high-tech manufacturer tested and delivered a demanding enterprise e-commerce site under time and budget.

In this case there was a lack of visibility and traceability that was posing a serious threat to mission critical business operations. Unlike some of our other cases, this company had complete control over the design and implementation of this complex multi-tier app, yet they still had no way to trace a single action to a single source in the maze of multiple service layers.

Ruston Vickers, iTKO's CTO & co-founder, worked on this effort. He said that before the test automation project, the company had to do manual sequential tests on each subcomponent to find error sources: "It was like having to place a plumber at every faucet to find the source of stoppage in a house with hundreds of rooms. Instead of having the plumbers yell back and forth across the large house as they progressed through the system, turning on faucets one step at a time, LISA gave the company an x-ray machine to see a top-down view of the entire system at once and determine the cause and effect."

This top-down viewing capability cut their best prior testing time in half. Formerly the company employed 10-15 consultants just to continually test only the end result UI behaviors of their system. Now, with a highly reusable and complete test suite in place through iTKO LISA, the consulting firm can conduct the same process with one part-time resource, at a higher level of accuracy, as the automation capabilities of LISA erase the margin for error inherent in manual testing.

The company is pleased with the reduced costs and increased speed of implementation, and the consultant on the project is pleased with the increased value they can provide to their client. With LISA they both have a reusable asset that will continue to provide value as they go forward together with new releases.

Reusable, continuous validation activities save hard dollars in testing labor costs, and the number of support costs incurred, but cost savings aren't the only value. In a customer-facing environment, eliminating 1-2 months of time to successful release can make a huge impact on incoming revenue potential.

Stories from the Field - Part Three: Major European Telco Company

This is the third installment of an open-ended series of enterprise quality & virtualization success stories from our field engineers. It covers how a major European Telco company ensured BPM quality with test validation across all application layers.

"This was a classic case of being caught in the middle with moving targets all around," reports Ruston Vickers, iTKO's CTO. The telco needed to be able to validate downstream B2B integration after taking an upstream provisioning request, processing it as the middleman, and passing it on. The problem was that they had no control over the upstream and downstream services. "Handling this validation issue by calling on the real up and downstream processes was not feasible, and mocking them up manually required too much work and left you trying to model and hit simultaneously moving targets," Ruston said. This made it almost impossible to track down the source of many bugs.

Ruston's team solved this mission impossible problem by providing the capability to accurately simulate both ends, the middle, and the entire process in real time. LISA can first act as the upstream requester, launching the process and then it following it through the multiple service layers to completion. This is done through invocation, validation, and where services are designed but not yet built, the addition of Service Virtualization to simulate the needed services to complete validation.

The actual live upstream and downstream services are then no longer required for testing and validation, freeing the middle level services to rapidly test independently, also avoiding any third party transaction costs or data management issues. The result of implementing iTKO LISA's Validate and Virtualize products was transformational for this telco, and included the functional validation of 120 different order types, as well as load testing of TIBCO BusinessWorks processes and database stored procedures.

Higher quality was achieved through near 100% test coverage to validate end-to-end business process outcomes. In addition, there was faster time-to-market by removing system availability constraints and reduced transaction costs. You can read the complete case here. Our next case covers how a global high-tech manufacturer tested and delivered a demanding enterprise e-commerce site under time and budget.

iTKO & Cognizant Talk about Software AG Quality - Webinar

We're having an iTKO-hosted webinar Wednesday on a topic we revisit quite a bit, ensuring quality and agility for Software AG implementations. Our Software AG customers have very unique needs - from building and validating next-generation SOA and BPM architectures, to regression testing and virtualizing test labs while conducting an upgrade of their infrastructure to the latest version of webMethods.

iTKO's own Founder & Chief Geek John Michelsen will be joined by an expert from one of our key SI partners from Cognizant, Alfredo P. Reyes, the EAI Practice Head - North America, who will talk about best practice methodologies. This should lead to a very good discussion, so bring your questions.

Registration is free here:
"iTKO Webinar: Reducing SoftwareAG Implementation Costs and Risks with Collaborative Quality Across the Lifecycle"
Wednesday, February 25, 2:00PM Eastern

According to the webinar invitation, "Enterprise software implementations today are up against extremely difficult cost and time constraints. The need for greater cost savings, risk mitigation, quality, and end-to-end control for SoftwareAG/webMethods implementations is driving more customers to evaluate new solutions to optimize their software testing and delivery lifecycle."

Field Reports on SOA Success - Part One: National Cable Provider

Recently, there has been a lot of fun discussion about the death of SOA. Well SOA successes are quietly happening every day, and one of the keys is the use of robust testing and validation and, some cases, the use of service virtualization to speed up the testing and validation process.

SOA's shine may have worn off for some, but it is just another practice for designing and building dynamic, distributed enterprise apps. SOA may share boundaries with other technology practices: BPM, integration suites, and ESBs. All of these describe systems that are heterogeneous, distributed, multi-tier, and subject to frequent change by nature.

So call it what you will, not meeting the new challenges raised by this class of apps can lead to various unintended consequences and failures, or create bottlenecks to agility and cost savings which were the ultimate goal. We want to add to the conversation with a series of actual examples from the field to show successful cases of dynamic enterprise apps. This first one is a national cable provider.

This company had a very complex provisioning system - literally the "brains" that enables or disables what programming subscribers can see on their TV, depending upon requested configuration, on-demand status and special offers. The application was a complex application with several components that needed to interact at runtime in a secure transaction environment, and they had to operate efficiently and accurately to maintain customer satisfaction.

Our iTKO architect involved in this effort, Ken, said that before he and our integration partners at pureIntegration arrived, the cable provider staffers were hand building manual tests to try to cover these complex processes. Even a simple change could take the staff three weeks to build the less than adequate manual tests. Just getting ready to attempt to test a major release could take them several months, so the cable company was limited to only two major releases a year. This put them at a competitive disadvantage and added unnecessary expense to the testing and validation process.

Moreover, the QA manager noticed that manually testing was like trying to do deep sea exploration from a glass bottom boat. Gaining end-to-end test coverage and control, with automation, enabled them to move out of their testing silos and see the entire process. Now they can quickly spot error sources and correct them.

Ken said that in one case dozens of reported bugs were found to result from a single change. Getting past the 80%-20% rule and covering the most important and likely risk areas with automated testing first enabled them to spot these dependency failures in almost real time with each change. In the past, it was taking over three weeks to isolate the problem. Ken said that LISA was employed, enabling them to increase test automation and validation coverage from 15 – 20% to over 95%.

Ken added that “Saving several weeks in each test process has created a lot of excitement.” The results included the reduction of regression test execution from 3 weeks to 5-8 hours. The major release cycle was reduced by 6 weeks. The integration of LISA with the test and validation process also saved more than 95% of per-test costs, which is also an incentive to increase the test coverage to new features.

To read the entire case, go to iTKO LISA Case Study: Major Cable Service Provider.

Our next installment of this series will cover how a leading financial institution eliminated a cumbersome and expensive manual testing process, freeing up resources and dramatically decreasing implementation time.

Can You Lend Me A Shovel So I Can Start My SOA?

Joe McKendrick (aka, Joe the SOA guy), put on his economist hat and recently commented on stimulus ideas for the downturned economy in his post, Shovel-ready SOA. He writes, "perhaps we should look at service orienting as a shovel-ready 'stimulus' program for corporate growth. Because it works both ways — the current rough-and-tumble economy may be good for SOA, and SOA may be good for the economy."

Good idea Joe. As we wrote earlier, on one hand SOA is not dead or going away. We see success stories quietly happening all the time. On the other hand, like the shovel, SOA isn't glamorous anymore, but it's being used to create real value. Not only that, but we recently were musing on how SOA virtualization can help cut carbon emissions while it is saving money, hitting another aspect of the current stimulus suggestions.

Joe takes this discussion of SOA and the economy a step further as he comments that it can also help in "revitalizing business growth as well ... Consider the advantages properly implemented SOA can deliver: Increase confidence through greater transparency of applications and data, especially at a time when markets have been quaking from fear of what they don’t know; help businesses collaborate better through more rapid integration… help organizations expand or contract pieces of the business, as needed, without fuss and muss."

In tough times, nothing boosts your chances to survive, and be positioned to thrive, like enhanced confidence, collaboration, and agility through SOA efforts. We'll just have to roll up our sleeves.

Of course, before you start digging, a little upfront architecture and planning goes a long way -- you don't want to hit a wire or gas line! OK, enough, we'll put away the shovel metaphor for now. Each of these themes can be further enabled by the confidence, speed, and agility that automated testing, ongoing validation and Service Virtualization can bring to SOA.

EuroSTAR 2009 Conference will be in Stockholm

I recently received the first issue of the STAR Tester Newsletter for 2009. We're still a long way off, but they are already accepting presentations. This year’s conference theme is: ‘Testing for Real, Testing for Now’. The EuroSTAR 2009 Conference will be in Stockholm, 30th November – 3rd December 2009.

This is a great annual European event, we enjoyed participating last year, and iTKO's own John Michelsen spoke on SOA Testing, Validation & Virtualization. Here are two extensive reviews of the 2008 version that occurred in den Haag, Thoughts About EuroStar 2008 by Randy Rice and Notes from EuroSTAR 2008 by Rikard Edgren. Randy wrote that “One of my tests of a good conference is whether I'm torn between attending track sessions - and I was.” The theme of that conference was "The Future of Software Testing.”

There is a EuroSTAR community blog to support the event. The blog says that it is “...dedicated to Software Testers worldwide. We hope that this blog will allow testers to share their individual EuroSTAR experiences and also give testers new to EuroSTAR, a glimpse of what the event is all about. This forum should also act as a place where testers can interact, discuss issues important to them and debate the most up to date trends within Software Testing.”  I am sure activity will pick up there as we get closer to the conference. 

Prioritizing Continuous Validation by Criticality: Private & Federal Sector

We just had a neat little brainstorm here, with some of our smart Solution Architects in town, on the criticality of Validation in Business vs. Federal Govt. So, I wanted to repeat some of the riff here and invite commentary.

In essence, organizations have key processes or mission threads that simply cannot fail, because failure is catastrophic to the bottom line. For instance, in the commercial space, the biggest asset is the customer. Companies place heavy emphasis on customer retention, so any process that directly touches the customer cannot fail. Failure may cause that customer to walk to a competitor. This is why many of iTKO’s customers utilize LISA's continuous validation service (CVS), to continuously validate that key customer-facing processes are behaving correctly.

Now say for instance with an agency like TSA, there are no customers, per se. However, you could consider any traveler a customer of TSA services "i.e. Keep us safe," but there are key processes/mission threads that cannot fail (i.e. Letting a terrorist board an aircraft).

The systems that support keeping terrorists off an airplane need to be continuously validated. Why? - Unexpected consequences. There are probably many other examples of critical mission threads like this in defense, where lives are at stake when processes fail.

Bear in mind, Continuous Validation does not solely get tied to “mission critical” processes, that is just where the most value is gained. Perhaps I want to continuously validate payroll processing or more back-office processes because they are important too, just not as much as keeping a terrorist off a plane.

Some validation processes are tied to physical activities -- mock crisis scenarios and fire drills. However, I don’t want my continuous test to cause a SWAT team to be deployed to the airport every time I validate my process or a plane to be stopped on the tarmac. Continuous Validation should be the "what if" that tells me about unintended consequences, without interfering with my day-to-day operations.

We are familiar with prioritizing Testing around criticality, since we don't have time to test everything, all the time. Maybe you have some similar thoughts about how Validation is prioritized according to real-world criticality?

Avoiding SOA Disillusionment - eBizQ Panel Recap

Joe McKendrick recently led an eBizQ panel on Avoiding SOA Disillusionment. Participants included Chris Harding, The Open Group; Phil Wainwright, eBizQ and ZDNet; Jignesh Shah, SoftwareAG; David Bressler, Progress Software; and myself, Chris Kraus, Product Manager, ITKO LISA.

The panel audio is available at this link to the eBizQ panel on Avoiding SOA Disillusionment. You have to register with eBizQ if you have not done so but it is quick and free. Joe put the term “disillusionment” in quotes, as this is a loaded term.  While SOA has shifted the burden from big-bang vendor solutions to internal IT alignment, the recent comments by analysts in this direction. The panelists I joined have not seen disillusionment, but they do see that SOA is no longer seen as the most exciting technology anymore. 

SOA has ridden out the hype curve and people are now settling into implementations and dealing with the new challenges SOA raises in realistic ways.  The implementation process is now more complex for IT because you are dealing with multiple services and not a single vendor you can beat up on.

Several panelists covered how SOA is best presented as a business solution and not an IT solution.  Chris Harding added that it is important to see how are consumers going to use these services now and how they can be reused by others in the long term. This is especially true, as conditions within organizations are constantly changing.

David said that project-oriented IT departments can be counter to achieving a long-term vision as they are rewarded on short-term performance. SOA can highlight some issues within IT departments that may be counter to long-term business success.  Perhaps SOA may require a cultural change within many IT departments.

There is much more to hear on the panel. I think the issues of “disillusionment” are the result of poor implementation strategies, lack of discipline around quality, and/or the use of out of date methods. These issues are not necessarily embedded within SOA. However, it does require new ways of doing things to be successful. The panel covered many of these changes such as Agile development and better use of virtualization to make resources available during the lifecycle. 

The panelists seem to feel that the down economy will help SOA as it is a good way to force companies to become more nimble and it brings cost savings. However, success will depend on how each company responds to the crisis. Do they hunker down, or do they look for innovative ways to survive and grow?

More on SOA Decline with Gartner's Frank Kenney

I recently discussed the Gartner report that claimed SOA was in decline – see Is SOA Really on the Down Swing? Say it ain’t so Joe. I came across this humorous take on the matter from a Gartner analyst we have had the pleasure of talking to many times, Frank Kenney. He writes, Ahh Shucks, SOA Is A Failure. In this post, Frank offers a memo to be sent to senior management by someone leading a SOA failure.

The memo begins with “under my direction... I have failed to associate our SOA initiatives with our business needs, therefore I cannot show any value for the hundreds of services we have created, I have failed to properly create and support an SOA Center of Excellence, Steering Committee or Competency Center, I have failed to enlist the executive staff as true supporters and evangelists for our SOA efforts...” It goes on and lists all the things you should do to make SOA successful that this person did not implement. It concludes, “Thanks for understanding and I’d like to declare in advance that Cloud Computing, Virtualization and SaaS will be failures under my direction as well.”

So who wants to sign off on killing SOA in their company?

SOA does require some new approaches, as well as some sound traditional business practices. It is not plug & play. If this approach is taken it will be more plug and pray. We are seeing SOA adoption holding strong but it needs to follow the success factors of governance: policy, management and validation-related -- that we have discussed at length here -- or stakeholders will be disappointed. Thanks to Frank for bringing some humor and good business sense to the debate.

Is SOA Really on the Down Swing? Say it ain't so Joe.

When I read about the recent Gartner survey that claimed that SOA implementations and enthusiasm were declining, I was skeptical. I also thought that I could count on Joe McKendrick to write something thoughtful that challenged these assumptions, but also acknowledged things that SOA supporters might need to do in these hard economic times.

I was right on the second count as Joe wrote the post, SOA’s ‘decline’ may be overstated, but companies still want results. Looking past the headline at the actual research reported by Gartner, I choose to interpret the results a little differently, just as Joe did. First here is what Gartner reported in their report summary, Gartner Says the Number of Organizations Planning to Adopt SOA for the First Time Is Falling Dramatically.

According to their survey, “53 percent of the respondents were already using SOA in some part of their organizations. Another 25 percent were not using it but had plans to do so in the next 12 months; and 16 percent had no plans to use SOA at all... Since the beginning of 2008, there has been a dramatic fall in the number of organizations that are planning to adopt SOA for the first time. In 2008, this was cut by more than one-half, down to 25 percent from 53 percent in 2007, while the number of organizations with no plans to adopt SOA more than doubled from 6 percent in 2007 to 16 percent in 2008.”

This seems more like market saturation, with 78% of firms either using SOA or planning to. The drop in first time adopters might be, in part, because of the shrinking pool of those who have not adopted it. Joe makes a similar point in his post mentioned above. I will look more closely at his post, but first more from Gartner.

The report summary went on to quote Daniel Sholler, research vice president at Gartner - “Organizations without a clear business case for SOA and without a plan to develop or acquire the necessary skills are justified in taking a cautious approach, and delaying SOA adoption plans for the coming year.” This sounds like a good strategy for any technology initiative. I do not see this as a criticism of SOA but a caution to make sure that SOA implementations are launched with clear business objectives.

The Gartner work does not represent the only polls, Joe counters with some work of his own. He writes, “...based on a survey conducted in September and October (when the financial crisis was hot and heavy), we that found all systems were still ‘go’ for SOA efforts — but with plenty of continuing reservations about ROI. Fifty-two percent had SOA-based projects underway, and another 31% were planning to start SOA projects over the next 12 months.”

Joe then adds that IDC’s Sandy Rogers finds in her polling that SOA adoption has deepened across many enterprises. He quotes her. “The issue is not necessarily deciding if they should go toward SOA. What we’re finding is that for most organizations this is the way that they are going to move, and the question is just navigating how to best do that for the best value and for better success.” This sounds like good caution in down times and not a rejection of SOA.

A snapshot in time can be deceiving, as we have seen recently in other areas. SOA isn't so sexy anymore. There will be lots of pundits calling SOA dead, and maybe it will get a new name, or merge into BPM or WOA as we've discussed. Either way it is still quietly going on and providing value in most leading global firms.

We are actually finding continued interest in SOA. It is a way to cut costs among other things so this should be appealing to many firms in down times. At the same time, we would be the first to agree that there should be a sound business objective and a careful implementation that includes the comprehensive and continuous testing necessary to ensure success.

For SOA Quality - there's just no free beer (Live from Gartner AADI)

Just exited a breakout session from Gartner's Tom Murphy, who is one of the main analysts covering the Testing side of our marketspace, as it edges into Agile development and ALM. His talk, "Application Testing and Quality in the Age of SOA" revisited a theme we haven't touched on in a while here: that in order to motivate and ensure good Reuse, you need Compensation.

Look at it this way: If I am a producer of a service - unless I'm a ramen-eating college student who just likes producing services for the glory of making the next cool widget, I am going to need some motivation to make that service as readily consumable and usable by my end customers as possible. So it falls to the Producer of a service to build quality into their service from the beginning.

Tom mentioned that SOA "offers a platform with a shifting foundation" so a Center of Excellence around SOA testing and quality is needed. The reward for good quality (and useful design as well) is Reuse, and that is compensated with real or "company store" budgetary dollars.

In SOA, quality is not as granular as checking the code for structural bugs or clicking on a UI to test at the end. It requires discipline and specialized tools that directly invoke and verify the services so they meet defined policies - structural, behavioral, performance, and that this process is carried out continuously throughout development and into runtime. That's a lot of responsibility if you have a well-used or critical service.

The part we often forget is the responsibility of the Consumer in this equation - If I am consuming a service, and its underlying architecture, there is no "free beer" here. I should be expected to reward the Producer that makes a service that has the most utility to my business. And in addition, I need to declare exactly how I am intending to use the service so I am not going to impact the promised service levels of other Consumers.

Check out this post from a couple years ago - "Consumption is not Free" is still one of our favorite themes and it's good to hear it again here at Gartner AADI.

Gartner AADI Summit 2008

We're here at the Gartner Application Architecture, Development & Integration Summit (AADI) will in Las Vegas on December 8 – 10. The subtitle is: SOA & Modernization: Beyond Business & IT Alignment.

The event site says, “Service-oriented architecture (SOA) is a reality no organization can afford to ignore. It fundamentally reshapes how you think, build and deliver applications. But SOA is not pursued in a vacuum, it needs to be part of a continuous IT modernization strategy that seeks to optimize your resources, and dynamically align with the rest of your business.”

We're looking forward to some of the discussions we'll have on SOA as it relates to bringing applications into the modern world, and will post some of our thoughts from the show here. iTKO will have a booth. iTKO's Chief Architect/founder John Michelsen will also be here, so come visit us at Booth M and schedule a talk if you are attending.

There are five new tracks focused on the 3 SOA maturity levels, Modernization, and Next Generation AD, and we are certain to hear more about Cloud Computing as well - we certainly wouldn't mind someone putting a stake in the ground on that definition.

There will be a special keynote by author and trendspotter Malcolm Gladwell, and a complimentary copy of his book Outliers will be provided to all keynote attendees.

But first, where's the coffee...

Agile SOA Across the Lifecycle - Part Six: Conclusion and Index of Links

This is the last of a six part series of posts on the Agile SOA life cycle. Here we will provide our conclusions. As the development lifecycle has changed from traditional waterfall development and testing, to agile, spiral and scrum models supported by ALM, the need for continuous testing throughout the lifecycle has become self-evident. In order to evolve these processes, and achieve the business responsiveness and cost benefits of SOA, there is a need for vastly improved processes in how we build, implement, and test systems.

This series only covers some broad brushes of how Lifecycle Quality can be embedded into your testing, ALM and Governance processes. We see customers using these same capabilities in new ways every day – for instance, using LISA Virtualize to create virtual service environments, Virtualization of the behavior of services, mainframes and data enables teams to get up to speed in managing and testing against a realistic environment that would be too costly to reproduce otherwise. This increases agility by moving teams into more parallel development cycles, through eliminating the constraints of services and components that are not available for testing.

Real cost savings are available here and this should resonate well with companies in today’s tight economy as we have discussed in recent posts, We also agree with Jeff Schneider’s post, Talking to the Business about SOA, that it is okay to talk to the business about these benefits. Most of those who commented on his post agreed, as well.  We are seeing strong business interest continue in these tough times.

Companies have invested significant cost and effort in establishing solid processes with market leading tools – and we need to bake quality and enforcement capabilities into these tools, without dictating the process. iTKO LISA’s automated testing, validation and service virtualization works alongside test management, ALM, IT Operations and SOA governance platforms to provide an exponential gain in application quality, development agility, and most importantly, trust across producers and consumers of today’s heterogeneous, interconnected applications.

This concludes our six part series on Agile SOA Across the Lifecycle with LISA. You can download this complete series at our ITKO LISA resources page. Below are links to the individual posts as they appeared on this blog. 

Agile SOA Across the Lifecycle with LISA - Part One: Introduction

Agile SOA Across the Lifecycle with LISA - Part Two: Test & Quality Management

Agile SOA Across the Lifecycle with LISA - Part Three: Application Lifecycle Management

Agile SOA Across the Lifecycle - Part Four: IT Operations, Monitoring & Performance

Agile SOA Across the Lifecycle - Part Five: IT and SOA Governance

Agile SOA Across the Lifecycle - Part Five: IT and SOA Governance

This is the fifth of a six part series of posts on the Agile SOA life cycle. Here we will at look at IT and SOA Governance.

With the introduction of agile, spiral, and scrum development methodologies, the traditional waterfall development approach of testing a near-finished app at the end of many Agile development cycles won't be agile at all, as the elements of the application are constantly changing.  Traditional models of IT governance will also not work.

To aggravate testing, the service-oriented architecture (SOA) design pattern is used to make IT  more responsive to changes requested by business. New process tooling has been introduced to specifically assist in the cataloging of service assets, and organization of policies governing SOA.

This new set of tooling created to support SOA revolves around governance platforms like HP Systinet / S2, SAG Centrasite, SOA Software, TIBCO ActiveMatrix and Oracle Fusion. These SOA Governance tools manage the collection and cataloging of metadata about services, and organize the interdependencies they have with each other, along with the documentation of SOA Policies to define how combined services should meet business requirements. 

Within the Governance toolset, SOA Registry/Repositories provide a new platform for LISA to automate testing and validation.  The traditional system requirement and functional testing will still occur, but in a SOA there are more opportunities to automate the validation and enforcement of policies.

The current thinking around SOA Governance is largely siloed and specialized around design-time WSDL validation, runtime performance, and security policy enforcement.  As SOA Governance matures to support robust and widely diverse SOA initiatives, this thinking must expand dramatically. One of the greatest values a SOA Governance platform must provide is the proof that policies written in human or business terms are in fact implemented in the system, on a continuous basis.

Validation with our LISA solution takes the form of positive and negative tests on the use of these policies, which are executed continuously at change and runtime, as services and their underlying technologies and data are constantly changing and evolving. The validation of the LISA test becomes a demonstrable artifact for exposing and enforcing SOA Policy, within the changing workflow of the SOA Governance framework.

Example Scenario of Validating SOA Policies

Let's look at a sample deployment, a series of web service definitions, as WSDL files, are loaded into the UDDI registry.  The relationships of the services to each other, and the documentation of how they can be tied together to create a business process is documented.  For a new employee introduction, there may be a series of services from the human resources package to identify the employee's location and managers, there are services that provision IT resources like computer and e-mail, and services to a external partner for insurance and benefits enrollment.  These services would be documented via their WSDLs and stored in the UDDI registry.  A HR-to-payroll business process will be defined utilizing these assets.

This all seems simple as long as each participant in the business process provides a well documented, functioning, reliable, and secure service.  The policies around the structural, behavior, and performance of these individual services are documented the governance tool.  For example, the business analyst will write policies in a tool like CentraSite ActiveSOA, describing the behavior of the services. 

The IT service will accept a new employee's name and id number and return an e-mail address, domain login, and default password.  The security service will accept the manager and role and provision the appropriate access control to systems for the new employee. The security manager will specify that the services must only run if encrypted and signed data is sent to them.  The signature must be validated against the companies master certificate provider to prove that the request is from only authorized HR personnel.  Lastly, the IT operations team will provide policies around how many transactions per second (TPS) the service should handle, and the maximum response time the service will take, so the component will be a good participant in a decomposed transaction.

That's far too many steps to carry out if it's a strictly manual validation process. The execution of the tests needs to live within the SOA Governance process.

Testing, Validation and Policy Enforcement Solutions

Given all of the processes enterprises must employ to manage the design, construction, integration and management of the IT environment, it makes sense to have a common, reusable way to apply validation and enforcement across these processes. SOA Validation becomes “the long arm of the law” with reusable and rich testing that merges into the workflow of each of these processes, and the tools that support them.

The Validation arm shown above (with our LISA as the "long arm") provides capabilities to run test cases to verify different rules and policies defined in leading IT process tools. The policing action is accomplished by tying automatic running of tests to the expected behaviors and policies in the workflow of the process tool.  The invocation of  tests from a governance platform to ensure SOA policy is similar to the policing action done for test management solutions, the biggest difference is the context and stage in the lifecycle of the component that needs to be verified. 

In a traditional test of a waterfall development there is a specific point in time in which the system is tested and deemed ready for production.  In today’s services-based applications, there is no longer one point in time in which we can identify a test run to ensure quality of the application.

In a SOA lifecycle, we think of design time, run time, and change time as the three stages of a service or business process being constructed within these loosely coupled systems.  In the example above some of the services are interactions with packaged applications (HR system), some from home grown systems (IT user and security provisioning), and other from external partners (benefits).  The development and release cycles of these three areas will be different and will not be coordinated for a single big bang release.  One advantage of SOA is the ability to decouple systems and utilize services created by different parties instead of building everything yourself.

At design time, a test will be run to make sure that WSDLs are WS-I compliant, that they follow an RPC or doc-literal call structure, and that a customer-specific XML tag is used for service identification.  Based the on passing of a compliance test, the WSDL will then be made available in the Repository for developers to code the functionality behind it. 

Once the developers have created the necessary code to implement the service and its underlying technology components, the behavior and security of the service needs to be validated.  The only way to verify if the code behind a service is working properly, since there are no hard-coded responses in the real world, and that security is respected, is to invoke the service, then verify that the underlying systems of record are updated properly.

Again, a test case is used to invoke and verify that the business requirements and the behavioral and security policies have been implemented. If the candidate Service can pass these Policy tests, it can now be published in the registry for a consumer to use.  If these tests fail, the service should not be promoted for availability to consumers in the registry.

As a best practice, the individual services should undergo load testing once they are made available. Far less risk is introduced if we test for load while the components are being built, when something can be done to improve performance issues, rather than waiting until the system is deployed, when issues are much more costly to repair. Progressive development shops start their load testing once the service functionality has been verified, so that developers can make code changes, and optimize logic and data access before consumers start utilizing the service.

In run time, the service is now promoted to the Registry, along with a test that demonstrates its functionality, and made available for consumers to leverage as part of their own application workflows. These consumers also bear responsibility for validating that their intended use of the service is actually supportable. The best way to do this is to create an executable test asset to validate that the structural, behavioral and performance aspects of the overall consumer’s workflow are supported when the service is leveraged. This will pay big dividends when the test is run continuously as part of the governance process.

The final stage is change time. This is when the true advantages of SOA and agile development come into play. If a new service needs to be added, or a modification to a existing service needs to be made to respond to changing business requirements, the policies for the service must be re-verified.  Not only do we want to update any behavioral policy with the new expected behavior, but the regression of existing policies must occur.

Put yourself in the place of the consumer of a service. How can you trust that changes will not create unintended consequences (e.g. failures) for a service you depend upon, and break your existing business processes?  Each consumer needs to place their expected behavior as a policy in the registry, with an accompanying test, so that it can be automatically validated before that change is made.  The automated structural and behavior policies in the test represent the consumer’s rights – and their responsibility for using the services as defined. When these tests run automatically as part of SOA Governance, trust between service consumers and producers is achieved. For more on SOA governance see Joe McKenrick’s posting of the full transcript of a recent eBizQ SOA governance panel that John Michelsen participated in.

In our next post, we will provide our conclusions. You can download this complete series as the "Agile SOA Across the Lifecycle" at our ITKO LISA resources page. 

Finding the Blind Spots in SOA Testing

Colleen Frye recently wrote about the consequences of overlooking SOA testing blind spots at SearchSoftwareQuality.com. She reported on an interview with ITKO’s John Michelsen, and Crosscheck's Rizwan Mallal, Green Hat's Madeline Bayliss, and Scott Barber, chief technologist of testing services at PerfTestPlus.

John said that unintended consequences result from failing to test the dependencies in SOA applications. The solution is not to retest all the services that have already been individually tested, but to conduct a test of the business process in context. Then you have to do continuous validation at all phases to guard against the effects of changes to a service.

Another potential blond spot mentioned in the article is the reliance on HTTP return codes to determine the success of an operation. You need to perform deeper content inspection. Another blind spot can occur when testers ramp up load by simply sending duplicate WS-Security-enabled message. Scott said that testers need to be careful to test all kinds of potential malformed data input and this is often overlooked. He added that testers need to look more carefully when third party services are in use.

John concluded that while quality still belongs in QA, the development team and the IT governance group also have responsibility. It was a rich conversation and there is more at Colleen’s post.

Application Development in the Clouds - Mike Vizard and John Michelsen

Mike Vizard at eWeek recently hosted a podcast with ITKO’s John Michelsen, on "Application Development in the Clouds." In this conversation, John and Mike talk about how cloud computing is going to upend the way IT organizations think about SOA and application development in general.

Mike started by mentioning the increased number of dependencies that SaaS brings to SOA testing. John said Mike really nailed the problem - it is something we are seeing in every engagement that relies on third party or shared services environments. The traditional waterfall method, and its assumptions, are now all undermined, and so much rethinking needs to be done. But the effort is worth it.
Mike, as seen on the left, next discussed the increased reuse happening in SOA. This broadens the need for trust. John, as seen below on the right, said that SOA requires good governance policies. In most organizations vertical trust already exists, as teams within groups and authority domains inherently trust each other.

Now with the need to go across the enterprise, and even to third parties and partners, horizontal trust comes into play and this is usually not in place. Service level governance can handle this issue and there are good frameworks available (in the form of SOA Registry/Repository solutions like Centrasite and Systinet). However, each organization will have to customize these frameworks for their unique environment, and think about how those teams can have a common way to validate that expectations are being met.
 
John added that SOA Governance needs to extend to the consumption of services, and this is where most organizations fall short. There is "no free beer" in consuming services - so if you have an intended workflow, that also needs to be validated. After all, is the business use case realistic, and how will the provider of services be compensated?

Mike asked about how these services be validated on a continuous basis, as the services may be constantly upgraded after deployment. John replied that LISA includes continuous validation since many applications allow for ongoing modifications. You need to test for the "unintended consequences" of these modifications.

Mike asked the need for visibility into services. John responded that this need is at the Behavior level to see if the desired behavior remains the same after changes in the application. This is where continuous validation can generate trust and allow for SOA in business critical functions.

How much time needs to devoted to validation, versus building applications? If there is too much time required for validation, developer productivity falls down. John said that this is where automated testing can play a key role and it needs to address dependencies.

Mike asked how developers can be encouraged to adopt this new thinking around continuous testing. Will a new generation of developers be required? John said that this approach is more evolutionary than revolutionary. However, most test organizations need to think in a new way.

Fortunately, there is greater awareness of this need than was in place a few years ago, when much of the SOA Governance conversation was only around Structural testing, for WSDL standards, WS-Security, WS-I etc., which are very important, but just one piece of the puzzle. The idea of design, change and runtime policies that incorporate behavioral and performance Policies and SLAs are now in focus. 

I have touched on the highlights. Go to the complete podcast for the details.

SOA in Action Virtual Conference Preview - Panel tomorrow

I recently had a chance to talk with Peter Schoof at eBizQ about the upcoming SOA in Action Virtual Conference (free signup here) on November 19. You can listen to our discussion and/or read the transcript at this post, The Acceleration of SOA: iTKO Explains.

I mentioned here we had a very productive SOA governance conference last month with John Michelsen sitting on that panel – see SOA Governance Panel Recap: Federated SOA for the Day. We had a lot of good questions as a result but they're mostly around how do I manage or architect my services.

We are part of a Roundtable session on "Disillusionment in SOA" that should both support and debunk a few rumors about the real issues companies are struggling with as SOA acheives more adoption.

I imagine the discussion might go a lot more to the brass tacks like: How do I actually test multiple layers of SOA, how do I orchestrate these services, what constitutes a Policy, and how am I actually validating that that policy is being met as the services are constantly changing?

The roundtable closes out the session at 2:00PM. iTKO's Chris Kraus (LISA Product Manager) will be on the panel with Joe McKendrick, Phil Wainewright, Dr. Chris Harding, Jignesh Shah, David Bressler.  Keynote speakers throughout the day include Yefim Natis, Gartner Research; Maneesh Joshi, Oracle; and Randy Heffner, Forrester Research.  

Please join us - no travel needed. Thanks to Immo Hüneke for mentioning it. If you are walking the "virtual floor" come by before or after the roundtable, and ask us a question or grab a virtual tchochke.

Mike Kavis Talks with John Michelsen about Successful SOA Testing

Mike Kavis writes the blog, Enterprise Architecture and Other Enterprise Topics. Mike is currently CTO/Chief Architect at technology startup and has his own consulting company called Kavis Technology Consulting where he provides advice, training, and professional services on key technologies. He is also known as the "Mad Greek" in blogger and Twitter circles.      

 Mike recently talked with ITKO’s John Michelsen about questions to consider before building a SOA testing team. His questions came from a recent post he did on CIO.com, Six Questions to Consider Before Building a SOA Testing Team. Mike’s first asked John to identify some of the characteristics of successful SOA implementations from his customer base. John said that integrating testing into the entire development lifecycle is the key. Successful SOA implementations stay away from old waterfall methodologies and involve QA at the earliest opportunity. In many cases, testers work side by side with developers.  

Mike referred to another of his posts, SOA Skillsets: Part 4 - The Testers, that provides a similar viewpoint. He recommends that the Test Architect become a member of the EA team. In this post, he goes on to describe the skill sets for SOA Test Architect, SOA testing leads, business process testers, integration testers, user interface testers, and data service testers. In their conversation John stated that “most QA folks are still business analyst types and it should stay that way. If QA gets too technical then they lose the focus on testing.” Mike agreed and John added that this sets up the requirement that SOA testing tools should let less technical people perform technical tasks. 

John then went on and described LISA Continuous Validation Service and Virtual Service Environment which both accommodate the requirements they discussed. There is much more in Mike’s post and we appreciate him posting the conversation.

Agile SOA Across the Lifecycle with LISA - Part Three: Application Lifecycle Management

This is the third of a six part series of posts on how SOA is becoming merged with the Agile life cycle. Here we will at look at Application Lifecycle Management (ALM). As Forrester’s Cary Schwaber writes, the newest ALM platforms will improve the development process by providing common services to practitioner tools.  This current generation of ALM software is doing even more to tie requirements, test cases, bug tracking and issue resolution together in one application suite. You can see a simple illustration of this cycle of sharing Tests as collaborative assets below.

The process of converting business requirements into software is being automated through ALM tools, by tracking completion of coding and storing source code, execution of test cases to ensure functionality, and documenting product bugs and enhancements for development to prioritize.  Automated testing has been traditionally plugged only into the “test execution” phase of ALM. This provided statistics on code coverage or relaying the success or failure of a build. However, stronger testing can add value at every stage of the lifecycle managed in ALM tools.

For instance, developers can check in tests alongside their source code in the ALM’s repository, as simplistic “happy path” tests that demonstrate the intended functionality of the delivered component. Then, QA teams can be alerted to these new assets, and run and iterate on those tests to prove out a larger set of possible activities. Still another team can use all the tests from Dev and QA in the process of integration and deployment, tying multiple tests together to ensure that the overall regression and functional coverage is high with less effort. Finally, as the business team and Support teams get involved, they can run, modify and check in tests to the platform to signify issues or define requirements for the next phases of the ALM lifecycle. When shared in this way, tests become collaborative assets for SOA, and increase the agility and speed of each iteration cycle, while better ensuring the success of end-to-end business processes.

This practice can be employed with any testing tool, whether you are coding JUnit or using a more automated, no-code testing tool. The advantage of using a no-code testing environment, that automates the execution of tests without requiring coding skills, is it allows non-developers to get involved in the cycle above. Of course, our LISA provides a very robust way for developers, QA/QE, business analyst and support participants to get involved in the Agile SOA lifecycle. LISA tests are easily stored and shared within ALM repositories as XML files, and they are executable from the command line or JUnit of a build script, as a double-click from the ALM interface, or automated as part of a check-in or validation cycle for each deliverable. The LISA test then passes back the results to fail the build or update the status in the ALM system. Integration of LISA to ALM and related SCM (Source Code Management) and RM (Requirements Management) tools from vendors such as MKS, Borland, Rational/IBM, Compuware, and others, allows tests to help teams achieve the predictable delivery timelines and quality they expect, while using they process tools they are most comfortable with.

As shown in the above example, everyone should be involved in contributing and executing tests as each iteration at the component level is introduced, and these tests are rolled up into a broader workflow and business process. An Agile organization allows both developers and non-developers, in multiple teams, to collaborate on contributing testing components into the application long before the end of development. This ensures continuous testing at each iteration cycle and a more reliable course correction and release process.

For instance an Insurance company may have a requirement to match the Zip code with a certain type of auto policy coverage. The developer creates a component and checks it in, along with a simple test case that shows a few matches of Zip and coverage. The QA team can then iterate on that test with a dataset of all needed Zip/coverage matches, and make that test a requirement for each successive build. If the developer later updates the component and the required LISA test doesn’t pass, it reports it to the ALM system with the root cause analysis, and the build is failed and communicated back to the relevant teams for correction, according to the issue resolution rules of the ALM system.

For more on ALM, we recently found a nice report at BusinessWeek’s technology Research Library on Application Life-Cycle Management.

In our next post, we will look at IT operations, monitoring and performance. You can download this complete series at our ITKO LISA resources page. 

Agile SOA Across the Lifecycle with LISA - Part Two: Test & Quality Management

This is the second of a six part series of posts on the Agile life cycle for SOA. Here we will at look at SOA test and quality management.

In the past, test “scripts” in TM tools were largely recorded as actual step-by-step instructions (in a Word doc or Excel spreadsheet) for a manual tester to point-and-click through a finished interface to test the application. When tests were completed, the tester would then “check a box” in the test management UI, to indicate that the test had passed or failed. With so many changing components and middle tier systems in today’s enterprise apps, that form of manual testing is no longer enough, so teams are trying to define (and run) far more sophisticated tests that can actually test behind the user interface, where the business logic lies.

In addition to deeper functional testing, teams using TM tools are asking for a higher degree of automation from their test suites. A thousand manual test scripts can delay cycles by days or weeks while they are executed, costing the team serious agility, and making the testing less valuable over time due to repetition and human error.

Therefore functional, regression and load/stress tests need to be stored directly within the TM tool – for instance as executable command line or XML scripts that launch a test from a ready server-side tool. In this way, a sophisticated set of tests can be executed with just one click in the TM tool, and the pass/fail and other results and issues are automatically written back into the TM tool.

In this way, the whole team gets to use the test process they are familiar with, while getting much deeper tests that can talk to middle tiers, and as much as a 95% improvement in test execution time compared to manual tests. In addition, quality is improved as the teams can focus on testing new functionality, instead of repeating the rote execution of tests against existing functionality.

We are asked for this kind of functionality at virtually every customer engagement - so I'll mention our solution here. iTKO LISA provides connectivity to Test Management (TM) tools, such as HP TestDirector for Quality Center (formerly Mercury TestDirector), giving them a key point of automated test execution, enforcement and test feedback. A welcome “byproduct” or end result of this integration between LISA and test management software is auditable documentation of the test, with details about the status of the many systems under test – showing that at a specific point in time, the system was working as expected, or not working as expected. The testing tool has then fulfilled the requirement of being the "long arm of the law," by spotting the issue, enforcing or “making judgment” on it, and reporting the issue to the TM tool.

The overarching goal of this process is to allow better Collaboration on testing, by sharing and iterating on tests across Development, Testing, business analysis and IT Operations teams. This increases the level of reuse in SOA and enables faster release cycles, as all teams can focus on delivering quality into new features, instead of repeating test creation and execution manually with each new release. In our next post, we will cover application lifecycle management.  You can download this complete "Agile SOA Across the Lifecycle" paper at our ITKO LISA resources page. 

Agile SOA Across the Lifecycle with LISA - Part One: Introduction

This is the first of a six part series of posts on how ITKO LISA works across the Agile life cycle. For years, testing was an siloed activity that used different and unconnected tools within specific phases of the application lifecycle.  First, a development team would run a suite of JUnit tests as part of a build process. Then a QA team would manually create and run a suite of functional tests against a user interface. Performance teams would use a separate set of load tests to exercise and monitor the completed application.

In this old world, there was not much sharing of information and test assets among these teams, beyond the occasional text “bug report” that carried little context into the root causes of problems that occurred. This occurred largely because the available test tools created scripted test assets that were incompatible, and largely irrelevant to the next team’s process in the chain. Unit tests were good for developers to test for structural bugs in code, but those tests didn’t translate into any business context for the QA team. The QA team would instead script their own UI tests, which don’t exercise components or code in a way that developers can use to track down root causes.

This was not a very efficient process so companies began to align their people around better process tools that reach across the silos to better manage development, integration and management of their applications.  Agility is not just about delivering faster, it also bumps up the reliability of enterprise applications, by instilling a higher degree of collaboration and control across these suites of application lifecycle tools and processes. Agile promises many development efficiencies and increased flexibility. See Chris Spagnuolo’s post on Embracing Change with Agile Practices for a nice description of the flexibility in agile ad the ability to make mid-course corrections. However, if testing and validation is not handled right, risks will emerge compromising these gains.

In this series we are going to look at four aspects of the Agile lifecycle: test & quality management, application lifecycle management, IT operations, monitoring and performance, IT and SOA governance.  But first, let’s take a look at a sample of leading application process tools we often encounter within iTKO customer sites.

The goal of all application process tools is to enforce some kind of workflow into an aspect of the software lifecycle, be it around testing, development, operations or governance and integration of the overall IT infrastructure. Direct testability and validation of most SOA and enterprise software components that are managed by leading process tools is a must.

By supporting the workflow of these tools with executable test assets, we provide a higher degree of predictability and performance from the resulting application, while allowing development, testing and operations teams to better manage their process with the methods and tools they prefer. While many of the above vendor solutions might work across different domains (for instance ALM might include some Test Management), we’ve provided the above table as a sample of the tools encountered in typical IT customer environments. Key to Agile development practices is the tight interaction of testing with development activities, for faster delivery time of prioritized business functionality.

In essence, a test or validation is an asset that is stored in, launched from, or provides feedback into the above process and automation tools, contributing to the goal of Agility for SOA. In our next post, we will cover the test & quality management phase.  You can download this complete series on "Agile SOA" at our ITKO LISA resources page.

 

Who Tests and Validates the Mashups? The Cave Man?

Joe McKendrick recently commented in his Fast Forward blog, Mashups: So Easy a Caveman Can Write Them? He makes a good case of the continued involvement of IT despite the claims of some mashup supporters. Joe quotes Ovum analyst, Tony Baer, “...no matter how visual mashup tools are, you still need developers or power users at some point of the lifecycle, whether it be to vet objects or sources than can be safely mashed up without violating some corporate policy, or to deal with some complexities of JavaScript under the hood.”

Joe concludes with the implication that even when the ease of use gets to the point where a caveman can do them, there is still a need for IT to play a supporting role in the foreseeable future. I agree since someone needs to test and validate the multiple data sources when business practices are at stake.

Skytap Provides Cloud Access and Licensing for ISV Solutions; ITKO LISA is an Initial Partner

Skytap, Inc., a leading provider of cloud-based virtual lab solutions, today announced its Independent Software Vendor (ISV) Alliance program to enable on-demand distribution of commercial software solutions through its Virtual Lab platform. Skytap's initial ISV partners include ITKO LISA. Skytap's Virtual Lab automation platform, combined with ITKO LISA, will enable customers to dynamically spin up entire IT environments for service-oriented testing or training labs.

I'm here at STARwest in hot Anaheim, CA, and a lot of the people I'm talking to are mighty curious about how Virtual Test Labs happen in the Cloud. For us, the ability to Virtualize a service-based application, and provision it with Skytap's virtual hardware, represents a great way to get the validation you need without the provisioning nightmare.

 

We think this is a great application for where the cloud is going - just like SOA, it is going to need better ways to test and validate the infrastructure (even if virtual) for expected business outcomes. Joe McKendrick mentioned my last post on this in his Service-Oriented blog on ZDnet: "SOA soon part of the Cloud: Hold that thought." The great thing is, if providers and customers are already thinking about these quality issues up front, the adoption of the Cloud model will catch up (and become part of) the wave of SOA.

Skytap allows companies to run their existing applications and virtual machines unchanged to take advantage of cloud economics without a major application rewrite. The company's Web services API and one-click VPN functionality allow organizations to create a "hybrid" IT model whereby cloud resources can be used and managed as an extension of existing onsite IT environments. With the introduction of the Skytap ISV alliance program, customers can now also access software from leading ISVs within the Skytap environment and gain the benefits of a pay-as-you-go model for the ISV software.

Jim Mackay, our CMO, said, "The cost model for using LISA Virtualize as a test and validation environment in the cloud makes perfect sense. In addition to offering Skytap customers complete SOA testing capabilities, LISA Virtualize abstracts away the actual system under test from dependencies on partner systems and third party resources, which means huge cost savings, as well as a much more rapid way to stand up a complex SOA environment."

Validation of cloud services makes them reliable for business. Virtualization of cloud services makes them available throughout the lifecycle. It's like two waves coming together for a bigger, stronger wave. Who can argue with that?

Dana Gardner Interviews John Michelsen on Roles and Methods for SOA Validation Across Complex Integration Lifecycles

Recently, ITKO’s John Michelsen was interviewed by Dana Gardner of Interarbor Solutions and ZDNet on his BriefingsDirect.  The session covered integration, validation, and testing for services-oriented architecture (SOA) and middleware -- particularly for business process management and to extend business processes more efficiently. As Dana said, “the whole notion of integration in the enterprise has exploded in terms of complexity -- but that puts more onus and importance on validation, testing and understanding what’s actually going on within these integration activities.”

John talked about the more rapid changes going on today to get to market faster, achieve regulatory compliance faster, etc. All this changes and the increasing interdependence between applications is creating many more ”testable” moments. This is providing increased opportunities for ITKO LISA.  Our growth rate is an indication -- as an IT discipline – the market has finally realized that companies have to get testing and validation right. You can listen to the complete conversation between John and Dana at this link to the Gardner - Michelsen BriefingsDirect podcast. Dana also commented on the interview in his post, iTKO’s SOA testing and validation role supports increasingly complex integration lifecycles. where you can also find a link to the podcast.

A full transcript is also available here at BriefingsDirect Transcripts if you'd rather read...

Turning SOA whining into debate, planning, and action

Joe McKendrick recently wondered if there is too much "whining" and not enough action on SOA? in a ZDNet post. He was responding to post by Nicholas Petreley of CIO.com on Less SOA QQ and More SOA Pew Pew.

Let me translate: QQ means whining and Pew Pew means action. To be more specific the urban dictionary offers some definitions of Pew Pew: Sound made by lasers, usually related to star wars.  And  also a common term in the world of Warcraft most typically meaning to do damage, or perform a combat action.

So is Nicholas telling us to ready-fire-aim?  Joe points out that there is already a lot of SOA action. He adds, "there is nothing wrong with a good rousing debate on what SOA means for an organization. Perhaps there should have been more debates like this before companies invested millions in ERP and CRM systems in the past."  So if whining becomes debating, this can be a good thing.

SOA can create agility and some incredible efficiencies, but there are a lot of issues to resolve when plunging in, unless you want to do a lot of pew pew (e.g., damage). For example, because of the complexity that comes with orchestrating multiple services, work out governance issues in advance.

Still, the QQ (whiners) about SOA may be surprised to find that they have development and integration teams that are already soaking in the SOA pool - creating services-based architecures - even if the IT management isn't classifying the efforts as such. We have seen this phenomenon at many large companies.

 

We also suggest looking at three key issues to consider before attacking an SOA strategy.

• First, acknowledge the fact that this is a heterogeneous layering of technology, so plan to verify outcomes in other layers and systems of record beyond the one you are building and testing.
• Second, enable more parallel activities to leverage assets more rapidly and take advantage of SOA. This is where Service-Oriented Virtualization comes into play -- if you can prune off those interdependencies upon other technologies by Virtualizing the rest of your architecture, it enables each team to be productive throughout the lifecycle.
• Third, we cant test as an event along a life cycle anymore, and think that this is enough. You need ongoing Continuous Validation, because services are coming from multiple sources which might change. Without this validation capability, it is impossible to know if your SOA Policies are actually being carried out in the real world, as there are multiple changing elements that can affect the workflow.

The factors can help with the SOA debate -- which is a better approach than either QQ or pew pew.

OutlookInterview Part Five: Conclusion and iTKO Overview

Here is the final installment of my recent interview with host Mike Lippis of the Outlook Series. We have broken the 45 minute podcast interview into five parts for easier reading. This section covers ITKO and concludes the interview.

Mike:  “Can you share a little background with the audience about iTKO’s general ecosystem, and as part of that could you give us have a use-case that you could share with us about how a given company may have used this LISA solution and benefited from it?”

John:  “I can even try to tie both of those together for you. 

“Our product LISA is especially suited for these complex heterogeneous type environments where there’s a variety of technologies at play, and the demands on these applications are extreme. That’s obviously companies like the financial sector and the insurance sector, and so we’ve done really well there, and in logistics and travel and a number of industries.

“For example, as in the US Department of Defense where they have the ‘everything’ problem, you layer in security and you layer in a lot of complexity there obviously, and a variety of different technologies and lots of legacy integrations.  We do really well when the problem gets harder and harder.  You know, any time your testing and your validation tool are trying to dumb-down the problem so they can get it tested … you should realize that this is signing you up for the harder part -- so that’s why we’ve gone to the approach that we have. So what’s that done from a customer base, has given us the most difficult areas of the largest companies and that becomes our target market. 

“We get partnerships from two different types of companies.  We get partnerships in the system integrator types, like Accenture, and Perficient who resell our product in areas, or any number of the largest outsourcing companies like Infosys, TCS, Patni, Satyam, Cognizant, Wipro, Mindtree, and I apologize to those listening who I haven’t mentioned -- I love every one of you guys so please don’t get me wrong… 

“From the services side:  we get a lot of help and that’s a major piece of our ecosystem because, quite frankly, those are the folks that are responsible for making something work for their customer and they are looking for the best practices way of doing this, and so they seek us out in many cases to do that. 

“Then the final part of our ecosystem is the actual ISVs -- the vendors who produce these SOA platforms, people in the integration space [e.g. TIBCO, Software AG, SAP, Oracle, Progress, etc.]. Of course that’s been a really good area for us as well, because again we’re uniquely suited in many cases to be able to do the proper testing, validation and virtualization of their platform.

“Just as an example of that:  TIBCO themselves use LISA throughout R&D in TIBCO to do the test and validation and virtualization work, so that they can decouple the independent components -- all of these different products that can be put together to create TIBCO solutions. They need to test them independently, and they need to validate them independently and so they do that and use us in all three ways.  Of course that helps TIBCO customers in many ways. 

“A number of our customers are on TIBCO, and on webMethods [Software AG] and on and on (here I go again when naming one I better be careful – (I love all of you guys so please don’t get me wrong) -- and a number of software companies are our partners because of that.”

Mike:  “Is there anything else that you would like to add to our discussion before we wrap up for today?”

John:  “If there was just one thing, it would be:  let’s bring it all the way back to the beginning and realize that this is really about a very key condition and that’s RISK.  That’s business risk and production outage. 

“I have yet to see the customer who is dramatically reducing their level of production outage, in fact it’s going up in many cases and this is a serious concern. Many companies that didn’t even measure risk as much in the past are now obligated to, through compliance regulations.  Because of this they are seeing that so much of their budget is caught up in maintaining the things that are already running, and the expense in keeping it limping along is so great they need to do something about it. 

“That is what we mean by this risk of failure that has been introduced; by so much more complexity and so much more rapid change.  That is why we are in the space that we’re in.  Every time you go SOA -- or every time you introduce some large-scale integration project -- or every time you bring in a new ERP system or you merge with another company -- you are agitating both the complexity and the pace of change.  If you do that without a consummate reaction on the risk mitigation side, that’s why your failure rate is going up, that’s why the cost of maintaining systems is going up.

“So for every one of those activities you are doing, you need to respond with a better, more appropriate, best-practice driven test, validation and virtualization process.  With that you’ll accomplish the change you are trying to make, and you’ll do so without increasing that level of production outage or cost of maintenance.”

Mike:  “Where can our audience go to learn more about iTKO and your LISA initiative?”

John:  “All things LISA and access to white papers and stored webinars and all sorts of things you can find at our web site which is: www.iTKO.com.  And I blog quite frequently and a few other folks with me blog on: blog.itko.com.  You can certainly look at what we’ve been talking about there and you can comment; we’d love to get back to you.”

Mike:  “I certainly appreciate your joining us today, and wish you the best of luck with your future endeavors.”

Outlook Interview Part Four: Virtualization and Testing

Here is part 4 of my recent interview with host Mike Lippis of the Outlook Series. We have broken the 45 minute podcast interview into five parts for easier reading. This section covers virtualization for SOA and how it affects the development and testing teams' productivity.

Mike:  “With this virtualized service edition of LISA.  What are you doing in the area of parallel development and test cycles?”

John:  “What we do is enable that parallel development, because we’re trying to decouple those service providers and consumers from each other during their lifecycle. 

“So a very reasonable approach towards meeting this kind of objective is that I might go through a requirement phase and a design phase and as I start development, I can discover my dependent systems, virtualize those dependent systems and then execute my development plan against my virtual services.  When I do this, I am decoupled from all of those constraints on my peers or on all those other services.

“The reason that I do this is because:  we’ve all heard our development leads say, ‘by the way I haven’t been able to do anything today because the “so & so system” is down.’  Or, ‘remember…they gave us a new build yesterday? Well the new build doesn’t work … they broke one of the major operations. So my team hasn’t had any productive time during the last two days.’ 

“We hear this all the time … this live connectivity to everybody (remember that white board I just mentioned a minute ago?)  Live connectivity to all of these boxes creates a large amount of friction or construction barriers, if you will, in the road. 

“What we can do, by isolating ourselves from that through virtualization, is we can parallel that development activity and, of course as a part of our final integration testing we go from virtual back to live services.  Once we are all live, we can obviously make sure that everything inter-operates the way we expect.  The value here is that instead of having to wait for my dependencies to complete, I can run in parallel with my dependency and get it done faster. 

“A quick example of this is:  If I’m asking you to implement a new feature in your service, I have two choices:  I either ask you to make that change and wait for you to make that change, test that change, produce that change into a build for me and make it available, or I can use a virtual service.

“I can make the change in the virtual model very quickly (we are talking minutes to do this with our product) … and I can be developing in parallel against the virtual version of your service just an hour or two later.  Meanwhile, you’ll take weeks to produce a new version but I’m developing against the new version in parallel in a virtual world.  Once your service is complete, I can point my application away from the virtual edition, and instead to the live edition, and we can make sure that it actually works as we expect.”

Mike:  “When it comes to testing in this area, we certainly want to simulate as many data set combinations as possible to see how effectively that this solution is working; how robust would release be in that area in terms of being able to simulate data set combinations?”

John:  “We’ve spent a lot of time on this.  It turns out … as we embarked on this virtualization strategy, we knew that virtualizing data layers would be a big thing, but I can’t tell you how many times I’ve heard our customers say “this is the big thing.”  It’s because of all this data synchronization, and all of this volatility of data – every team wants to reset the database every time they run their test and every time that they do a development cycle, and that means that no two teams can be sharing the database at the same time, and it’s always a mess!  We’ve spent a lot more time here than we’ve even thought and we’ve solved a lot of problems here.

“A classic example of this is:  When you are working on the purchase ordering system, and the supplier system is your partner, you don’t actually produce the supplier’s service, but you talk to a supplier … and yet you’re supposed to build an application that tightly integrates with your supplier … how do you do this?  Well again, you virtualize either at the behavioral layer or the data layer.  And, why do you virtualize at the data layer?

“Think about it … the supplier could respond to a PO request by supplying all of the requested goods at the requested date with no issue; but the supplier could just as easily split the order, could partially cancel, could respond to you that these items were no longer carried, could warn you with a credit check, could warn you … The number of possible data scenarios in the response to a requested PO -- it reaches the point where we wonder how we can possibly test all these out. 

“Since we can’t drive data scenarios in live systems, we can’t drive data scenarios with our partners -- at least we can’t effectively -- we need to virtualize that database so that we can drive all those.  I can make certain that every kind of possible response to a PO request is available to me all of the time.  That’s what data virtualization does for us.  It makes me able to make certain that every time I need the stars to align a certain way for me to make a certain condition occur, I can do that.  I’m certainly not strong enough to literally make the stars align but, in a virtual world I can, so in that way I can always make sure that everything works the way I expected.”

Mike:  “There are a number of load and performance testing tools such as LoadRunner or SilkPerformer, and a lot of companies have made investments in those technologies, so how can they leverage those investments that they’ve made and expand on those with LISA Virtual Service Edition?”

John:  “That’s absolutely true Mike, there’s plenty of successful LoadRunner shops and SilkPerformer shops and others, and LISA does load testing, but we’re certainly not interested in trying to fix what’s not broken.  We’re certainly quite comfortable with working with companies who are doing load testing with LoadRunner.  A number of our virtual services environment customers are doing exactly that.

“In fact, if you picture in your mind:  the left-hand side of the graphic is Load Runner generating a load on a system in the middle of this graphic and on the right-hand side of this graphic is LISA Virtual Services environment.  What we do is enable more and better load testing even with the tools that we did not provide.  What we’re doing is we’re freeing the system that you’re testing from its constraints.  We’re not eliminating that system nor are we trying to affect the load test product itself.  We’re actually trying to virtualize away the constraints on the right hand side of the graphic. 

“So, it’s LoadRunner, versus the System Under Test, which can be LISA … and that triad, if you will, makes for a very powerful load testing combination.  What we have found is that you have to have production hardware or at least a reasonable production performance lab for your own application. 

“Our customers understand this, what they have trouble with is:  ‘Wait a minute, my application no longer lives on its own island, it now goes and invokes its own transactions on a main frame or it now integrates between a TIBCO-based system or this web message-based system and it goes and talks to this ERP system.’ 

“In order for me to performance test my application, my performance lab just went from a half million-dollar investment to a five million-dollar investment … and that’s what they can’t do.  So when you go Load Runner/Your Application/LISA, you eliminate the five million-dollar investment. You literally eliminate it.  What you go from instead is the half million-dollar investment in your hardware platform -- what you have to run your application and your service on, and we virtualize away all the heavy iron that you have to connect to.  We put that on a commodity box, what would be a rounding area on a half-million dollar investment -- we would run on a $20,000 server.”

[This concludes part four. In the next post we wrap up the interview and provide an overview of ITKO.]

Outlook Interview Part Two: SOA Testing Best Practices

Here is part two of my recent interview with host Mike Lippis of the Outlook Series. We have broken the 45 minute podcast interview into five parts for easier reading. This section covers SOA testing best practices.

Mike:  “Service Oriented Architectures really have been around for a long period of time, and it’s really not a certain thing but more of a style of doing something.  If it’s a style, there’s going to be different kinds of approaches -- lots of interdependencies and something that’s going to continuously change and evolve.  What are you doing in the area of SOA testing and addressing those kinds of structure flow issues?”

John:  “That’s a great point because you’re right; the concept of component-based applications has been around for a long time.  I’m going to show my age here, by the time we come to the end of this conversation, you’ll know that I’ve been around a little while… 

“Our approach is basically the same as it has been for many years:  let’s take business logic and let’s encapsulate it into objects so that they can be leveraged across applications.  Let’s start to layer systems by their responsibility or domain so that we can build applications that are specific in terms of their interface.  For example, I might have a voice response unit for telephone callers and I might have a call center application for users on the phone but I don’t have to rewrite all of that logic every time, and I can make changes to business logic more quickly that way.  We’ve been talking about that concept since the late 70’s.  What makes SOA different today is that we are accomplishing it more than we ever had in the past due to the commoditization of middleware.  The fact that there is highly available, very reliable and also very understandable and cheap middleware means that it is possible to build these applications today.  So we are actually accomplishing it more than we used to in the past.”

Mike:  “I’ve seen in some of your collateral that you talk about the three C’s for some time now.  Can you tell us what those three C’s are and educate us as to what you have been doing and kind of describing the problem in that framework?”

John:  “The 3 C’s are our framework for best practices for SOA testing.  What we say is that if you have those 3 C’s right, than you are in good shape for having sound SOA test strategy. And, you can imagine our product LISA is a great way to accomplish these three C’s.  However you do it, you need them because essentially this is the standard now, so how do you reach it? 

“The first of these three C’s is “COMPLETE” and it essentially acknowledges the fact that this is a heterogeneous layering of technology. We must support a concept that we call Invoke & Verify.  If I invoke at one layer of technology I have to actually verify the outcome I am expecting by going to a different technology layer.  For example, if I go to an ATM and I deposit the money in the ATM and if I trust the receipt that it prints, that’s not really testing.  The way I test the ATM is by calling the bank teller or by going to my web site and looking on the web -- I go around the actual device that I am testing so that I can prove the outcome that I’ve expected. 

“The second of these three C’s is COLLABORATIVE.  Collaborative basically requires us to do more parallel activities and to leverage assets more rapidly than we had in the past.  Agile and SOA are usually tightly aligned.  In fact, most CIO’s tell me ‘I’m going SOA so that I can become agile.’  They are very linked at the highest level in most of our accounts.  Because of that, we need to do more in parallel and more shared. 

“The third of these three C’s is CONTINUOUS.  This essentially acknowledges the fact that we can’t test as an event along a life cycle anymore, and think that this is enough.  The fact that I’m dependent on let’s say your services Mike, means that I may test my applications today -- deploy this Friday and the following Monday you update a service underneath my application and threaten or risk – there’s our word - risk failure of my application in doing so.   So our strategy calls for a continuous validation of functionality, performance and certain policies on a continuous basis, - that’s the
third ‘C’.”

Mike:  “You have a product that’s called LISA.  How does LISA specifically achieve these three C’s that you just described?”

John:  “Breaking it down a bit, we support all of those different technologies from a COMPLETE point of view.  Whether it’s web technology, Thick UIs, messaging layers, web services, older protocols like CICS and CORBA … all of those kinds of things; we support all of those different technologies from the same test case.  It’s very important that you be able to do this in one test with one product.  If you buy three or four different products to accomplish one test scenario, that is way too much effort and it is very difficult to try to integrate a bunch of testing tools to make one test.  It’s really not feasible to do -- you need to have all of these technologies covered in one workflow.  That is why we differentiate in many ways on the ‘COMPLETE’ C. 

“For COLLABORATIVE, we provide the ability to do Development type testing, (whether that’s Unit testing or Service level testing), Functional and System type testing, Performance Testing, even Functional Monitoring all as one unified test from one product.  We’ve got to leverage each other’s assets more.  Developers need to start the testing process, QA needs to consume and extend, if you will, with development based testing, etc. so that’s how we support the COLLABORATIVE.   

“For CONTINUOUS, we actually provide both continuous integration support so we integrate continuous integration-type tools … build tools like ANT and CruiseControl and all those types of things.  We also provide our own infrastructure for integration labs, for production, for pre-prod type systems enabling us to provide our own infrastructure for continuous validation.  Therefore, we specifically support those three C’s in our product development and design, so we can accomplish these best practices for our customers.”

Mike:  “It really sounds like SOA testing is a key part of overall IT governance then.”

John:  “You know testing clearly is -- it’s gotten much more relevant in the last few years.  It used to be just a step along a time line: ‘we make sure we test it before we let it go into the wild’ -- and that was just fine.  Today, because we have a greater degree of reuse expected, and because we have a more distributed development force -- where they are not even always us -- sometimes it’s ‘us & them,’ unfortunately, where I’m supposed to trust people I don’t even meet or know.  Because of that, the need to test has become greater and the possibility of failure becomes greater along with it.  So testing has become a much more critical focus of both development and even of governance.  In many ways test is the means to instill trust which is, of course, the goal for governance.”

[This concludes part two. In our next post we cover performance management and testing.]

Outlook Interview Part One: Current Business Trends and SOA Testing

I was recently interviewed on the program:  Outlook Series with host Mike Lippis, and we covered a variety of SOA-related topics, from Testing -- all the way to Virtualization.

You can listen to the 45 minute podcast interview here, but our team has also broken the interview into five parts for easier reading. Here is part one -- which focuses on SOA Governance and how companies are attempting to use this discipline to meet  business goals.

Mike:  “Can you share with the audience what are some business trends that might be driving any kind of shift in IT environments currently?”

John: “The business is why we are here.  So what we need to do is to align our initiatives, as an IT organization, toward what the business is trying to accomplish.

“Among these (initiatives) are to increase the agility that we have to their customers in terms of responding to regulatory pressure…new market competitive pressure, etc.; to reduce cost…to go global…to either be acquiring or to become a part of larger organizations (M & A is becoming quite common, especially right now), and then, of course, compliance is over-riding all of this in that -- even though we’re increasing the speed of the train, we need to be more careful about how we execute along the path.”

Mike:  “What kind of IT initiatives would be driving that besides things like SOA and virtualization for example?”

John:  “Those are certainly two of them, as the business is trying to reduce cost, we need to increase our leverage and reuse, and so Service-Oriented is certainly one way to do that.  To increase our speed of development we need to take a more agile process, we need to leverage and extend technologies more often than bringing in new siloed systems.  Reducing costs often is reuse, but it is also outsourcing and off- shoring and virtualizing things.  So there is quite a bit going on in IT in order to accomplish these business objectives.”

Mike:  “Sounds like there’s a new IT environment driven a lot by SOA and Virtualization.  Could you describe some of the downsides associated with these new IT environments?”

John: “For all of the goodness that we are trying to bring about by increasing inter-connectedness to leverage and extend -- reusing existing assets, and the improvements we’re trying to provide by increasing our pace of change, (we’re releasing software updates to our business users more frequently than we were in the past) -- those changes are more interconnected with lots of different systems than they were in the past. While those are good things; the unfortunate outcome is that we’re increasing our risk of failure every time we increase the number of times we do something and every time we increase the complexity in which we do something, we necessarily have an increase in the possibility of failing, so we have to address that.”

Mike:  “So, it sounds like you’re really looking at minimizing business risk in terms of what your overall mission is at iTKO.  Could you kind of drill down into that a little bit for us to kind of see how you are addressing some of those issues?”

John:  “You bet - and that’s a good point Mike because we frequently get asked: ‘If you’re in the virtualization space why do you also offer testing?’  Well, because the real business we’re in is helping to resolve the risk associated with service-based architectures and agile processes, and more generally increasing complexity and increasing paper chains.  So what we try to do is:  if we can help enable these positive effects without the risk; we’re doing so to shorten the time to market, to reduce the failure rate in production, to increase our recovery time when there is a failure, and in general then to reduce the total cost of maintaining our own ESB systems that we produce.”

Mike:  “When it comes to SOA testing which is a big part of what you do, what are some of the challenges surrounding SOA testing?”

John:  “There are a number; the one that’s most common is that SOA increases the likelihood that we just can’t manually test -- we have an inadequate level of testing in many organizations from an automated standpoint and this becomes the governor on how frequently we can make changes to our system.  If I have to manually revalidate an application any time anything changes, then I can’t change it fast enough.”

Mike:  “I would think that today, based on what we’ve discussed so far, in terms of all the complexity, you really have a lot of heterogeneous environments and or most IT environments are very heterogeneous in terms of the technology that‘s now being used.  So how does this come into play with the SOA and what you are doing?”

John:  “Well it definitely scratches that problem quite a bit.  The reason it does is because:  Instead of thinking: my application is this completely self-contained box that I deliver to my users and they use it directly; rather, many teams are now building components of an integrated system and those components are, (as you say yourself) in heterogeneous technology, where there may be existing legacy CORBA applications and main frame applications and even mid-80’s mainframe messaging architectures, and modern messaging architectures, and all of these things being brought together for one application.

“So, we have a real challenge here when we try to test a system like this, we can’t just test each individual box independent of each other (although we should do some of this certainly) we must be able to test across the work flow as the business executes, so we must be able to test across these heterogeneous boundaries.”

Mike:  “You had mentioned earlier-on that IT initiatives, such as outsourcing and off- shoring for example, are very common in today’s global economy, how does this come into play with collaboration and to what extent does this frustrate collaboration; what are you doing in that area to make that a more seamless exercise?”

John:  “So the short version of a very long best practices story is that it necessitates collaboration, and it really does because:  what we can’t do is to have completely disjointed applications being developed -- we need collaboration.  Collaboration comes in a variety of ways.  One is we need visibility, the ability to touch all of these different technologies and view through all of these business processes all the way down into the implementation technology.  That’s just one simple way.

“Another way is we need governance in place that better creates a structure in which we all can predict how we are behaving.  The more reasonable governance is in place, the better we will all function and the better the trust there will be in this very disconnected world that we’re building and maintaining with that.  So there are a variety of things that collaboration encompasses but that’s certainly two of them.”

[After setting a little context, the next post picks up with a discussion of SOA testing best practices, our three C's. Stay tuned.]

Forrester's Five Step Model - Balancing the Strategic and Tactical Aspects of SOA

I had a chance to look at Forrester analyst Randy Heffner’s report on the best way to build SOA (via Joe McKendrick’s Forrester's Five-Step Path to Building SOA). Joe provided a link to the complete report, How to Build Your SOA Platform on ebizQ.  Randy writes in the summary, “Building an effective service-oriented architecture (SOA) platform requires cohesive integration of both new product categories, such as SOA repositories and enterprise service buses (ESBs), and existing product categories, to which vendors are making many SOA enhancements.” He argues against a big-bang transformation except for the very wealthy, but also says an ad-hoc plan can be a waste of money. Instead, he suggests that the best approach is to “place tactical SOA platform investments within a planning framework for evolving a strategic SOA platform. If, along the way, you are able to justify large investments to accelerate your platform build-out, so much the better.”

Joe does a nice job of summarizing the five steps that Randy advocates: Identify existing infrastructure’s SOA capabilities, identify priorities for new SOA capabilities, identify your long-term needs for SOA capabilities, match your platform plans to your organization’s investment strategy, and evolve your SOA platform in line with the business value of solution delivery projects. 

I wanted to address Randy’s opening statement, effective SOA “requires cohesive integration of both new product categories, such as SOA repositories and enterprise service buses.”  Making this integration and transformation simpler is the goal of SOA Testing, Validation & Virtualization practices. In fact it is hard to get to the point of integration without conducting these activities throughout the full lifecycle of the application.

Randy makes a point that there is no one dominant way to build an SOA; companies might start with a specific technology like REST or SOAP or ESB in mind, but that isn't what drives the budget. Indeed it is a heterogeneous problem for any ongoing business with existing technologies to evolve to some future of services-based applications and reuse.

There is value in thinking about SOA from a strategic perspective, and we are big advocates of architecture and design driving selection processes. Before we start, we need to rationalize those existing and proposed technologies for relevance to the business needs and budget.

Now when it comes to articulating these Strategies into SOA Policy, this is where I really feel we have something to add to the discussion. The business-driven expectations of Performance, Scalability and Reliability (PSR) need a mechanism that can verify and enforce these Policies to ensure that they are being met. This needs to happen at the initial component level of design and development, as well as continuously at runtime.

But think about this concept a bit further -- why not Validate your expectations as part of the vetting and selection process for your design?  If you could have an idea of the performance and logical correctness of using a suggested ESB or Service layer, you could recommend it as a next budgeted step in your SOA migration with a lot more confidence. The level of testing doesn't have to be prohibitively strenuous, but being able to verify some business logic and performance expectations of that candidate service early on would save plenty of time and money, compared to testing at integration time.

I certainly agree with Randy that SOA implementation needs to be a balance of strategic and tactical activities, and let's continue to think about how we can help with both ends.

SOA Pushes IT Leadership beyond Traditional Enterprise Architect Role

Dana Gardner recently reported on an interesting Open Group panel in his post, SOA places broad demands on IT leadership beyond traditional enterprise architect role.  At the 19th Annual Open Group Enterprise Architect’s Practitioners Conference in Chicago, Dana moderated a panel on the role and impact of skills and experience for EAs in both the public sector and the private sector.  Joe McKendrick was one of the members and he also wrote about it, At Open Group: business and government face similar SOA growth pains.

The panel members conveyed that the complexity and interdependencies of SOA are pushing IT leadership more toward a business strategy role. As one panelist said, “The enterprise architect is really more focused on trying to bring the organization together around a business strategy or mission. And, certainly, they understand the tooling and how to translate the mission, vision, and strategy into an SOA architecture.” And “what we are really looking for are people who can then move into the business space, who have a lot more of the softer skills, things like influencing.” You can find the complete transcript of the session through Dana’s post.

Technical skills remain a necessary but no longer sufficient requirement for IT leaders in the new SOA space. I certainly agree with the premise here. To support this requirement, we are constantly striving to make the enabling technologies of SOA more transparent, so that architecture is more of a business process and Policy activity than a programming one.  For instance, enabling the "auto-discovery" of a service and its downstream dependencies, and providing that as a view without all of the technical poking and prodding usually needed to get a diagnosis.

As you go through testing and validation of all the interconnected pieces of SOA, it is important to keep the business side and impacts in mind.  As one of the panelists said, “we are software architects, but we are really trying to solve the business problem.” It is nice to see a group of IT industry leaders taking this position.

Overcoming the “If Onlies” of Application Development

Forrester's Carey Schwaber recently wrote a nice post on the typical obstacles that application developers face, The "If Onlies" of App Dev. For example, if only “requirements were complete” or “there were enough time to test at the end of the life cycle" or "we weren’t constrained by legacy" among others.  Using a transportation metaphor, Carey then suggests that changing the vehicle through approaches such as SOA and Agile processes can negate the problems raised by the 'if onlies.'  I especially like her insertion of the image of Homer Simpson’s dream car as a note of irony.

She asks her readers about their 'if onlies.' It is a good question for people who create products that support application developers. It is also a good question for the architects, testers and developers delivering functionality for end consumers. We are constantly to talking with clients about their 'if onlies' to discover new ways we can change their means of transportation. It made me think: what are the 'if onlies' our customers are saying about their own process?

Testing:

• If only we had the [time/budget/resources/tools] to test... or "If only management cared more about quality before the project due date"
• If only the [QA team didn't see UI testing at the end as application testing] OR [development team didn't see unit testing as actual test coverage], we would have better delivered quality.
• If only I could use my component tests in regression and then load testing phases it would save us a lot of trouble.

Validation:

• If only I could invoke a behavior somewhere in my application, like at the UI or web service, and then verify it somewhere else in a database or other system...
• If only BPEL provided enough information to validate my business process (sorry it won't)...
• If only I could get business stakeholders like analysts, partners and customers involved in validating my applications.

Virtualization:

• If only I had access to the mainframe for more than 2 hours a week
• If only the transaction or information Service my application leverages didn't charge me fees even for my test transactions while I'm developing
• If only we didn't "go Agile" I wouldn't be waiting on the other small team to finish developing their own component before I can continue developing and testing mine...
• If only I could have my own test environment including our SAP
  financials, Oracle order management, multiple databases & CORBA,
  etc. that I can't just virtualize with hardware methods, without
  basically doubling our team size and budget to support that instance!

Thanks to Carey for this image to consider when facing the challenges of application development. I think if more AD vendors prioritized their own development efforts around the "if onlies" of their customers... there'd be less of them.