The ITKO Blog: Service Virtualization and Application Lifecycle Optimization - CA Technologies

A humble test may just make SOA Governance Policy someday...

The marketing guys are at it again - read David Thompson's recent discussion on SOA Testing.

Actually, we are very glad this discourse is happening out there. SOA Testing has long been relegated to a project milestone somewhere after development and integration happens. Our CMO Jim Mackay just had to chime in. SOA Testing is a continuous part of SOA Governance, and not an event that unit tests a specific technology as a pre-release feelgood.

True, you do need to test the Web Services (if that is a part of your SOA approach) to be compliant to WS-I, etc. And of course, you need to be able to test the Messaging layers for compliance with the platform as transactions move through the architecture. But these kinds of tests don't tell you if the business requirement is being missed. Or if the upstream and downstream dependencies are creating problems.

You also need to test the database, and the front-end HTTP. And the EJB itself. And both the legacy CORBA objects and the new POJOs sitting at the implementation layer. If SOA is making that round trip, your testing should make the round trip as well.

And testing should support the entire extended set of players collaborating on SOA, from the people writing the requirements, to the developers implementing them in SOA, and the QA teams verifying the functionality and performance.

And SOA testing should be continuous, not just in development and integration, but in deployment, because an SOA by nature is never a static application.

Remember the old School House Rock educational commercials? "I'm just a bill, yes I'm only a bill, and I'm sittin' here on Capitol Hill..." Well, the humble Test is just like that bill waiting to become a law (or an SOA Policy). In order to get there, it is going to need the support and nurturing of the SOA community at a company level, and at large.

In other words, we promote the humble test as a actionable aspect of the vaunted concept of SOA Governance.  To be part of Governance, the SOA Test must contribute more than a specific technology checkpoint at a specific point in time. A test must span the SOA application, continuously, and in so doing, it becomes a verifiable SOA Policy.

More on this concept to follow - and your thoughts welcomed.

Global SOA - Quality = A World of Trouble?

We have a customer who was a highly advanced and ambitious early adopter of Services-based applications. Prior to adopting some of our best practices and our product for continuous testing, they had come to the point where their Services Group (a shared services team that was the producer of a set of services for an entire division of a company) had to go to an annual release basis.

Much worse, because of the fact that there were so many applications built on top of the shared services platform, and they were so distributed globally, they really couldn’t keep their hands around everyone’s use of these shared services.

They had to make a hard stop on November 1^st of every year, and go into a 6-week procedural test cycle, where any failure anywhere within the entire division would stop the rest of the division from being able to complete the test cycle. And, upon completion (hopefully in that six weeks) to find what could run for that month’s test cycle, finally release all the upgrades to not only the shared services that were built that year, but all of the applications that were depending upon that shared services platform.

Now, hopefully, that gives you some insight into what happens when you try to go to a services approach, but you don’t marry it with very rigorous and continuous automated testing. At the heart of the matter for these guys, SOA lifecycle quality was the missing piece. When you have a shared set of services being leveraged by a variety of consumers across the globe, the changes that are seemingly irrelevant almost, at the shared services level, can have a catastrophic effect on the applications that depend upon them. The worst part is that the developers of those shared services may not even be aware of the potential catastrophic effects of their application cycles on the service consumers.

All it takes is once or twice of having those unintended consequences show up as huge performance issues, or as huge production outages, or, my worst nightmare, actual bad functional integrity where the business is actually getting the wrong answers (I don’t mind slow answers or no answers -- but wrong answers are worse). No one is sitting with a calculator beside their computer verifying all of the transactions that the system is doing. So if the global system is doing them wrong -- you are in a world of trouble.

So what they had to do was coordinate the release of every one of these applications within this very large division of a large company. With our solution, taking some good SOA best practices, and using LISA from a continuous testing/continuous integration kind of basis, they are now going to be able to start dismantling this annual release cycle. As they increase the level of automated coverage of shared services and of the applications that depend on them; they have the freedom to change without the unintended consequences catching them off-guard.

So, in a nutshell, continuous SOA Lifecycle Quality testing to prove functional integrity has huge returns for the business. They see the value of going from an annual Big Bang release, to more quarterly releases, then monthly releases or “as needed” releases so that they can realize the agility they expected from SOA, with the reliability required for business. John Michelsen, Founder and Chief Architect, iTKO

A LISA WS-Testing (Web Services) Testing Update

I got a notice in my Google News Scanner E-mail, that one of the so-called major testing vendors is in beta with an SOA testing product.

Well, good for them, we grabbed a beta copy and we did our own self-evaluation of this, and we think that by the time they get to Version 2 or 3, they might have what our free LISA WS-Testing release currently provides.

I thought that we’d do the market a favor by putting out a completely free -- as in free beer -- version of a web services testing product for a while, so that all of these guys don’t have to throw out yet another point solution and try to get their 5K of flesh per user.

Evidently, that’s not going to happen, these guys are going to peddle SOA testing as JBOWS testing (Just a Bunch of Web Services testing) only, and they are going to try to tighten around the point solution for WSDL/SOAP testing as SOA. Just take the “P” off SOAP tests, and you have SOA tests, right?

We know that this is just not the case. In fact, one of our favorite case studies is of a system integration firm. The lead developer of the test strategy said to us: “One in every 10 of their test cases was purely web services tests” -- and this was an SOA-based implementation. So it’s just irrational to think that an SOA testing product that is just Web Services is going to carry the load.

We’ve had thousands of down-loaders -- thank you guys! -- we have a fairly active Forums Area for community support, and of course we provide lots of answers ourselves there.

In the end, Web Services testing with LISA WS-Testing won’t be enough either, but it’s a start, and it’s going to be MORE than you’re going to buy from another vendor.

Your SOA Governance Policy Isn't Big Enough

As we continue to mature the SOA Governance space, the Policy area appears to be the one that is the most immature.

Vendors that are focused on Policy have a big, big job ahead of them. In fact, we believe that the Registry/Repository area and the Testing area of governance will have a lot to do with policy definition and enforcement in addition to those tools that are called governance policy products.

The reason is because policy is a VBT (Very Big Thing). In fact, I think 3 years from now governance will really just be defined by policies and in so doing it will take all three aspects of governance to make policy work.

A simple example -- today’s Policy typically sounds like:

• “Interoperability checks”
• “ws-security verification”
• “Hit testing on XML documents (for you bit twisters like me)”

Pretty granular stuff. That’s not a very big policy.

In fact, the business won’t want to mandate policy that has much to do with things in XML and what standards you’re using. Of course they want standards to get the reuse and interoperability, but their version of policy will be:

• “I really need functional integrity on this particular transaction activity of the system”
• “I really need customer credit checks to always be accurate to within 30 minutes of the customer’s activity.”
• “I can’t allow stale data to be reported as customer’s current activity.”

All of these kinds of things – that’s Policy.

When the business talks about policy, they are not going to read standards definitions from OASIS. They are going to talk about what their business functions are and what kind of expectations they want to put on the systems that are implementing those policies. That is Policy – that is a Very Big Thing.

So, short and sweet, if you are talking to a vendor who has got a so-called complete SOA Governance Policy Management System -- I haven’t yet seen that thing. Their definition of Policy is way too small.

You’re going to have to sign off on the much bigger problem of Policy at the business level, which is what SOA should be all about. Remember, we are aligning IT to the business, but not if we force-feed a definition of Policy to business. Business will take on a "free-market definition" of SOA Policy, whatever the consortiums try to dictate.

When the definition of policy is business-aligned it’s going to require very rigorous, here’s the bad word, “TESTING” -- on a continuous basis of the functional integrity of the system as it is running. Of course, things like IT guidelines are good policy, Performance Expectations are good policy, Down Time and Notification and all of those other things, very good policy. But if it isn’t also heavily involved in the way the business wants to set up expected behaviors, it’s not big enough.

-- John Michelsen, Founder and Chief Architect, iTKO

Web 2.0 versus SOA?

WeB 2.0 versus SOA? So I’m reading Blogs -- which is suspect to begin with because I’m sure there’s just a bunch of random content out there in the blog world right? And, I run across several Blogs on Web 2.0 versus SOA”.

Let me tell you friends -- if someone is trying to pit Web 2.0 versus SOA they either don’t understand Web 2.0, they don’t understand SOA or both. I thought Joe McKendrick did a good job of describing the irrelevance of this argument as far as CTOs are concerned in Another view: A CTO underwhelmed by SOA,.

It’s complementary to SOA, which is a way to establish the decomposition of business logic to establish service-based workflows--to break apart monolithic systems into services such that collaboration among services is more rapid and more agile.

So what you can accomplish with Web 2.0 is synergistic with what you can do with SOA. By the way, don’t misread this and therefore presume that SOA requires Web 2.0.(because that Blog, I am sure will be the next Blog I see if I go Googling a little bit more) and that’s not correct either.

You can build SOA with C code and Windows Client UI written in VB if you wish. The goal here is to try to establish a little more sanity in what these things actually are. Web 2.0 is a great way to put a thin client yet interactive user experience on the desktop. SOA is a great way to go about the process of decomposing systems into services and making business alignments through business process management.
- John Michelsen, founder and Chief Architect, iTKO LISA