The ITKO Blog: Service Virtualization and Application Lifecycle Optimization - CA Technologies

Computerworld: SOA Testing Goes Beyond Web Services

Mark Hall talked to iTKO's John Michelsen (the guy who writes most of this blog) about a constant theme near and dear to our hearts.

And we'll keep talking about it until they are not exactly the same.

Bonus: Our 5 9's of SOA Business Continuity article in eBizQ ... the whole enchilada. Read on.

Someone else talking about 5 9's in SOA

Business Continuity is an interesting topic. As an SOA testing firm we like to make the distinction that "reliability" at a statistical uptime level, is only part of the challenge of Five Nines. As hardware and bandwidth become more commoditized, that becomes something you can buy fairly easily. Ok, so, the title looks familiar?

Service Architecture - SOA: Five Nines in a Service Oriented World - Part 1 the problem

Now we are seeing companies thinking of 5 9's in a Functional sense: Can I trust the accuracy and integrity of the process. "Failure" isn't downtime of the connection or the app server, etc., it is a failure of the system to provide the right business logic.

In any case, thanks for the discussion and we invite you guys to check out our whitepaper on this topic:

http://www.itko.com/site/resources/59s.jsp

The Big Difference Between Web Services and Full SOA Testing

We here at iTKO say all the time that there’s a big difference between web services and full SOA testing.

What do we mean by that? Usually we get that goofy blank stare because so many people equate web services with SOA and so from a testing point of view they equate the testing of web services with the testing of SOA.

While it is true that a number of initiatives for doing SOA are very web services centric, Aberdeen’s last research on this points out that only about 50% of the SOA initiatives at large companies are web services based. There are a variety of technologies being used to create that commoditized middleware for SOA. While we love web services for this, other technologies are very valid, possibly better for a given organization than a web services stack. I’ll maybe spend some time blogging on that topic later.

The biggest difference between web services testing and full SOA testing, even for a web services shop, is the concept of testing implementation and side-effects as opposed to just the middleware layer.

Let me give you a scenario:
Let’s say that you are trying to test the functionality of a web service that encapsulates two or three different systems, and provides one façade for those three systems back to a potential service consumer -- this is a really good use of web services. So, you have a web services testing tool invoke web services and get payloads back in the form of SOAP XML docs. Now let’s say you get something back that doesn’t look right. Where’s the problem?

• Is it in the web services stack?
  
• Is it in the implementation of that wrapper?
  
• Is it in the implementation of one of the three underlying services?
  
• Or, is it in the interaction among those three components, (four when you include the web service itself)?
  

Without complete heterogeneous testing of all of the components involved in the SOA, you’re never going to know the root cause of the problem. That is the huge distinction between the ability to test a real SOA implementation vs. the ability to do WSDL/SOAP testing of Web Services alone.

Here’s an example of how this plays out in the real world:

I was with a prospect a few months ago and going down a path somewhat similar to this with them, when someone just jumped right out of the blue and said:
“You know I had a situation just like that -- I’m a tester and I brought a SOAP document to one of my developers and said: Hey, I believe that this proves that there’s something wrong with the service.”

The developer took one look at that XML document and said, “This doesn’t resemble anything I do.”

While that kind of brush-off might not be warranted, the developer has a point. Our tendency is to suspect the validity of the indirect vs. proof that is directly connected to the implementation. In order to really help that developer see what’s going on in his or her own component, we needed to see a test of their implementation directly.

We’ve got to test as close to the implementation as possible because that’s where the bug is. We don’t want to test indirectly. Other moving parts of the environment will cloud our visibility into the root cause and tend to blame the wrong things. It sounds like UI testing all over again only instead of a user interface, we’re using a web services interface that hides the business logic as it is programmed.

Hopefully I’ve given you some insight on one of the many differences in fact there are between web services testing and SOA testing.

SOA as a SaaS or SOAaaSaaS

Serendipitously to John's recent post on "Consumption is not Free," Dave Linthicum just put out a great note on SOA as a SaaS (or "SOAaaSaaS") that nicely covers requirements of the Publish side of delivering reusable services. Merits further thought... see SOA as a SaaS...What to Plan For

Where is all of this going? SOA and SaaS are mutually very compatible and may overlap, but they are not mutually inclusive.

SaaS is really more of a business model for reaching end customers - delivering a business over the Internet, generally through a browser instead of requiring the end user to install a client or other software on their side. The method of creating a SaaS can be done as a SOA, but in most cases, it is developed as a vertically integrated platform that the SaaS provider can control, even if some parts of the SaaS are exposed as Services.

On the other hand, delivering a reusable Service in SOA fashion, even for profit, is not necessarily a SaaS. That Service might need interpretation and certification as a "good citizen" before being offered to the end customer. The real question, and maybe this is just a point of distinction for the pundits and analysts, is: Does a SaaS need to be a complete business offering - or core to the business model, or is any Service offered for profit technically a SaaS?

For my part, I'd say the terms belong to different software disciplines, one architectural, the other marketing related:

1. SOA is a business strategy for aligning IT assets as Services in a loosely-coupled way to support business processes with more flexibility and agility.
2. SaaS is a business model for offering or selling software (or a business service, with a small "s" and not a technology Service) to end customers without the installation or client requirements. Most SaaS offerings are actually vertically integrated web implementations of software under a single brand's authority domain, which isn't always in the SOA spirit.
   

Of course, the discussion does have value, especially if we look back at it in a year or so. We expect someday to offer certified Services that obey expected behaviors in a "for profit" way, yet to do SOA these services need to be cooperative at the same time. But enough for now, we could make distinctions with SOA, SaaS, Web 2.0, etc. all day long, without even getting into the underlying technologies or the overarching Governance strategy.

Labels: saas, soa business

SOA as a Marketplace

Consider that there is a cost reduction or profit motive to an SOA initiative. Also consider that I’ve already seen a financial incentive to those who successfully create reusable services that are leveraged throughout the organization. Before long, you realize that SOA is likely to create a competitive marketplace for services within an organization.

This marketplace will have most or all of the common attributes of any other marketplace, i.e.: promotion, profit incentive, and a winner and loser type mentality to those participants in the marketplace.

This is a good thing. The alternative is hope that altruism reigns throughout organizations and people make corporate decisions with no financial incentive to do so. Good luck!

Let’s walk through a scenario: Let’s say that I create a service that I wish to offer for reuse across the enterprise. In order to do so I must meet certain objectives, certain criteria that make it available for reuse. In doing so, I have to create additional investment before there’s ever any reuse possible. To justify this, my consumers are going to end up either directly or indirectly compensating me for the additional burden of creating that reusable service and at the very least maintaining their behavioral expectations on my services over time.

As soon as this occurs, (as soon as the producer is getting financially compensated for creating the reusable service) I’ve just reduced the cost of producing the service. There goes the marketplace.

(now I’m going to put on a divisional development director kind of hat…)

My goal becomes: To see how many globally reusable services I can mine from my requirements. I will begin promoting my development of those services. In fact, I’ll make sure that my teams will do the best job in producing those kinds of services so that I can cover the long term needs of the enterprise, thereby defraying the cost of producing these applications that have caused the construction of those services. If I’m better at it than the other guys across the organization, my services will tend to get leveraged more than theirs, and my budget indirectly grows.

The phenomenon here is simple enough: all it takes is to incent SOA is money. Since we can’t expect SOA to survive long-term with the concept of “free beer” reuse, or “anarchy,” we’re going to have more pay-for-use kinds of consumption of services.

Now this doesn’t mean that there’s likely to be some kind of transaction meter deployed in the infrastructure, or that consumers in one division will have to negotiate a price to buy those services, that’s not what I’m saying (this could be, but I doubt that it will happen in all but the rarest of cases). What I’m saying is that budget allocated to Division “A” to build an application that is at least partially consumed by using other services in Division “B” will create a horizontal shift in that budget.

If project “A” gets a million dollars to build a project, but $200,000 of that million goes to the producer in a different division to maintain an existing service, that is a net decrease in the budget of that project and a net increase in that other division’s budget. This will create that competitive marketplace.

Those who produce the best services and get them leveraged through reuse will become more critical to the business overall, and will be able to demand more budget; and will, in fact will be getting more budget. Those who do not over time will get smaller and smaller budgets. Even if the real dollars don’t change, the net dollars do - because there’s more shifting across the organization instead of direct build and maintenance of their own services.

One more way to look at it: If one division does a great job and three divisions do very poor jobs of producing reusable services, those other three divisions will ultimately become more business process level development teams while that first division will not only build their own applications but be the primary authors of all the core services. Thus the budgetary allocations will reflect that.

So when I mentioned this to Jason Bloomberg some time ago, he equated it to a “survival of the fittest” SOA mentality. I don’t think of it that way; I think it’s good old market dynamics. It’s healthy within an organization for those who do the best job to be empowered to do more and for those who don’t to reap their reward as well.

SOA CoE PDQ

If you built a CoE (Center of Excellence), every time an analyst told you to, there’d be more CoE’s than there were desks in the IT Dept. But of all the areas where a CoE might be most critical, an enterprise-wide SOA initiative would be one of them.

In most organizations, vertical trust up and down the chain of command is relatively easy to come by and already exists. The line of reporting structure typically creates the sense of reuse, standardization, and the ability to leverage vertically within an organization.

But at the highest levels, horizontally across peer groups, that is usually not the case. Ironically, a lot of organizations I’ve seen tend to foster distrust and create an anti-collaboration spirit between those horizontal peer groups. We’ll have to resolve this for SOA to work. That is why creating horizontal trust is such a critical aspect to SOA adoption.

Grossly generalizing, when bad things happen within an organization vertically, the thinking is ‘hey we’re in this together so let’s work it out’. That same issue occurring across an organization horizontally would more likely sound like ‘those guys are morons -- I’m not going to make the mistake of relying on them again!’…

One of the ways that it’s possible to combat that trust concern, and instill a sense of trust, is through a center of excellence around SOA. To instill trust horizontally and make SOA adoption possible, we need to use an analogy from a States’ rights vs. Federal rights debate. The individual divisions need to consider themselves autonomous on certain levels, and bestowing certain rights to the Federal level; but they also have the opportunity to participate at that Federal level, so that it isn’t just a “Down from Above” edict that will immediately create a defensive reaction instead of an embrace.

So, if we think about what the SOA CoE must look like then, certainly there needs to be a set of participants that are not beholden to any particular one of the divisions involved, but there also needs to be significant membership from all of those who will be involved so that we get the participation – this isn’t US vs. THEM – this SOA CoE is a self-governing model.

This organization will model what policies need to be so-called Federal Policies and which can safely be handled at the State level. For example, we talk about policy that can be automated along the three domains: structural, behavioral, and performance. It might be quite accurate to say that at the Federal level, structural policy (or compliance) might be the most important aspect, while from a State-to-State kind of policy, behavioral and performance policies will perhaps be most important to define at those levels.

Take that one step further: SOA Governance needs to be explicit about how to publish a service, and precisely what is involved in consuming an existing service. Whether it’s within the same division or across divisions, the behavior and performance policies need to be authored, automated and enforced.

I hope this has made some sense in that without this kind of CoE, we are in a very difficult position when we are trying to create that horizontal trust. We must create a better way to provide SOA visibility across the organization, which is historically not very easy to do.

Consumption is NOT Free

I recently blogged on the topic of an SOA marketplace dynamic that I believe will take shape over the next many years. It reminded me that I never have articulated my case for the fact that consumption of services is not free, so it shouldn’t be free.

Here’s what I mean: If I create a service for my own internal use, the certification effort (the level of structural, behavioral, and performance policy) will not be nearly as high. Over time my maintenance of that service will be much lower. There will be much less risk in changing that service when I have a well known, finite set of consumers that are under my control as opposed to a broadly leveraged service that will have a widely distributed use among lots of folks that I’m not even aware of.

All of this contributes to an increased cost of producing a service that is robust over time and reusable. So for the producer, consuming the service is not free. Given that, the consumption itself should not be free. There must be a reckoning of some sort when a project team gets the benefit of an existing reusable service that is properly maintained and therefore increased in its own cost for doing so. That reckoning has to occur and if it doesn’t either one of two things occur:

• Either we penalize the production of robust services and, therefore, we don’t get as many--
  
• Or we produce reusable services that will die on the vine as it were.
  

The increased burden without an associated increase in the budget for that service to be reused will actually threaten that service’s long term quality and adherence to policy.

So in the end, consumption of existing services does have a net decreasing effect on the cost and time of producing a solution. We’ve got to love that – but we can’t ignore the fact that it doesn’t come without cost even if it is less, much less than building from scratch.

All this means consumption isn’t free.

Governance Tools -- Buyers Beware

Enterprise software has a sometimes-sordid history of greatly over-promising and under-delivering. Some vendors I see in the governance space for SOA are repeating that bad history. That’s unfortunate given the tremendous promise in SOA and the tremendous capabilities that are possible for SOA governance.

No one vendor, even my own company iTKO, can provide 100% of what I would think is a robust governance solution today. It may never be that one vendor delivers 100% of a governance solution. Yet I’ve just been reading many vendors talk about their complete robust end-all-be-all governance solution for SOA.

While several vendors have a fairly common definition of a Governance, it’s entirely too inadequate to really suffice for a complete, robust, or end-all-be-all solution. In fact most Governance solutions right now in SOA are little more than bundling WSI tools and scanning XML with Xpath queries at run time for automation, and document repositories for non-automated Governance.

This is barely the beginning of what will create real governance. The outcome of a robust governance strategy is increased visibility, trust, and an infrastructure that has longevity compared to the many times we’ve attempted greater reuse and better IT alignment with the business.

One of the key areas that my company focuses on around governance is in the behavioral area and in making the definition of the word “policy” more rigorous than the meanings frequently used by the so-called governance tools also in our space.

At the end of the day, policy will need to be all of the kinds of things that you think of legislatures doing or any kind of rule-making body in our real world. Think about at the Federal level, very broad stroke kind of Policies. These are the things that some governance tools can do like “Thou shall use WS security for all of the access points from external sources coming on to our network. But there is also the State level, County level, City level and even Home Owners’ Association, if you think about it. You will never see a Federal Law about where I’m allowed or not allowed to place my US flag. But my Homeowners’ Association has all kinds of rules and restrictions about where I can and can’t place my flag.

So policy is really going to be where we, as an industry, focus Governance tools and within that focus, we’ve got to make sure that our definition of policy captures enough of the “what it takes” to get the value from Governance.

So the next time you are talking to a Governance tool vendor, let me suggest two things:

The first is to make them fully explain themselves about what their product does in terms of allowing the definition, modeling and enforcement of governance policy.

And, second, don’t just listen – make them show you.

We have a customer that spent 7 figures on a tool that was supposed to do a bunch of really interesting policy and governance work that’s barely doing much of that at all. You’ve got to see how the product works.

Governance buyers beware.