The ITKO Blog: Service Virtualization and Application Lifecycle Optimization - CA Technologies

Will SOA Fold into SaaS? Will Quality Happen Along the Way?

I'm on my way to STARwest in Disneyland, CA today, and since it's been a little while since my last software quality-focused conference, I'm particularly interested to see how these testing experts and vendors are addressing the needs of a more distributed computing environment. Our own expert Rajeev Gupta is coming in to present "Building the SOA Quality Center of Excellence" at 11:30 PT on Thursday Oct. 2.

Joe McKendrick recently posted, CTO Predicts SOA Will Fade into SaaS.  He quoted Ron Huxter, CTO of the provincial government of Ontario, “Over the last ten years, we’ve got most of the common business practices identified. The issues that we’re coming into now are sourcing the actual services -- in two to three years, SOA will fade into yet another sales banner like everything else has and it will transform into software-as-a-service (SaaS).” Ron acknowledges that new issues in cost structures and security will go along with this transition.

Joe concluded that, “Every paradigm, every change that sweeps the IT space promises greater integration, ease of use, and cost efficiencies. However, they also introduce new vulnerabilities that need new types of solutions. The rise of SOA, and now melding with cloud computing, is no exception.”

This is certainly the case, whether or not SOA is a leading indicator of SaaS.  However, it will take a while for SaaS in a Cloud Computing model to catch up with SOA  (see ReadWriteWeb’s post -  Fastest Growing US Companies are Rapidly Adopting Social Media for a presentation of this particular Gartner Hype Curve.) One obstacle for the pure plays in the cloud to join SOA is to recognize and deal with the increased testing and validation of data from even more diverse sources, as well as the governance and validation of all the underlying technologies that contribute to specific business outcomes.

SOA and Cloud Computing Both on the Rise but at Very Different Parts of Gartner Hype Curve

ReadWriteWeb recently posted an interesting piece on a research showing that the Fastest Growing US Companies are Rapidly Adopting Social Media. What interested me the most was the Gartner Hype Curve for Emerging Technologies 2008 that was presented at the bottom of the post.  It has a July 2008 date. As most of you know the curve starts with the rapid climb to the peak of inflated expectations, followed by a dive into the trough of disillusionment, and those that have market legs get up the slope of enlightenment and reach the plateau of productivity. A technology can fall off at any point.

Both SOA and cloud computing are on the rise. However, SOA, having been around much longer is climbing the slope of enlightenment. Cloud computing is still on its way up to the peak of inflated expectations. They are both given a 2-5 year to mainstream adoption. I think that cloud computing can learn from SOA, its more advanced technology cousin.  They are starting to become connected and one lesson is the importance of testing and validation as I wrote about recently, continuous testing and validation of all these combined data sources from the cloud mitigates the risk of invalid data getting mashed up in your workflows, and your business rules going off track. It will be interesting to see how the cloud adopts this SOA quality lesson.

What we've found is that there is a bigger constraint to establishing complex, critical business workflows, whether you are calling it SOA or considering a cloud model. The bottleneck for SOA is one of availability. What do I do if I cannot develop and test my intended functionality against the services I depend on? I need a virtualized model of the services, mainframes and data I depend on to build my own successful workflow.

Here again, SOA's need for Virtualization of application behaviors is a canary in the coal mine for Cloud concerns, where your dependencies are even more abstract and ephemeral. In relying on Cloud resources, there is a lot of variability of data - making it hard to hit development targets without a "virtual cloud" model in place. We'll address this topic in more detail soon.

Virtual SOA Gov Conference September 24

This full day virtual SOA conference will cover the full range of governance issues and solutions including IT governance, SOA governance, data governance, registries, repositories, and run-time service governance. We are pleased that ITKO’s John Michelsen will be part of a Panel Discussion: "SOA Governance and IT Governance" at 12pm EDT. The complete panel includes Anne Thomas Manes, Ronald Schmelzer, Frank Martinez, David Bressler, Ed Horst, and Joe McKendrick.

We have already had some very interesting briefings and discussions with this group of analysts and industry peers. If you have any questions about SOA Governance, Policy design and enforcement, Agility, quality or management of Virtualized Services, this would be a great time to ask them.

There is more detail on the panel members, the other events, and the ability to register for the free event at the virtual SOA Conference site. Some iTKOers will also be manning a virtual iTKO LISA booth on the virtual Expo, to answer any followup questions in a more relaxed environment, or supply whitepapers and other downloads.

*Note: the SOA Gov e-Conference is now available post-event in Archive form at the link above.

SOA Governance: Separating Success from Chaos – Aberdeen Report

The Aberdeen Group just released a report, SOA Governance: Separating Success from Chaos, that speaks to the heart of some of the most critical issues in SOA success.  We have written about SOA governance many times on this blog. As the report pointed out, SOA governance expands on the notion of IT governance, because of the increased complexity and interrelationships between services. To get real business benefits, proper SOA governance needs to be set up from the start. At the same time, the greatest challenge they found for managing SOA was the establishment of security, governance, and management. Aberdeen looked at what best-in-class SOA companies are doing in terms of governance and related issues.

First they defined best-in-class SOA companies (the top 20%) as those that achieved these four business results:

• Reduced costs from re-use of existing services (23% on average)
• Reduced time needed for IT to respond to business demands (18% on average)
• Reduced application development time (23% on average)
• Increased the number of services re-used. (37% on average)

The results showed that those enjoying the best-in-class performance shared several characteristics.

• 82% provide training to SOA project staff
• 78% establish and monitor Service Level Agreements
• 77% monitor efficiency

SOA governance is implemented to do more than reduce costs. That was actually the number three driver after the desire to increase business agility and optimize business and IT alignment.  Drivers four and five that followed cost reductions, were to better respond to business demands for new applications and increase the effectiveness of existing applications. So while cost reduction is often a selling point for SOA, increasing business performance seems to be emerging as the major rationale.  This more strategic focus requires the attention and guidance of senior management -- obtaining the commitment of senior execs to SOA implementation was seen as the top strategy success factor.

Companies are also using SOA to help with risk management, which is very important in today’s economy. The increased visibility, decreased time for application development, and established management tools mapped to business processes that effective SOA governance provides can help organizations more efficiently response to business or regulatory changes.

The report provides many specifics on what best-in-class companies are doing to achieve the goals and the business results they obtained. I also found it interesting that, in addition to governance concerns, the survey also found that two of the top six SOA challenges were related to testing. You can view the complete report at this link to SOA Governance: Separating Success from Chaos. (It is a protected document, so you can Print it to paper or a PDF if you want to read it offline.)

Dana Gardner Interviews John Michelsen on Roles and Methods for SOA Validation Across Complex Integration Lifecycles

Recently, ITKO’s John Michelsen was interviewed by Dana Gardner of Interarbor Solutions and ZDNet on his BriefingsDirect.  The session covered integration, validation, and testing for services-oriented architecture (SOA) and middleware -- particularly for business process management and to extend business processes more efficiently. As Dana said, “the whole notion of integration in the enterprise has exploded in terms of complexity -- but that puts more onus and importance on validation, testing and understanding what’s actually going on within these integration activities.”

John talked about the more rapid changes going on today to get to market faster, achieve regulatory compliance faster, etc. All this changes and the increasing interdependence between applications is creating many more ”testable” moments. This is providing increased opportunities for ITKO LISA.  Our growth rate is an indication -- as an IT discipline – the market has finally realized that companies have to get testing and validation right. You can listen to the complete conversation between John and Dana at this link to the Gardner - Michelsen BriefingsDirect podcast. Dana also commented on the interview in his post, iTKO’s SOA testing and validation role supports increasingly complex integration lifecycles. where you can also find a link to the podcast.

A full transcript is also available here at BriefingsDirect Transcripts if you'd rather read...

At VMworld - Provisioning Plays in the Virtual Box

I've been here at VMworld and I just walked every booth on the floor. A few notes: Half of the exhibitors are all about the monitoring and management of VM images. Lots of desktop virtualization companies offering a way to look inside the performance or operation of the VM "boxes" running in your environment. It is literally like everyone realized that a management/metrics dashboard for VMs was missing at the same time.

Several companies offering I/O virtualization -- which is a way to provide one big pipe and manage the provisioning of the users of that pipe. So for instance, we have a 100-meg connection to the Internet, and some team wants to consume some of that, so we need to provide a "virtual T-1" on top of the real 100MB ethernet... saving the dedicated capacity of that bandwidth "box."

Application virtualization (aka "Endpoint Virtualization" by some) is nice in that you aren't just getting a whole box, instead you get a set of apps that are provisioned just for you.  So I log in and get a programmer's IDE but our sales guy logs in and doesn't -- in fact he's not allowed near it ;)  Cool way to provision team members with their individualized boxes.

Virtual Test Lab vendors were there too like VMlogix and Surgient.  I don't know how many of these attendees are concerned with virtualization in development lifecylces but this is clearly an important space for virtualization in the larger context.

What's new? Skytap and other Cloud platforms were there, mostly grouped in the back by the fire escape in case there was a virtual battery fire. What is interesting is that VMWare has competing product with just about every software vendor in the building, save cloud provisioning, and I'm sure that's coming out soon enough.

I turned every corner looking for the banner that said "What about all those systems you can't image in a VM instance?"  Of course that's our offering with LISA Virtualize. So my reaction is that the market is still digesting the fundementals of virtualization.  It's a very operations focused show.  That's not "wrong" -- it's just leaving a lot of what virtualization can do on the table. I'm looking forward to virtualization concepts pervading the whole enterprise.

Turning SOA whining into debate, planning, and action

Joe McKendrick recently wondered if there is too much "whining" and not enough action on SOA? in a ZDNet post. He was responding to post by Nicholas Petreley of CIO.com on Less SOA QQ and More SOA Pew Pew.

Let me translate: QQ means whining and Pew Pew means action. To be more specific the urban dictionary offers some definitions of Pew Pew: Sound made by lasers, usually related to star wars.  And  also a common term in the world of Warcraft most typically meaning to do damage, or perform a combat action.

So is Nicholas telling us to ready-fire-aim?  Joe points out that there is already a lot of SOA action. He adds, "there is nothing wrong with a good rousing debate on what SOA means for an organization. Perhaps there should have been more debates like this before companies invested millions in ERP and CRM systems in the past."  So if whining becomes debating, this can be a good thing.

SOA can create agility and some incredible efficiencies, but there are a lot of issues to resolve when plunging in, unless you want to do a lot of pew pew (e.g., damage). For example, because of the complexity that comes with orchestrating multiple services, work out governance issues in advance.

Still, the QQ (whiners) about SOA may be surprised to find that they have development and integration teams that are already soaking in the SOA pool - creating services-based architecures - even if the IT management isn't classifying the efforts as such. We have seen this phenomenon at many large companies.

 

We also suggest looking at three key issues to consider before attacking an SOA strategy.

• First, acknowledge the fact that this is a heterogeneous layering of technology, so plan to verify outcomes in other layers and systems of record beyond the one you are building and testing.
• Second, enable more parallel activities to leverage assets more rapidly and take advantage of SOA. This is where Service-Oriented Virtualization comes into play -- if you can prune off those interdependencies upon other technologies by Virtualizing the rest of your architecture, it enables each team to be productive throughout the lifecycle.
• Third, we cant test as an event along a life cycle anymore, and think that this is enough. You need ongoing Continuous Validation, because services are coming from multiple sources which might change. Without this validation capability, it is impossible to know if your SOA Policies are actually being carried out in the real world, as there are multiple changing elements that can affect the workflow.

The factors can help with the SOA debate -- which is a better approach than either QQ or pew pew.

Parsing The Cloud – Adding a Fifth Dimension

Dan Woods provides an interesting analysis in Forbes, Parsing The Cloud, on where Cloud Computing may be going next.  He makes a good case that the cloud will become more complex and diverse. He offers four drivers of this diversity.

First, compliance is going to happen. Governments have already stepped in. The Swiss banking laws dictate that computers storing customer information must be housed inside its borders. Other countries may follow so they can tax, inspect, or otherwise exert control.

Network topology will also affect the speed of data access. Different industries have different speed requirements and this may lead to the market verticalization of clouds.

Levels of services and associated costs will be another driver. Dan suggested that it is even possible that companies wanting the highest quality of service and the least risk of failure will hire a cloud company to create a private label cloud for them.

Dan suggests that a fourth driver will be application and tool expertise. Clouds will develop to run specific types of applications, such as CRM, ERP, supply chain, manufacturing, retail applications, etc. This would further enable teams to move focus away from managing applications, and focus on managing the processes themselves.

However in reading this I would add a fifth item to the menu for cloud apps: reliability. What is the level of testing and validation? How do your know the data coming in is accurate? Are continuous tests part of the process?

With Cloud Computing thought already spanning to government compliance issues and "right sizing" on a per-industry, and possibly per-organization basis, we are going to see a much stronger need to verify that our use of cloud resources is meeting specific business needs in a responsible fashion.

OutlookInterview Part Five: Conclusion and iTKO Overview

Here is the final installment of my recent interview with host Mike Lippis of the Outlook Series. We have broken the 45 minute podcast interview into five parts for easier reading. This section covers ITKO and concludes the interview.

Mike:  “Can you share a little background with the audience about iTKO’s general ecosystem, and as part of that could you give us have a use-case that you could share with us about how a given company may have used this LISA solution and benefited from it?”

John:  “I can even try to tie both of those together for you. 

“Our product LISA is especially suited for these complex heterogeneous type environments where there’s a variety of technologies at play, and the demands on these applications are extreme. That’s obviously companies like the financial sector and the insurance sector, and so we’ve done really well there, and in logistics and travel and a number of industries.

“For example, as in the US Department of Defense where they have the ‘everything’ problem, you layer in security and you layer in a lot of complexity there obviously, and a variety of different technologies and lots of legacy integrations.  We do really well when the problem gets harder and harder.  You know, any time your testing and your validation tool are trying to dumb-down the problem so they can get it tested … you should realize that this is signing you up for the harder part -- so that’s why we’ve gone to the approach that we have. So what’s that done from a customer base, has given us the most difficult areas of the largest companies and that becomes our target market. 

“We get partnerships from two different types of companies.  We get partnerships in the system integrator types, like Accenture, and Perficient who resell our product in areas, or any number of the largest outsourcing companies like Infosys, TCS, Patni, Satyam, Cognizant, Wipro, Mindtree, and I apologize to those listening who I haven’t mentioned -- I love every one of you guys so please don’t get me wrong… 

“From the services side:  we get a lot of help and that’s a major piece of our ecosystem because, quite frankly, those are the folks that are responsible for making something work for their customer and they are looking for the best practices way of doing this, and so they seek us out in many cases to do that. 

“Then the final part of our ecosystem is the actual ISVs -- the vendors who produce these SOA platforms, people in the integration space [e.g. TIBCO, Software AG, SAP, Oracle, Progress, etc.]. Of course that’s been a really good area for us as well, because again we’re uniquely suited in many cases to be able to do the proper testing, validation and virtualization of their platform.

“Just as an example of that:  TIBCO themselves use LISA throughout R&D in TIBCO to do the test and validation and virtualization work, so that they can decouple the independent components -- all of these different products that can be put together to create TIBCO solutions. They need to test them independently, and they need to validate them independently and so they do that and use us in all three ways.  Of course that helps TIBCO customers in many ways. 

“A number of our customers are on TIBCO, and on webMethods [Software AG] and on and on (here I go again when naming one I better be careful – (I love all of you guys so please don’t get me wrong) -- and a number of software companies are our partners because of that.”

Mike:  “Is there anything else that you would like to add to our discussion before we wrap up for today?”

John:  “If there was just one thing, it would be:  let’s bring it all the way back to the beginning and realize that this is really about a very key condition and that’s RISK.  That’s business risk and production outage. 

“I have yet to see the customer who is dramatically reducing their level of production outage, in fact it’s going up in many cases and this is a serious concern. Many companies that didn’t even measure risk as much in the past are now obligated to, through compliance regulations.  Because of this they are seeing that so much of their budget is caught up in maintaining the things that are already running, and the expense in keeping it limping along is so great they need to do something about it. 

“That is what we mean by this risk of failure that has been introduced; by so much more complexity and so much more rapid change.  That is why we are in the space that we’re in.  Every time you go SOA -- or every time you introduce some large-scale integration project -- or every time you bring in a new ERP system or you merge with another company -- you are agitating both the complexity and the pace of change.  If you do that without a consummate reaction on the risk mitigation side, that’s why your failure rate is going up, that’s why the cost of maintaining systems is going up.

“So for every one of those activities you are doing, you need to respond with a better, more appropriate, best-practice driven test, validation and virtualization process.  With that you’ll accomplish the change you are trying to make, and you’ll do so without increasing that level of production outage or cost of maintenance.”

Mike:  “Where can our audience go to learn more about iTKO and your LISA initiative?”

John:  “All things LISA and access to white papers and stored webinars and all sorts of things you can find at our web site which is: www.iTKO.com.  And I blog quite frequently and a few other folks with me blog on: blog.itko.com.  You can certainly look at what we’ve been talking about there and you can comment; we’d love to get back to you.”

Mike:  “I certainly appreciate your joining us today, and wish you the best of luck with your future endeavors.”

Outlook Interview Part Four: Virtualization and Testing

Here is part 4 of my recent interview with host Mike Lippis of the Outlook Series. We have broken the 45 minute podcast interview into five parts for easier reading. This section covers virtualization for SOA and how it affects the development and testing teams' productivity.

Mike:  “With this virtualized service edition of LISA.  What are you doing in the area of parallel development and test cycles?”

John:  “What we do is enable that parallel development, because we’re trying to decouple those service providers and consumers from each other during their lifecycle. 

“So a very reasonable approach towards meeting this kind of objective is that I might go through a requirement phase and a design phase and as I start development, I can discover my dependent systems, virtualize those dependent systems and then execute my development plan against my virtual services.  When I do this, I am decoupled from all of those constraints on my peers or on all those other services.

“The reason that I do this is because:  we’ve all heard our development leads say, ‘by the way I haven’t been able to do anything today because the “so & so system” is down.’  Or, ‘remember…they gave us a new build yesterday? Well the new build doesn’t work … they broke one of the major operations. So my team hasn’t had any productive time during the last two days.’ 

“We hear this all the time … this live connectivity to everybody (remember that white board I just mentioned a minute ago?)  Live connectivity to all of these boxes creates a large amount of friction or construction barriers, if you will, in the road. 

“What we can do, by isolating ourselves from that through virtualization, is we can parallel that development activity and, of course as a part of our final integration testing we go from virtual back to live services.  Once we are all live, we can obviously make sure that everything inter-operates the way we expect.  The value here is that instead of having to wait for my dependencies to complete, I can run in parallel with my dependency and get it done faster. 

“A quick example of this is:  If I’m asking you to implement a new feature in your service, I have two choices:  I either ask you to make that change and wait for you to make that change, test that change, produce that change into a build for me and make it available, or I can use a virtual service.

“I can make the change in the virtual model very quickly (we are talking minutes to do this with our product) … and I can be developing in parallel against the virtual version of your service just an hour or two later.  Meanwhile, you’ll take weeks to produce a new version but I’m developing against the new version in parallel in a virtual world.  Once your service is complete, I can point my application away from the virtual edition, and instead to the live edition, and we can make sure that it actually works as we expect.”

Mike:  “When it comes to testing in this area, we certainly want to simulate as many data set combinations as possible to see how effectively that this solution is working; how robust would release be in that area in terms of being able to simulate data set combinations?”

John:  “We’ve spent a lot of time on this.  It turns out … as we embarked on this virtualization strategy, we knew that virtualizing data layers would be a big thing, but I can’t tell you how many times I’ve heard our customers say “this is the big thing.”  It’s because of all this data synchronization, and all of this volatility of data – every team wants to reset the database every time they run their test and every time that they do a development cycle, and that means that no two teams can be sharing the database at the same time, and it’s always a mess!  We’ve spent a lot more time here than we’ve even thought and we’ve solved a lot of problems here.

“A classic example of this is:  When you are working on the purchase ordering system, and the supplier system is your partner, you don’t actually produce the supplier’s service, but you talk to a supplier … and yet you’re supposed to build an application that tightly integrates with your supplier … how do you do this?  Well again, you virtualize either at the behavioral layer or the data layer.  And, why do you virtualize at the data layer?

“Think about it … the supplier could respond to a PO request by supplying all of the requested goods at the requested date with no issue; but the supplier could just as easily split the order, could partially cancel, could respond to you that these items were no longer carried, could warn you with a credit check, could warn you … The number of possible data scenarios in the response to a requested PO -- it reaches the point where we wonder how we can possibly test all these out. 

“Since we can’t drive data scenarios in live systems, we can’t drive data scenarios with our partners -- at least we can’t effectively -- we need to virtualize that database so that we can drive all those.  I can make certain that every kind of possible response to a PO request is available to me all of the time.  That’s what data virtualization does for us.  It makes me able to make certain that every time I need the stars to align a certain way for me to make a certain condition occur, I can do that.  I’m certainly not strong enough to literally make the stars align but, in a virtual world I can, so in that way I can always make sure that everything works the way I expected.”

Mike:  “There are a number of load and performance testing tools such as LoadRunner or SilkPerformer, and a lot of companies have made investments in those technologies, so how can they leverage those investments that they’ve made and expand on those with LISA Virtual Service Edition?”

John:  “That’s absolutely true Mike, there’s plenty of successful LoadRunner shops and SilkPerformer shops and others, and LISA does load testing, but we’re certainly not interested in trying to fix what’s not broken.  We’re certainly quite comfortable with working with companies who are doing load testing with LoadRunner.  A number of our virtual services environment customers are doing exactly that.

“In fact, if you picture in your mind:  the left-hand side of the graphic is Load Runner generating a load on a system in the middle of this graphic and on the right-hand side of this graphic is LISA Virtual Services environment.  What we do is enable more and better load testing even with the tools that we did not provide.  What we’re doing is we’re freeing the system that you’re testing from its constraints.  We’re not eliminating that system nor are we trying to affect the load test product itself.  We’re actually trying to virtualize away the constraints on the right hand side of the graphic. 

“So, it’s LoadRunner, versus the System Under Test, which can be LISA … and that triad, if you will, makes for a very powerful load testing combination.  What we have found is that you have to have production hardware or at least a reasonable production performance lab for your own application. 

“Our customers understand this, what they have trouble with is:  ‘Wait a minute, my application no longer lives on its own island, it now goes and invokes its own transactions on a main frame or it now integrates between a TIBCO-based system or this web message-based system and it goes and talks to this ERP system.’ 

“In order for me to performance test my application, my performance lab just went from a half million-dollar investment to a five million-dollar investment … and that’s what they can’t do.  So when you go Load Runner/Your Application/LISA, you eliminate the five million-dollar investment. You literally eliminate it.  What you go from instead is the half million-dollar investment in your hardware platform -- what you have to run your application and your service on, and we virtualize away all the heavy iron that you have to connect to.  We put that on a commodity box, what would be a rounding area on a half-million dollar investment -- we would run on a $20,000 server.”

[This concludes part four. In the next post we wrap up the interview and provide an overview of ITKO.]

Outlook Interview Part Three: Performance Management and Testing

Here is part three of the interview with host Mike Lippis of the Outlook Series. We have broken the 45 minute podcast interview into five parts for easier reading. This section covers performance management and testing.

Mike:  “When it comes to really achieving business value from IT investments, lots of folks will really look at performance.  Can you give us some background on some current performance management strategies that are out there?”

John:  “I can compare that to what our traditional approach to performance management has been and again, if in the old world, entire applications were produced and then they were performance-tested in a lab and released for production.  This worked okay when the thing we built was a 100% self-contained thing. 

“But now back to an analogy of a service-based architecture, where I am producing applications that consume services from you, Mike.  What do you do to performance test or to manage the performance of your service?  You are no longer a complete application.  I can’t ask you to produce your service, and send it individually into the performance lab if the performance lab hasn’t retooled?  I’ve got to think about performance testing at the services layer.  I’ve got to think about the performance testing even at the orchestration layer.  By that I mean there may be two or three services that collaborate to make one business process or even one sub-process of the business function. 

“Once we start to think about performance management, now that I’ve broken my big box application into a number of components; I’ve got to do the same thing with performance.  I’ve got to break my run-time SLA into the component-level SLA  (Service Level Agreement) that’s basically my performance and resource expectations that I’ve put on my applications.  So, Mike, I might need to do something that we call decompose the SLA where my application must run at 2 seconds or less response time for the user.   

“If 2 seconds is what I have -- I need to get your performance under 2 seconds so that I can meet my SLA.  Your response time is a component of my response time, because I am invoking you -- I am dependent on your service.  What I need to do is negotiate (for lack of a better phrase) your SLA with you--we call that decomposing the SLA.  You need .5 seconds, then my application can consume 1.5 seconds, and therefore, I can meet my overall SLA.  This is in many ways a new way of thinking about performance than previously.   We used to just think about 2 seconds -- we dropped the application in a lab to see if we had 2 seconds.  Now Mike’s team has to go to the lab and has to meet .5 seconds, and my team has to go to the lab to meet not 2 seconds but 1.5 seconds. I’ve got to make sure that my 1.5 seconds will combine with your .5 seconds.
Make sense?

Mike:  “Once you have an application completed, a lot of times they are difficult to tune for performance.  What are you specifically doing to address that?”

John:  “This notion of taking our SLA apart into components actually helps us do this.  It does for a couple of reasons the biggest is:  if we can performance test within the life cycle, if we can do our testing as a part of our own development than we can find and resolve our performance issues much more quickly.  In my experience, and I bet many people in the performance field will nod in agreement, so many times if we build an app that does not scale and then try to tune it into scalability, the best we’ll do is 10 or 20% improvement.  You can’t add hardware and make a product that is un-scalable all of a sudden scalable--it just doesn’t work.  If you had known earlier where those scalability issues are--than you can build in the scalability along the way a whole lot easier.

This is essentially the same reason why we iterate from a requirements definition standpoint, because the more we iterate thru the scope of our work the better we are at delivering on it and the faster we are   The same is true with performance, we have to iterate and we have to do performance testing at the component level in order to get that performance up and the scalability up.”

Mike:  “Are you doing anything to address the challenge of discontinuous development cycles?” 

John:  “Yes.  Because of the fact that you may be building your component at a different lifecycle pace than my application even though I am a consumer of your component, (usually we call that service) our challenge is that I got to have access to your service maybe at a time you don’t even have it ready for me.  You might have down stream dependencies of your own.  As soon as I think of you as “Order Management” -- you might turn around and say: Yes, I do order management but I sit on top of the customer database that I don’t own, and I sit on top of the ERP system that I don’t own either.  So, you have your own set of dependencies.  So if we actually fit it on a white board, most large-scale enterprises would literally fill a white board with very small little boxes all interdependent on each other and they are all iterating on their own life cycle.  It drives me nuts just thinking about how anything actually works!

“So in order for us to support this kind of distributed lifecycle, we’re doing a lot more virtualizing of the components underneath each other. 

“What we mean by this is:  since I am invoking your service, I am dependent on your service, but what if your service isn’t available to me yet, what if I need you to change your service and your changes aren’t ready yet?  What if I need to do performance testing of my application on an interim basis but I don’t have scalable production quality hardware to run your service on?  Or, maybe worse, maybe your service talks to the mainframe, but I don’t have availability of mainframe resources to do this?  All these are what we call constraints.  These are real world constraints which we are dependent upon. 

“What we do is:  we virtualize the behavior of these things.  What I will do is construct a model (our LISA Virtualize product is really good at doing this) -- it constructs a model of the behavior of your service, and once that model exists I can run that model in a virtual environment, it doesn’t have to have the real world constraints; it’s a virtual thing.  So, if I need it early, I’ve got it. If I need it to scale to 500 transactions a second, I can do that. Even though I don’t have production type hardware for your real service, I can virtualize it and get the kind of performance that I’m looking for.” 

Mike:  “It sounds like you would be able to provision the right hardware and platform for a given solution then.”

John:  “Right, because what we do is that we dramatically reduce what that requires.  A classic example for one of our customers is:  I’ve got an application that is a portal-type application that invokes a set of services.  Some of those services talk to SAP.  SAP is a great ERP system and many large organizations use it; but of course we don’t want a lot of performance and load testing type transactions against that system.  We don’t always have availability when we need it.  We don’t always have the data in the database when we want to -- SAP testing is a very complex field -- so there are all kinds of real world constraints.  What we do is we virtualize the behavior of SAP and in doing so, your portal has access to what it needs 24/7 and I don’t have to provision the entire SAP environment to do it.  All I have to do is provision my own application, we can’t virtualize away you --but we can virtualize away your dependency. and once we do that, we decouple you away from a lot of the challenges that those constraints have.”

Mike:  “When it comes to performance labs, it seems like a lot of them really rate data management as a very big problem.  What can you tell us about dealing with that challenge in leveraging LISA?”

John:  “One of the great challenges with data in performance labs … is really two challenges. One is that you can never talk to just one database in a performance test. You are usually dealing with multiple data sources.  For example (as I mentioned a minute ago), order management is dealing with a customer database in one system and with the orders database, let’s say, underneath SAP -- so here’s two completely independent systems that are now interdependent on each other with regard to data.  Now here I am in a performance lab and I’m trying to support hundreds of scenarios of real world use of my application.  In order to do that, I need the data between those two databases populated in a certain way and synchronized between each other and whatever changes are made in one database, I need those changes to be reflected appropriately in the other database.  Do you know how often this happens? Much, much less frequently than Christmas! 

“What we have is a dynamic that we see all the time, you’ll go into the performance lab wanting to do some 100 different scenarios of use, these are all the use cases you want to be sure are performing and scalable in your application; but by the time you figure out just how to get all the data to synchronize between these two systems, the best you’ll come out with is about 20.  Now the problem is that you’ll performance test just 20 scenarios, but you’ll go to production with all 100.  You’ll find out the hard way whether the other 80 are a problem or not, because your real world users will be doing those other 80 (the one’s you couldn’t performance test) and the problem is:  if you have scalability or performance or resource issues in those other 80 … then you have a problem.”

Mike:  “When it comes to best practices in the area of performance and load testing, what can you tell us about decomposing performance expectations?”

John:  “I gave a quick example of this earlier, I’ll elaborate on this a little bit. 

“If the production SLA for a particular application; let’s say mine is 2 seconds…that’s the example that I gave previously … where my response time to my user is 2 seconds … but one of the things that my application does is to consume services from your team, Mike, I need to negotiate your response time as a part of my response time.  So what I do is decompose that SLA.  What I’ll do is say, ‘okay, I’ve got to make sure that I can get Mike’s service calls in a short enough amount of time so that my application overall’ -- once you add up all of the time consumed – it doesn’t trip over our expected response time. 

“So we help by giving the customer the ability to do two things.  The first is:  we’ll help them do performance testing at the component level.  Mike, you can performance test your service.  You can say, ‘hey John, I can hit my half-second response time metric independent of you.  I owe you the right functionality at the right performance.  I can functionally test this system to make sure it works and I can performance test my service (independent to the rest of the world) at .5 seconds so I have done my job.’

“Now, I come into the picture and I can do performance testing on my application and make sure I hit my SLA as well.  In order for me to hit my SLA, I have to take you out of the mix. That’s why we talk a lot about virtualization in the performance lab. 

“So second, I can simulate your service’s behavior and I can even simulate your behavior at .5 seconds.  I can say, ‘alright Mike’s virtual service; I want you to behave like Mike’s virtual service, and I want you to take .5 seconds whenever you respond to me.’  By the way, I can even say ‘what if.’  What if your response time takes .6 seconds, what does that do to my overall response time? What if you’re actually running a little faster at .4 seconds …what does that do to my response time? 

“Sometimes the fastest way for me to get better scalability is to negotiate you a little faster so that my application hits its SLA:  stuff like that.  So that’s why performance testing as a component and virtualizing married together really give you a solid foundation for performance management in SOA.”

Mike:  “We had mentioned provisioning before, how does that come into play in terms of optimizing provisioning relative to managing performance; should one be done before the other and how is that relevant?”

John:  “The relevance is that we need to be able to capture virtual services as a part of the provisioning activity.  Where in the past, I needed the entire SAP system to be available for provisioning, now I just need the virtual services that encapsulate the behavior of SAP; I need that to be the provision item instead of the whole SAP system. 

By the way, when you think about these large-scale applications, often there are not only many servers with a very specific and complex kind of configuration, but there are huge databases underneath these big systems.  So very frequently we have customers tell us that they can’t do development work against this massive system because it’s got a terabyte-sized database underneath it.  In order for me to get access to it, I have to wait till midnight to 2 a.m. because that’s the only time they will give me time on the system. When I try to do performance testing, they refuse to let me because in order to avoid impacting production, they limit the amount of non-production activity on that particular system.  All of these kinds of constraints are those very same kinds of things. 

“So, back to provisioning, if I need access to this particular database or this particular large application that spans many servers, I encapsulate all that behavior into one virtual service and I can provision that one unit to represent that entire set of systems.”

This concludes part three. In the next post we cover virtualization and testing.

Outlook Interview Part Two: SOA Testing Best Practices

Here is part two of my recent interview with host Mike Lippis of the Outlook Series. We have broken the 45 minute podcast interview into five parts for easier reading. This section covers SOA testing best practices.

Mike:  “Service Oriented Architectures really have been around for a long period of time, and it’s really not a certain thing but more of a style of doing something.  If it’s a style, there’s going to be different kinds of approaches -- lots of interdependencies and something that’s going to continuously change and evolve.  What are you doing in the area of SOA testing and addressing those kinds of structure flow issues?”

John:  “That’s a great point because you’re right; the concept of component-based applications has been around for a long time.  I’m going to show my age here, by the time we come to the end of this conversation, you’ll know that I’ve been around a little while… 

“Our approach is basically the same as it has been for many years:  let’s take business logic and let’s encapsulate it into objects so that they can be leveraged across applications.  Let’s start to layer systems by their responsibility or domain so that we can build applications that are specific in terms of their interface.  For example, I might have a voice response unit for telephone callers and I might have a call center application for users on the phone but I don’t have to rewrite all of that logic every time, and I can make changes to business logic more quickly that way.  We’ve been talking about that concept since the late 70’s.  What makes SOA different today is that we are accomplishing it more than we ever had in the past due to the commoditization of middleware.  The fact that there is highly available, very reliable and also very understandable and cheap middleware means that it is possible to build these applications today.  So we are actually accomplishing it more than we used to in the past.”

Mike:  “I’ve seen in some of your collateral that you talk about the three C’s for some time now.  Can you tell us what those three C’s are and educate us as to what you have been doing and kind of describing the problem in that framework?”

John:  “The 3 C’s are our framework for best practices for SOA testing.  What we say is that if you have those 3 C’s right, than you are in good shape for having sound SOA test strategy. And, you can imagine our product LISA is a great way to accomplish these three C’s.  However you do it, you need them because essentially this is the standard now, so how do you reach it? 

“The first of these three C’s is “COMPLETE” and it essentially acknowledges the fact that this is a heterogeneous layering of technology. We must support a concept that we call Invoke & Verify.  If I invoke at one layer of technology I have to actually verify the outcome I am expecting by going to a different technology layer.  For example, if I go to an ATM and I deposit the money in the ATM and if I trust the receipt that it prints, that’s not really testing.  The way I test the ATM is by calling the bank teller or by going to my web site and looking on the web -- I go around the actual device that I am testing so that I can prove the outcome that I’ve expected. 

“The second of these three C’s is COLLABORATIVE.  Collaborative basically requires us to do more parallel activities and to leverage assets more rapidly than we had in the past.  Agile and SOA are usually tightly aligned.  In fact, most CIO’s tell me ‘I’m going SOA so that I can become agile.’  They are very linked at the highest level in most of our accounts.  Because of that, we need to do more in parallel and more shared. 

“The third of these three C’s is CONTINUOUS.  This essentially acknowledges the fact that we can’t test as an event along a life cycle anymore, and think that this is enough.  The fact that I’m dependent on let’s say your services Mike, means that I may test my applications today -- deploy this Friday and the following Monday you update a service underneath my application and threaten or risk – there’s our word - risk failure of my application in doing so.   So our strategy calls for a continuous validation of functionality, performance and certain policies on a continuous basis, - that’s the
third ‘C’.”

Mike:  “You have a product that’s called LISA.  How does LISA specifically achieve these three C’s that you just described?”

John:  “Breaking it down a bit, we support all of those different technologies from a COMPLETE point of view.  Whether it’s web technology, Thick UIs, messaging layers, web services, older protocols like CICS and CORBA … all of those kinds of things; we support all of those different technologies from the same test case.  It’s very important that you be able to do this in one test with one product.  If you buy three or four different products to accomplish one test scenario, that is way too much effort and it is very difficult to try to integrate a bunch of testing tools to make one test.  It’s really not feasible to do -- you need to have all of these technologies covered in one workflow.  That is why we differentiate in many ways on the ‘COMPLETE’ C. 

“For COLLABORATIVE, we provide the ability to do Development type testing, (whether that’s Unit testing or Service level testing), Functional and System type testing, Performance Testing, even Functional Monitoring all as one unified test from one product.  We’ve got to leverage each other’s assets more.  Developers need to start the testing process, QA needs to consume and extend, if you will, with development based testing, etc. so that’s how we support the COLLABORATIVE.   

“For CONTINUOUS, we actually provide both continuous integration support so we integrate continuous integration-type tools … build tools like ANT and CruiseControl and all those types of things.  We also provide our own infrastructure for integration labs, for production, for pre-prod type systems enabling us to provide our own infrastructure for continuous validation.  Therefore, we specifically support those three C’s in our product development and design, so we can accomplish these best practices for our customers.”

Mike:  “It really sounds like SOA testing is a key part of overall IT governance then.”

John:  “You know testing clearly is -- it’s gotten much more relevant in the last few years.  It used to be just a step along a time line: ‘we make sure we test it before we let it go into the wild’ -- and that was just fine.  Today, because we have a greater degree of reuse expected, and because we have a more distributed development force -- where they are not even always us -- sometimes it’s ‘us & them,’ unfortunately, where I’m supposed to trust people I don’t even meet or know.  Because of that, the need to test has become greater and the possibility of failure becomes greater along with it.  So testing has become a much more critical focus of both development and even of governance.  In many ways test is the means to instill trust which is, of course, the goal for governance.”

[This concludes part two. In our next post we cover performance management and testing.]

Outlook Interview Part One: Current Business Trends and SOA Testing

I was recently interviewed on the program:  Outlook Series with host Mike Lippis, and we covered a variety of SOA-related topics, from Testing -- all the way to Virtualization.

You can listen to the 45 minute podcast interview here, but our team has also broken the interview into five parts for easier reading. Here is part one -- which focuses on SOA Governance and how companies are attempting to use this discipline to meet  business goals.

Mike:  “Can you share with the audience what are some business trends that might be driving any kind of shift in IT environments currently?”

John: “The business is why we are here.  So what we need to do is to align our initiatives, as an IT organization, toward what the business is trying to accomplish.

“Among these (initiatives) are to increase the agility that we have to their customers in terms of responding to regulatory pressure…new market competitive pressure, etc.; to reduce cost…to go global…to either be acquiring or to become a part of larger organizations (M & A is becoming quite common, especially right now), and then, of course, compliance is over-riding all of this in that -- even though we’re increasing the speed of the train, we need to be more careful about how we execute along the path.”

Mike:  “What kind of IT initiatives would be driving that besides things like SOA and virtualization for example?”

John:  “Those are certainly two of them, as the business is trying to reduce cost, we need to increase our leverage and reuse, and so Service-Oriented is certainly one way to do that.  To increase our speed of development we need to take a more agile process, we need to leverage and extend technologies more often than bringing in new siloed systems.  Reducing costs often is reuse, but it is also outsourcing and off- shoring and virtualizing things.  So there is quite a bit going on in IT in order to accomplish these business objectives.”

Mike:  “Sounds like there’s a new IT environment driven a lot by SOA and Virtualization.  Could you describe some of the downsides associated with these new IT environments?”

John: “For all of the goodness that we are trying to bring about by increasing inter-connectedness to leverage and extend -- reusing existing assets, and the improvements we’re trying to provide by increasing our pace of change, (we’re releasing software updates to our business users more frequently than we were in the past) -- those changes are more interconnected with lots of different systems than they were in the past. While those are good things; the unfortunate outcome is that we’re increasing our risk of failure every time we increase the number of times we do something and every time we increase the complexity in which we do something, we necessarily have an increase in the possibility of failing, so we have to address that.”

Mike:  “So, it sounds like you’re really looking at minimizing business risk in terms of what your overall mission is at iTKO.  Could you kind of drill down into that a little bit for us to kind of see how you are addressing some of those issues?”

John:  “You bet - and that’s a good point Mike because we frequently get asked: ‘If you’re in the virtualization space why do you also offer testing?’  Well, because the real business we’re in is helping to resolve the risk associated with service-based architectures and agile processes, and more generally increasing complexity and increasing paper chains.  So what we try to do is:  if we can help enable these positive effects without the risk; we’re doing so to shorten the time to market, to reduce the failure rate in production, to increase our recovery time when there is a failure, and in general then to reduce the total cost of maintaining our own ESB systems that we produce.”

Mike:  “When it comes to SOA testing which is a big part of what you do, what are some of the challenges surrounding SOA testing?”

John:  “There are a number; the one that’s most common is that SOA increases the likelihood that we just can’t manually test -- we have an inadequate level of testing in many organizations from an automated standpoint and this becomes the governor on how frequently we can make changes to our system.  If I have to manually revalidate an application any time anything changes, then I can’t change it fast enough.”

Mike:  “I would think that today, based on what we’ve discussed so far, in terms of all the complexity, you really have a lot of heterogeneous environments and or most IT environments are very heterogeneous in terms of the technology that‘s now being used.  So how does this come into play with the SOA and what you are doing?”

John:  “Well it definitely scratches that problem quite a bit.  The reason it does is because:  Instead of thinking: my application is this completely self-contained box that I deliver to my users and they use it directly; rather, many teams are now building components of an integrated system and those components are, (as you say yourself) in heterogeneous technology, where there may be existing legacy CORBA applications and main frame applications and even mid-80’s mainframe messaging architectures, and modern messaging architectures, and all of these things being brought together for one application.

“So, we have a real challenge here when we try to test a system like this, we can’t just test each individual box independent of each other (although we should do some of this certainly) we must be able to test across the work flow as the business executes, so we must be able to test across these heterogeneous boundaries.”

Mike:  “You had mentioned earlier-on that IT initiatives, such as outsourcing and off- shoring for example, are very common in today’s global economy, how does this come into play with collaboration and to what extent does this frustrate collaboration; what are you doing in that area to make that a more seamless exercise?”

John:  “So the short version of a very long best practices story is that it necessitates collaboration, and it really does because:  what we can’t do is to have completely disjointed applications being developed -- we need collaboration.  Collaboration comes in a variety of ways.  One is we need visibility, the ability to touch all of these different technologies and view through all of these business processes all the way down into the implementation technology.  That’s just one simple way.

“Another way is we need governance in place that better creates a structure in which we all can predict how we are behaving.  The more reasonable governance is in place, the better we will all function and the better the trust there will be in this very disconnected world that we’re building and maintaining with that.  So there are a variety of things that collaboration encompasses but that’s certainly two of them.”

[After setting a little context, the next post picks up with a discussion of SOA testing best practices, our three C's. Stay tuned.]

Forrester's Five Step Model - Balancing the Strategic and Tactical Aspects of SOA

I had a chance to look at Forrester analyst Randy Heffner’s report on the best way to build SOA (via Joe McKendrick’s Forrester's Five-Step Path to Building SOA). Joe provided a link to the complete report, How to Build Your SOA Platform on ebizQ.  Randy writes in the summary, “Building an effective service-oriented architecture (SOA) platform requires cohesive integration of both new product categories, such as SOA repositories and enterprise service buses (ESBs), and existing product categories, to which vendors are making many SOA enhancements.” He argues against a big-bang transformation except for the very wealthy, but also says an ad-hoc plan can be a waste of money. Instead, he suggests that the best approach is to “place tactical SOA platform investments within a planning framework for evolving a strategic SOA platform. If, along the way, you are able to justify large investments to accelerate your platform build-out, so much the better.”

Joe does a nice job of summarizing the five steps that Randy advocates: Identify existing infrastructure’s SOA capabilities, identify priorities for new SOA capabilities, identify your long-term needs for SOA capabilities, match your platform plans to your organization’s investment strategy, and evolve your SOA platform in line with the business value of solution delivery projects. 

I wanted to address Randy’s opening statement, effective SOA “requires cohesive integration of both new product categories, such as SOA repositories and enterprise service buses.”  Making this integration and transformation simpler is the goal of SOA Testing, Validation & Virtualization practices. In fact it is hard to get to the point of integration without conducting these activities throughout the full lifecycle of the application.

Randy makes a point that there is no one dominant way to build an SOA; companies might start with a specific technology like REST or SOAP or ESB in mind, but that isn't what drives the budget. Indeed it is a heterogeneous problem for any ongoing business with existing technologies to evolve to some future of services-based applications and reuse.

There is value in thinking about SOA from a strategic perspective, and we are big advocates of architecture and design driving selection processes. Before we start, we need to rationalize those existing and proposed technologies for relevance to the business needs and budget.

Now when it comes to articulating these Strategies into SOA Policy, this is where I really feel we have something to add to the discussion. The business-driven expectations of Performance, Scalability and Reliability (PSR) need a mechanism that can verify and enforce these Policies to ensure that they are being met. This needs to happen at the initial component level of design and development, as well as continuously at runtime.

But think about this concept a bit further -- why not Validate your expectations as part of the vetting and selection process for your design?  If you could have an idea of the performance and logical correctness of using a suggested ESB or Service layer, you could recommend it as a next budgeted step in your SOA migration with a lot more confidence. The level of testing doesn't have to be prohibitively strenuous, but being able to verify some business logic and performance expectations of that candidate service early on would save plenty of time and money, compared to testing at integration time.

I certainly agree with Randy that SOA implementation needs to be a balance of strategic and tactical activities, and let's continue to think about how we can help with both ends.