The ITKO Blog: Service Virtualization and Application Lifecycle Optimization - CA Technologies

Q&A with Butler Group's Rob Hailstone

We recently had an extensive briefing and demonstration process with UK-based analyst firm Butler Group's Rob Hailstone - Practice Director, Software Infrastructure, and his team, the result of which is their independent Technology Audit report covering iTKO. We paid for download rights to this analyst document for limited time from our site, and we'll cover the report in a future post.

So after much scrutiny of iTKO's LISA product on their part, I decided to turn the microphone around, and had some Q&A questions for Rob, which I think provide some insight into his perspective on the enterprise software space.

Q: With economic pressure to optimize IT greater than ever, what issues are currently "top of mind" for your practice right now?

RH: My own practice area focuses on SOA. The most immediate impact of the global economic downturn is that organisations that don’t have a really concrete plan for how SOA is going to benefit the business are suspending or abandoning those plans. This is a good thing. Business benefits don’t automatically follow the deployment of SOA, they have to be targeted and worked towards. It is now more important than ever to understand exactly what has to be delivered in order to justify the cost of SOA deployment.

Q: What are your thoughts on the rate of innovation going on today in fields such as virtualization and cloud computing in the enterprise?

RH: Virtualisation has been central to IT ever since the first multi-tasking machines, but the apparent commoditisation of IT hardware caused us to lose focus. We are now much more aware of all the peripheral costs, such as management, power, floor space, etc., and it is obvious that the requirement for virtualisation has never really gone away.

Cloud has a different set of considerations. Some CIOs that will happily embrace virtualisation will think long and hard before relinquishing their physical resources. Cloud provides a great way for packaged application vendors to deliver the application as a service without having to make the heavy investment in scalable, fault-tolerant infrastructure, and it also makes a lot of sense for smaller organisations who need cost-effective off-site backup or a full application platform. However, organisations with an established IT infrastructure that might look to the cloud as a top-up computing capability will probably find the cost of effectively managing a hybrid environment too high in the immediate future.

Q: You've conducted a number of Technology Audits with Butler. How do customers best leverage these documents?

RH: The obvious and most frequent use of Technology Audits is to provide enough information so that a subscriber can decide whether to include a particular product on an evaluation shortlist. They are also used as a reality check on a particular type of strategy to establish whether there is sufficient supporting technology in place to be worth persevering with a more detailed investigation. We try to ensure that we point out where a particular product is not likely to be appropriate as well as showing its strengths.

Q. Any other thoughts in terms of this report?

RH: SOA is all about enabling a higher rate of change, but just putting the architecture in place without the control mechanisms will ultimately lead to chaos. Implementing a change management process backed up with a sound set of testing technologies is an important part of building for long-term success with SOA.

Thanks for letting me ask some questions, Rob, and we had fun digging deep into LISA for the audit.

Eric Roch on Starting SOA Right, Again

There has been a lot of discussion on the success, or lack thereof, of SOA. A number of people have pointed out that the issues are more likely implementation shortcomings than problems with SOA itself (for example, More on SOA Decline and Avoiding SOA Disillusionment).

Since we have an upcoming webinar next week with SOA expert/blogger and Perficient chief technologist Eric Roch, I decided to review one of his recent takes on the topic in his "The SOA Blog." Eric provides some useful guidelines for those who are taking a second look at their SOA efforts in his post, Getting Started with SOA - Again? Eric advises us to start simple, tackle a business problem, and pick your first project carefully. He works off some advice from IBM. He suggests that process and connectivity efforts are two good ways to get restarted.

Eric adds, “The connectivity entry point is often driven by the purchase of new software packages that require integration. The process entry point is an opportunity to work with the business to improve business processes. Both of these entry points offer quick successes and ROI.” Eric adds to begin with BPM, and let that be a driver of SOA. So far, I agree.

As I have said before, getting the business teams to think in terms of optimizing, integrating and reusing processes, makes it easier for the IT side of the house to architect and manage their SOA around these vertical and horizontal processes. When interactions between groups, decision points and constraints are modeled successfully, it makes the case for technology that can support it. Eric then provides three more entry points in addition to process and connectivity. Supporting enhanced collaboration, better information access, or asset reuse are other likely starting points for SOA success. These are great suggestions that all can show the value of beginning together multiple services. With all of these, proper testing and validation is a prerequisite to SOA success. 

We hope the above advice gives you some starting points for, uh... "un-re-disillusionizing" your SOA initiatives and getting some initial wins. We also hope you can join us March 31 (or afterward) for the webinar with me and Eric.

Minimize IT Outsourcing Risk with Collaborative Quality - Part Four

This is the fourth in a four part series that summarizes the key points in Minimize IT Outsourcing Risk with Collaborative Quality. You can click on the title to get the complete white paper. In the previous post in this series we looked at how to handle changing components and dependencies and achieve validation on an ongoing basis, as well as how to enable parallel development and testing without constraints.

Too often, companies allow the success of their outsource efforts to hinge on the performance of the services partner alone. In this post, we summarize best practices and responsibilities for the outsourcing consumer, after all, successful collaboration must be a two-way street.

• Reward Outsource Partners for Delivery Success... Not Just Delivering. – Do not prematurely sign off on delivered code assets just to meet a deadline, before adequate testing to ensure requirements are met.
• Trust, But Verify... Continuously - Distributed teams need to document component dependencies and test them as part of their deliverable. The consumer then needs to add tests at a business process level to ensure that expected behaviors are verified in the context of the consuming workflow.
• Enable Greater Productivity by Removing Constraints - Service Virtualization can enable multiple teams to work freely in parallel, rather than at odds due to the constraint of having access to live or incomplete systems. For instance the VSE in LISA Virtualize your partners with a way to decouple themselves from dependencies, by modeling test resources as Virtual Services that behave just like the rest of the application for testing and development purposes.

In summary, a partnership between outsource consumers and outsource providers based on shared objectives for collaborative quality is the best approach to maximize the benefits of IT outsourcing. Success depends upon a combination of people, processes and enabling technology to foster greater productivity, agility and quality across distributed teams. Key capabilities for this success include a collaborative framework for testing functional requirements, continuous test and validation of delivered components within the context of larger systems, and virtualization of constrained resources for greater productivity and improved time-to-market for distributed teams.

This ends our brief series on Outsourcing with Collaborative Quality - however we will continue to post on this important topic as we work with an ecosystem of customers and partners with the shared goal of delivering successful software releases on time and under budget. Your comments and ideas for future topics welcomed.

Monsters of SOA Webinar 3/31: Reducing SOA Costs with QA

We're looking forward to what I am informally dubbing a "Monsters of SOA" online discussion and presentation titled "Realizing the Benefits of Enterprise Integration and SOA at Lower Costs" coming up on IDevNews March 31, 1pm Eastern with iTKOs Chief Geek & Founder, John Michelsen (left), and Perficient's Chief Technologist Eric Roch (right). iDevNews' editor Vance McCarthy will moderate.

We've had the pleasure of collaborating with Eric's consulting teams on  the ground at  several large-scale SOA migration and implementation customer sites, and we have found there are many points when our best advice and tools for testing and validation merge with Perficient's best practices and delivery process - so much so that they have developed a specialized QA program for SOA - which according to real project timelines can consume up to 30% of a typical SOA initiative's budget.

The fact is, the more services-based functionality you add, and the more agility you introduce, the faster your rate of change is. And we aren't throwing anything away by adding SOA, so that means more you need to test, and more demand for stronger validation throughout the project lifecycle.

Don't get me wrong of course, agility and the distributed approaches that enable it are great things - and QA and validation are great things, but while we deliver that, we need to deliver it in more automated, smarter ways so it drives agility and cost reduction instead of expanding in cost with the complexity of the architecture.

Eric also puts out a SOA blog we frequently read on Toolbox. In this talk we have two IT leaders with hundreds of SOA-style project engagement experiences between them, so stick around for the insights and Q&A.

Anyway, hope you can tune in March 31 or afterward to watch the session archive, signup is free here: https://secure.soatube.com/registration/w_iTKO2/?code=itko. A preview copy of the slides can also be downloaded.

Minimize IT Outsourcing Risk with Collaborative Quality - Part Three

This is the third of a four part series that summarizes the key points of our paper on Minimizing IT Outsourcing Risk with Collaborative Quality. You can click on the title to get the complete white paper.

In the last post in the series, we started to look at how to meet the risks and challenges of the outsourcing of IT development. In this post, we will look at how you can handle the changing components and dependencies and achieve validation on an ongoing basis. We will also look at how iTKO enables parallel development and testing without constraints.

In a SOA environment, a change to any service can create "unintended consequences" by causing unexpected failures on other dependent services. Throw in third parties through outsourcing and you compound the risk. To mitigate this risk, you need to continuously validate that delivered components have accounted for other elements in the workflow, and declare those in your tests, providing other team members with visibility into those dependencies.

Now, we need to "coat dependencies with testing" from this perspective, to prevent unintended consequences and run automated tests as needed on an ongoing basis. As a best practice, teams often use our solution to segment continuous validation activities along business process lines. To achieve maximum efficiency, distributed teams need to decouple their own environments from dependencies on live or incomplete systems, so they can continue working in parallel, rather than waiting on each other.

Think about many services engagements, and the amount of effort wasted on providing test access on a live transaction service or critical data source to a service vendor. Often there is a very limited window of time that can be offered by the companies IT ops team. Service Virtualization solves this bottleneck by allowing teams to design, develop and test their own components against changing environments by capturing and modeling the rest of the system (services, data, mainframes, etc) as running Virtual Services that simulate the behavior of the needed target environment.

Now multiple teams can design and test services in parallel and thus eliminate the constraints and dependencies of accessibility, availability and security for incomplete components and live service implementations. In the fourth and last post in this series we look at best practices for the outsourcing consumer.

Analysts & Pundits: Will IBM buy Sun, and is that a Good Thing?

Big news on the technology horizon, and here at iTKO we are watching the development with much interest: See Reuters' "IBM in talks to buy Sun" story, among many other stories on it.

Network World's Denise Dubie caught several analyst insights in their coverage of the emerging story, "Could IBM be Sun's savior?"

Also check out eBizQ's discussion board if you care - lots of the usual suspects we talk to in SOA/BPM/enterprise architecture chiming in over there. Here was my (quick) comment on the topic:

"Yes I think it will happen, and from my development perspective, while we might like healthy competition and choice in vendors, it is better a strategic move now than a fire sale for Sun later. IBM practically owns enterprise computing, and they have gone out of their way to own a big chunk of the Java world with it.

Sun has tried very hard to attract developers, lately, open sourcing as much as they could - but open source is a hard space to compete in because it doesn't generate cash! By hugely funding Eclipse, IBM spoiled the IDE market for Sun and a number of other players.

Still this will be an interesting development to watch, as we are talking about a boatload of innovations and 2 very different cultures that would need to be folded together here."

Perfecting IT Strategies in a Down Economy

In today's tough economy, you can't even read an article that doesn't start out: "In today's tough economy..." This one is no different. Fortunately when it comes to IT, the message does not need to be accompanied with knee-jerk cost-cutting measures.

Last month Todd Biske offered some solid advice on what to do in today's down economy in his post, "Don’t Go On an IT Diet, Change Your Behavior". Needless to say, his post remains relevant. I especially liked his comments:

“The whole point of Enterprise Architecture, SOA, and many of these other strategic IT initiatives is to allow IT to be more agile- to respond more quickly to changes in the business objectives. Guess what? We’re in the middle of a big unprecedented change in our lifetime. My guess is that the best survivors of this meltdown will be organizations that don’t go on a starvation diet, but instead simply recognize that their priorities and goals have changed and execute without significant disruption to the way they utilize IT.”

Todd also wrote that cost cutting efforts to respond to the economy should not mean that organizations drop their SOA initiatives, or other long-term initiatives, especially when most of these efforts are trying to reduce costs cost by eliminating redundancy in the enterprise. It would be ironic as these efforts are exactly what are needed today.

SOA efforts should be more closely managed and held to their ROI objectives in any economy - otherwise, they will not have a chance to create the value you expected. It may be that closer review will actually improve the number of SOA efforts that are successful, as people will be more highly motivated to get them right. There is no time for poor execution now but there is a lot of room for innovation. As Jonathan Schwartz, the Sun COO, wrote in "Innovation Loves a Crisis," now is the time to make changes that will make you most adaptive to the new economy and best positioned to come out of the downturn on top.

McKinsey also supported this strategic thinking in their report, "Leading through uncertainty": “The task of business leaders must be to overcome the paralysis that dooms any organization, and to begin shaping the future.” Transforming to SOA - not because it's "cool" but because we have real value expectations, is one way to become more flexible and cost effective at the same time.

Accompanying that, we are also sure to mention Service Virtualization as a key means of increasing the availability of SOA assets in the lifecycle, at a small fraction of the environmental cost. It is money well spent not just in terms of cost savings, but more so in coming out of a lean time with a more agile and efficiently delivering dev and test organization.

Minimize IT Outsourcing Risk with Collaborative Quality - Part Two

This is the second of a four part series that summarizes the key points in Minimize IT Outsourcing Risk with Collaborative Quality. You can click on the title to get the complete white paper.

In the last post, we looked at the risks that can come from outsourcing IT development. Solving for these risks, not just through contractual negotiations, but in every day collaboration, can unlock the benefits of outsourced IT and address larger distributed development issues. In fact, if the appropriate IT infrastructure is available to better manage accountability and quality across distributed teams, while eliminating dependencies and constraints that restrict the agility and productivity of these teams, then it follows that these benefits could apply not just to outsourcing situations, but to any distributed team environment.

The first step in meeting the challenges of distributed teams is to gain a commitment across all teams to deliver components against well-defined requirements, in an auditable form, with the ability to accommodate the highly interdependent nature of today’s applications. This requires more than the right attitude, it requires the right tools.  Key testing, validation and virtualization capabilities will help firms mitigate these risks, and foster more collaborative quality across interdependent teams. These capabilities include:

1. A collaborative testing framework that directly verifies all technology layers with a high level of automation, so the team can better meet defined requirements at lower cost.
2. Continuous Validation so both parties can assure that each software component supports all upstream and downstream dependencies, in an auditable form.
3. A Virtual Service Environment (or VSE) which simulates the behavior of the highly interdependent and critical systems the teams must build and test against, even when they may not have access to live or completed components.

With these capabilities, the consumer of services should not just receive code assets, but also the deliverables they need to rapidly validate them - collaborative testing is matched to functional requirements. This is important, since resolving a misunderstanding early in the requirements phase can be orders of magnitude less expensive than fixing it later in deployment. So how can we move testing earlier into the lifecycle, to save costs and improve quality?

If the offshore partner’s testing is not tied to functional requirements, there is no way to know if there is any value to the test. The key to achieving collaboration is to demand that each component a team delivers is also comes with a suite of working tests that prove its functional requirements have been met. That way the next team, whether local or offshore, can iterate on testing from there.

For instance LISA tests are saved as .XML documents, then the business team can augment these processes by combining them with traditional requirements docs as executable test assets in ALM and Test Management tools. From this foundation, the tests become part of a continuous build and validation process, where regressions are proven to match defined requirements in later testing cycles. In our next post, we look at how you can handle the changing components and dependencies and achieve this kind of validation on an ongoing basis.

Minimize IT Outsourcing Risk with Collaborative Quality - Part One

This is the first of a four part series that summarizes the key points outlined in our paper on Minimizing IT Outsourcing Risk with Collaborative Quality. You can click on the title to get the complete white paper.

In the past ten years we have seen explosive growth in the practice of outsourcing IT functions to SIs (System Integrators) and services firms around the world. One of the fastest growing areas was the practice of outsourcing software testing to overseas teams. How as enterprise systems change from huge monolithic applications to more componentized, services-based solutions, the risk involved in outsourcing testing rises. This series looks at these risks and offers a solution.

First let’s set the stage by looking at four common IT outsourcing models:

1. Single Outsource Vendor where the buyer outsources all IT development and testing functions to a single SI partner. It offers the benefit of a single source for delivery of business applications, and one vendor to call if something goes wrong. However, it can also create high switching costs if you decide to change vendors later.
2. Outsourced Testing appears to be the easiest route if you decide to keep software development close to home. Since testing is by nature the process of “breaking” an application, many organizations find that a removed third party tester can be very valuable and impartial. This model was very easy to adopt when testing was a manual, point-and-click-on-the-screen endeavor that was best done by less costly staff. But times have changed, forcing testing offerings to evolve in sophistication.
3. Development Support. The many companies following this strategy are seeking both cost efficiency for non-differentiating development tasks, and access to developers with specialized skills needed to carry out specific tasks, such as migration between two different integration platforms that they may not have the expertise to pull off in-house.
4. Multi-Vendor is most common when development and testing activities are outsourced to different SIs. In this model, the company hopes to keep costs down, while the multiple providers keep each other "in check" to ensure project success. While you can mitigate some risks associated with a single outsource vendor, it can also be harder to manage multiple firms and get them to collaborate as efficiently.

Now let's look at the risks, as managing risk can be more important than obtaining upfront savings, if these cost reductions come back to haunt you later. After all, what good is a lower cost project if it takes 3 times as long to repair, or if your app fails in front of customers? Your outsourcing partner needs to be incented to build quality into the application, and not simply cut costs and initial project time.

This balance of quality and speed gets more complex when multiple teams are involved. While many larger companies have a multi-vendor strategy of procuring IT services, the more companies and teams that are involved in a project, the more opportunities there are for the information sharing and collaboration to break down.

What is needed is greater team accountability for shared deliverables all the way across the software lifecycle, which includes visibility into problems early, versus later in the deployment where they become significantly more costly. There is also the risk generated by dependencies on unavailable or inaccessible systems. Interdependent teams working together can be constrained by limited access to live applications and components that are not yet completed or available for testing. We see this issue often in the field as reported in our series.

When there is increased contention for shared resources, agility suffers as teams are forced to queue up for access, introducing scheduling delays and project risks. In some cases, teams may attempt to maintain their own test environments by replicating the components they need in a testing lab or custom test harness.

These team-to-team constraints on environments can become very costly to manage, requiring a high level of configuration, licensing costs and maintenance to keep current, even if it is running on virtualized hardware (which also has incremental licensing costs). And of course, some sensitive systems such as ERP mainframes are simply too big and have too much overhead to be replicated by conventional hardware virtualization means. In our next post in this series, we start to look at ways to meet these challenges and make more effective use of IT outsourcing.

ebizQ Discussion on SOA & Cloud Computing - Everyone's got a point...

The eBizQ site recently hosted a discussion is based on JP Morgenthal's feature article, Differing Views on SOA & Cloud Computing, and the question, which seemed to cause a bit of disagreement between the old "gray hair" and new "newbie" schools of IT, is: SOA & Cloud Computing new and unique or a continuation of the same old stuff?

Occasionally you come across a topic like this that touches off a flood of responses from the community. There are a range of insights from a "who's who" of SOA and IT pundits on this particular topic worth looking at on the ebizQ discussion site: SOA & Cloud Computing: New & Unique or Continuation of the Same Old Stuff?

Here was my response:

The future risk of the cloud is the current risk of SOA. I agree with the above statements about the hype factor of cloud computing, however there's still one more challenge that both strategies will share -- dependency. In SOA engagements, we encouraged the importance of good architecture and governance, but often it was just too easy for companies to hook up a couple web services to a legacy system for a quick ROI.

Dependency didn't rear its ugly head in SOA until the services started to be used by multiple customers, each with their own needs, and then those services were connected with the services of business partners, and used in a variety of unintended ways. A lack of access to dependent services and systems throughout the development lifecycle created a constraint on the agility we expected from SOA.

Similarly, the true cost and complexity of using cloud-based solutions will become evident only at the point when companies rely on them for critical needs. The low cost of entry of the Cloud's easy provisioning and pay-as-you-use can end up costing more than expected once the solution reaches business volume. If I pay-per-use for a cloud-based solution, I am establishing recurring costs, and increasing dependencies in my testing, development and release of new functionality that need to be addressed. Hopefully this shared concern of dependency will encourage enterprises not to sacrifice good architecture, governance, and virtualization practices for short term ROI gains.

We see many companies getting past the hype of SOA to generate real value today by applying the approach with more discipline, and many of the same lessons will apply to cloud computing.

Field Reports on SOA Success - Part Six: Major Enterprise Software Firm

In life, there are some good surprises (birthdays, hearing a great new song, meeting that special someone), but in enterprise software, not so much. This sixth example in our stories from the field is about a major enterprise software firm.

This firm decided to move much of the sales of their software upgrades and maintenance contracts to a web portal, where clients could just buy and download the updates they needed. Previously the sales force would not have to get involved with contacting each client and conducting manual orders on routine stuff, which wasted time for both parties. Many clients had open POs so they could just add on these incremental purchases. This was to be a win for both sides.

To set this up, they needed to access the customer data and their software inventory. They could then make suggestions about what additions might be needed, offer the ability to instantly purchase, and download. They did manual testing of the system UI, and thorough unit testing of the software. Everything seemed okay for rollout.

However, as they discovered three months after the launch, there was a problem with the all-important middle step. The system was supposed to push the order into the enterprise invoicing system, SAP in this case. Then the client would be billed.

Ninety days after launch, the guy responsible for the SAP transaction system asked if they had started this new system since no orders had entered into the system of record. To their embarrassment, the clients were getting their software downloads, but their invoices were not getting recorded.

It seems another group implemented a policy so that each account had a password change every 90 days. These changes occurred between the application test time and the launch time. So fulfilled upgrade orders were getting logged as bad data, and then flushed on a regular basis. The clients were not complaining because they were getting the software, and likely did not mind that the invoicing appeared slow.

The firm brought us in to do end-to-end business process testing to make sure that all steps in the process worked on sample cases. We tested not just at the web ordering UI layer, but followed that transaction through the app server, ESBs, CORBA and all the way to confirming that it made it into the system of record. We call this practice “invoke and verify.”

After software was downloaded, LISA made sure the invoice was processed for that specific download, and the team quickly rooted out the “changed password” culprit that was causing orders to get routed incorrectly from within the ESB layer. This validation was also done on a continuous basis to make sure any new changes did not disrupt the flow. Now all steps in the process work, and the original goals of better customer service at lower cost -- without the unwelcome surprises -- were achieved.

You can find the complete case details here.

Field Reports on SOA Success - Part Five: Major Airlines

We’ve talked a lot about how ensuring quality is a big challenge for complex SOA and enterprise software environments. But there is a bigger-picture risk that can drive any project past deadline and over budget: constrained and costly access to necessary systems and data throughout the development and testing lifecycle. This fifth example is about a major airline.

Like many companies, they faced constraints in their testing environment. In the airline industry, this is magnified, as all carriers have to make use of third party reservation systems. The billing for this access runs just like cell phone billing (but on a much bigger pricing scale!), with a set limit, followed by overage charges. Anything over the limit can get very expensive. In addition, the access itself is not cheap to begin with.

This airline wanted to expand their testing lab environments from four to nine. However, given the transaction fees I just described, and the associated hardware cots, this change would run into millions of dollars of extra costs on an ongoing annual basis. In addition to the extra transaction costs, the airline would have to purchase more hardware. Since you cannot just deal with airline and customer data over the public web for very real security purposes, they needed to provision special gateways and data circuits to interact with the third party source.

However, this was just the beginning of the problems they faced. Travel data is very complex and changing. There are weather delays, fare changes, rewards programs, partner systems, flight operation systems, and so on. To build applications, you need access to stable customer and flight data scenarios for testing and validation.

The airline found that when they ran regression tests, about 10% of the test cases failed. The failures were not mostly due to bad software, but data volatility. They had to find a way overcome a huge number of false negatives due to bad test data. Teams would often waste half their day chasing down these errors. Increasing the number of test labs by over 100% would only compound these data troubles.

Introducing a strategy that covered both automated testing and Service Virtualization addressed these issues, so we brought LISA to bear and demonstrated how to virtualize these kinds of services that are prohibitively costly and complex to replicate using conventional means.

First, they eliminated the need to incur almost all the transaction costs to the third party system by accurately simulating the required data as virtual services for development and test purposes.

This also took the additional hardware costs for gateways and data circuits out of the equation by reducing infrastructure needs and virtualizing any new infrastructure requirements. These hard dollar cost savings more than covered their investment in automated testing.

There was more, as the cycle of false negatives due to bad test data was broken, as stable test data could be provided. This may prove to be the largest source of saving overall. Now application development could proceed at a much faster clip, helping the airline generate customer benefits and revenue from the new application much quicker, and reducing development costs.

They are also planning to make use of the automated testing system and the data it generates to streamline their training programs and make them accessible on a local basis, reducing the travel expenses associated with getting their employees flying on the new systems they develop.

You can find the complete case study here.