The ITKO Blog: Service Virtualization and Application Lifecycle Optimization - CA Technologies

Getting ready for SoftwareAG Innovation World: The Tested, Validated & Secured SOA

We're all packed for the upcoming SoftwareAG InnovationWorld 2008 conference in Miami next week. Looking forward to a lot of good dialogue with customers and partners there - last year's was one of the most valuable SOA/BPM/Integration conferences we've seen.

If you're going, check out our session Wednesday morning (10:30am) with Software AG's VP of SOA Products Jignesh Shah, and iTKO's CMO Jim Mackay on "The Secured, Tested & Virtualized SOA."

There is a lot of confusion in the market about where the external concept of Security ends, and where the Validation of secure and appropriate behaviors within SOA begins. In reality, to avoid risk, you need both. Proper Security protocols, and strong management of who is authorized to participate in SOA apps is critical.

But once you have dozens of partners inside the "gate," Validation needs to take over, to ensure that the kinds of Policies that can't be managed structurally are being met. For instance, are participants ONLY seeing the information they should see? Is there a limitation on transaction volume, or being able to edit/delete orders and data? As you can see, there are many Policies to validate within our applications, especially when you have multiple partners consuming and sharing Services.

Then, Service Virtualization adds to the mix by allowing all of the teams to have very "safe" Virtual Services available 24/7 that they can develop and test throughout their own lifecycle, without impacting the performance or security of live business services, or being privy to live customer transactions. Of course the best byproduct of this is the agility and cost savings you can gain, through elimination of constraints throughout the lifecycle of development and testing.

Hope to see you in Miami - perhaps the best place to be in the continental US in November!

Agile SOA Across the Lifecycle with LISA - Part Two: Test & Quality Management

This is the second of a six part series of posts on the Agile life cycle for SOA. Here we will at look at SOA test and quality management.

In the past, test “scripts” in TM tools were largely recorded as actual step-by-step instructions (in a Word doc or Excel spreadsheet) for a manual tester to point-and-click through a finished interface to test the application. When tests were completed, the tester would then “check a box” in the test management UI, to indicate that the test had passed or failed. With so many changing components and middle tier systems in today’s enterprise apps, that form of manual testing is no longer enough, so teams are trying to define (and run) far more sophisticated tests that can actually test behind the user interface, where the business logic lies.

In addition to deeper functional testing, teams using TM tools are asking for a higher degree of automation from their test suites. A thousand manual test scripts can delay cycles by days or weeks while they are executed, costing the team serious agility, and making the testing less valuable over time due to repetition and human error.

Therefore functional, regression and load/stress tests need to be stored directly within the TM tool – for instance as executable command line or XML scripts that launch a test from a ready server-side tool. In this way, a sophisticated set of tests can be executed with just one click in the TM tool, and the pass/fail and other results and issues are automatically written back into the TM tool.

In this way, the whole team gets to use the test process they are familiar with, while getting much deeper tests that can talk to middle tiers, and as much as a 95% improvement in test execution time compared to manual tests. In addition, quality is improved as the teams can focus on testing new functionality, instead of repeating the rote execution of tests against existing functionality.

We are asked for this kind of functionality at virtually every customer engagement - so I'll mention our solution here. iTKO LISA provides connectivity to Test Management (TM) tools, such as HP TestDirector for Quality Center (formerly Mercury TestDirector), giving them a key point of automated test execution, enforcement and test feedback. A welcome “byproduct” or end result of this integration between LISA and test management software is auditable documentation of the test, with details about the status of the many systems under test – showing that at a specific point in time, the system was working as expected, or not working as expected. The testing tool has then fulfilled the requirement of being the "long arm of the law," by spotting the issue, enforcing or “making judgment” on it, and reporting the issue to the TM tool.

The overarching goal of this process is to allow better Collaboration on testing, by sharing and iterating on tests across Development, Testing, business analysis and IT Operations teams. This increases the level of reuse in SOA and enables faster release cycles, as all teams can focus on delivering quality into new features, instead of repeating test creation and execution manually with each new release. In our next post, we will cover application lifecycle management.  You can download this complete "Agile SOA Across the Lifecycle" paper at our ITKO LISA resources page. 

Agile SOA Across the Lifecycle with LISA - Part One: Introduction

This is the first of a six part series of posts on how ITKO LISA works across the Agile life cycle. For years, testing was an siloed activity that used different and unconnected tools within specific phases of the application lifecycle.  First, a development team would run a suite of JUnit tests as part of a build process. Then a QA team would manually create and run a suite of functional tests against a user interface. Performance teams would use a separate set of load tests to exercise and monitor the completed application.

In this old world, there was not much sharing of information and test assets among these teams, beyond the occasional text “bug report” that carried little context into the root causes of problems that occurred. This occurred largely because the available test tools created scripted test assets that were incompatible, and largely irrelevant to the next team’s process in the chain. Unit tests were good for developers to test for structural bugs in code, but those tests didn’t translate into any business context for the QA team. The QA team would instead script their own UI tests, which don’t exercise components or code in a way that developers can use to track down root causes.

This was not a very efficient process so companies began to align their people around better process tools that reach across the silos to better manage development, integration and management of their applications.  Agility is not just about delivering faster, it also bumps up the reliability of enterprise applications, by instilling a higher degree of collaboration and control across these suites of application lifecycle tools and processes. Agile promises many development efficiencies and increased flexibility. See Chris Spagnuolo’s post on Embracing Change with Agile Practices for a nice description of the flexibility in agile ad the ability to make mid-course corrections. However, if testing and validation is not handled right, risks will emerge compromising these gains.

In this series we are going to look at four aspects of the Agile lifecycle: test & quality management, application lifecycle management, IT operations, monitoring and performance, IT and SOA governance.  But first, let’s take a look at a sample of leading application process tools we often encounter within iTKO customer sites.

The goal of all application process tools is to enforce some kind of workflow into an aspect of the software lifecycle, be it around testing, development, operations or governance and integration of the overall IT infrastructure. Direct testability and validation of most SOA and enterprise software components that are managed by leading process tools is a must.

By supporting the workflow of these tools with executable test assets, we provide a higher degree of predictability and performance from the resulting application, while allowing development, testing and operations teams to better manage their process with the methods and tools they prefer. While many of the above vendor solutions might work across different domains (for instance ALM might include some Test Management), we’ve provided the above table as a sample of the tools encountered in typical IT customer environments. Key to Agile development practices is the tight interaction of testing with development activities, for faster delivery time of prioritized business functionality.

In essence, a test or validation is an asset that is stored in, launched from, or provides feedback into the above process and automation tools, contributing to the goal of Agility for SOA. In our next post, we will cover the test & quality management phase.  You can download this complete series on "Agile SOA" at our ITKO LISA resources page.

 

Making IT More Green and Saving Money at the Same Time Through SOA

Saving money though reducing the environmental impact of IT has gained increased urgency in our down economy. Green computing even has a Wikipedia entry. Fortunately, many of the Green IT efforts also result in cost reduction. One of our business partners, Software AG, has looked at ways that SOA can help here – see Eco-computing: Software AG and the Hasso Plattner Institute Demonstrate SOA’s Green Potential.

Software AG and the Potsdam-based Hasso Plattner Institute for Software Systems Engineering (HPI) completed a one-year research project examining the impact of service-oriented architecture (SOA) governance on Green IT practices. They developed methods and tools for optimizing resource sharing of IT services, thereby reducing energy consumption and lowering hardware costs. Using these methods, administrators of complex IT landscapes can determine exactly how much computing power is needed, as well as the maximum amount of energy each service might use. With proper SOA governance you can regulate and more tightly control energy costs by taking advantage of the more efficient distribution of services that SOA can bring. This makes sense and is one more reason to make sure your SOA governance is carefully conceived.

There are other ways SOA can have a positive eco-impact, as well as a bottom line impact. For example, virtualization of hardware saves the materials and energy consumption of so many servers - as much as 10 to 1. Most companies have already gone that far, but what about the kinds of applications that are too distributed (i.e. shared services and Cloud-based apps) or too monolithic (i.e. mainframes, major enterprise apps) to be replicated via hardware virtualization?

We call this Service Virtualization and it allows the behavior of complex SOA applications to be captured and simulated - which is key to achieving the productivity and efficiency you expect from SOA. Therefore, if you could save the cost of attempting to replicate several distributed applications and services, that's saving a lot more processing power throughout the lifecycle and not just servers in your own data center. In addition, it allows you to scale down hardware and processing power in your test lab, and in deployment, as you no longer have to try to make a live instance available to so many teams, or scale up the hardware of the live instance to handle a bunch of extra capacity.

There are more savings to be harvested. When you break up project teams into smaller "agile" teams focused on enabling certain Services for use and reuse, you gain additional efficiencies besides Agility and labor cost savings (see our recent paper: "Agility Across the SOA Lifecycle" for more).

Think about the reduced need for travel, and for dozens of people to occupy an office configuring and maintaining separate test environments per partner and per test/development team. The more we can do to decouple our dependencies on constrained servers and applications, and decouple our interdepency on other teams' lifecycles through service virtualization, the bigger the positive ripple effect between the interaction of technology and the environment.

We welcome any discussion and ideas about how Virtualization in a Service-Oriented World can make a difference in the world.

First comes BPM, then SOA...

Had a chance to catch a good talk on Business Process Management with our partners at Software AG today.

Mike Lees delivered an extended definition of BPM  in their eyes, and this practice is not centered around technology at all - it is really a management discipline involving the non-technical side of the house, and getting them to understand the "3 A's" (Alignment, Automation & Agility). So, BPM excellence is delivered first as process and later reflected in the software infrastructure.

Another interesting aspect of BPM is the idea of Reuse. Just like the underlying technology in SOA, a defined business process is a piece of metadata that you want to optimize and repeat, especially if it is an effective one. It is interesting to see how "behaviors" and "events" in business can be modeled to meet business needs - they are really more IP than the details of the software itself.

Lastly, I found an interesting "helping-hand" portion of the talk that really hits home - if you want SOA to be a success, get BPM to be a facilitator to SOA. Getting the business teams to think in terms of optimizing, integrating and reusing processes, makes it easier for the IT side of the house to architect and manage their SOA around these vertical and horizontal processes. When interactions between groups, decision points and constraints are modeled successfully, it makes the case for technology that can support it.

It is easy when you are working in the technology/terminology bubble to remember that things like "process transparency" and "optimization" are not about debugging and monitoring the performance metrics of software and integration servers - for the business, they are exactly what they mean.

SOA Governance Panel Recap: Federated SOA for the Day

iTKO's John Michelsen recently served on ebizQ's SOA Governance panel for a spirited online discussion. Joe McKendrick moderated the panel which also included: Anne Thomas Manes, vice president and research director with Burton Group, and former Chief Technology Officer at Systinet; Ron Schmelzer, managing partner with ZapThink; Frank Martinez, senior vice president, product strategy for SOA Software; David Bressler, SOA Evangelist for the Actional products at Progress Software; and Ed Horst, vice president of marketing and product strategy for AmberPoint. Joe provided both a summary, Panel: SOA Governance Answers 'What's in it for Me?' Question, and the full transcript: see Transcript: SOA Governance Panel.

Joe charged the panelists to comment on how far along we really are with SOA, and what is needed to take things to the next level. He also wanted them to be practical, and explore the tools and techniques that are essential to getting started with SOA governance. In addition the discussion focused on the implications of SOA governance for the "Web 2.0-ish" activities (like Cloud and WOA) that are often well under the radar of corporate IT management. Lastly, he wanted to cover one of our favorite topics, virtualization and SOA.

As Joe said in the summary, setting up proper governance seems like the logical approach to introduce SOA to the business. However, in the real world, SOA proponents run up against a series of obstacles. Many stakeholders have to be signed on to the SOA effort. These include developers who are expected to reuse someone else's code, departments that are expected to share assets and timelines, functional groups that are expected to provide more funding than usual to the IT underlying their processes with the hope for future cost savings and greater agility.

In response to these challenges John suggested that a Federated approach may be essential to SOA governance. In effect, people may be more willing to accept governance at  more decentralized levels, rather than more centralized approaches:

"We found that it is very analogous to the governance analogy in the physical world. That federating or layering of the governance policy is how it happens. If you think about it, my homeowners association, it wants to know where I'm going to put my flag. But at the Federal level.. that would be an absurd notion for them to decide such a thing. So we are more tolerant of greater control closest to us, than we are far away. So if there’s this very distant group of people who decide on how we are supposed to build and consume services, then we will tend to react negatively toward that. But if there is even.. a fairly onerous set of policies, but it’s shared, and its owned by me and my close constituents, I tend to be more willing to do it."

Later John added that since SOA has been around for a while, we know in advance a number of things that require planning, including aspects of the governance requirements. The solutions require more that installing infrastructure, they require an understanding of the desired business behaviors and how this infrastructure will support these behaviors.  He also said that there are two types of governance: what operates during design and what operates during run time as the transition to the new SOA services takes place and new complexities come up. 

While the panel was not able to cover virtualization like we hoped in the hour, they did manage to cover  other topics. The panel’s responses provide a rich resource on SOA Governance and make a good read (or listen). Here are the participants. They are left to right: Anne Thomas Manes,  Ronald Schmelzer, Frank Martinez, David Bressler, John Michelsen, Ed Horst, and Joe McKendrick.

Thanks for the questions and comments, and those of you who stopped by our "virtual booth" for more questions. We're looking forward to our next one at the "SOA in Action" virtual conference Nov 19th 2008, which has a who's who of SOA experts throughout the day. John is participating in the "Avoiding SOA Disillusionment" panel at 2:00 PM Eastern with Joe and a new cast.

Some More Good SOA Business Blogs - Enough for Now

I have now written a six part series on software community blogs. This has included some of our larger partners such as Oracle, TIBCO, and Software AG who maintain multiple blogs with a community.  I also covered the Agile Commons blog by our partner, Rally.  We have some other partners who have excellent individual blogs, or in the case of Infosys, a great collection of blogs. A list can be found in the right side of this blog. Here are three more of them.

dynaTrace is a leader in lifecycle application performance management.  The dynaTrace blog covers issues related to performance, scalability, and architecture. The authors are Alois Reitbauer and Andreas Grabner. The most recent post when I looked was on Error Analysis Process: The Weak Spot in Manual and Functional Testing. It resulted from Andreas Grabner’s participation in the recent QAI Quest Testing Show in Toronto, ON. He said that he found that much time is still wasted in the testing process with excess needs for communication between testers and developers. He offered the solution that “capturing diagnostics information on the tested application provides all the information for developers to analyze and fix the problem without having to request additional information and without reproducing the problem. It saves a lot of time and overhead and frees valuable resources to do the work they should really do.”

Infosys defines, designs and delivers IT-enabled business solutions for Global 2000 companies. It offers a collection of blogs that “discuss the business of technology and the technology of business in the Flat World.”  One goes by the name Think Flat and has the tagline, “The business world is being disrupted by the combined effects of growing emerging economies, shifts in global demographics, ubiquity of technology and accountability regulation. Infosys believes that to compete in the flat world, businesses must shift their operational priorities.” The most recent posts covered the evolving financial crisis. There is a total of 14 blogs when I last looked.  One of particular interest is on SOA - Service Orientation Applied. One recent post, Making Your SOA Journey Successful – Key Aspects, has a number of useful tips to avoid SOA mistakes.  It included the importance of governance and where to focus this governance.

Patni Computer Systems, Ltd. is partnering with ITKO to deliver SOA Testing services as part of its comprehensive Verification and Validation (V&V) practice. Patni also provides the blog, In Sync. There are multiple authors. A recent post by Sandeep Patil covered, Customized product inquiry - Eliminating "Muda." The post quotes a study done by Giga Information group, citing that the acceptable response time a customer would expect for viewing the result of a modification demand is up to 20 seconds. He goes on to add that such a response from a salesperson using the manual configuration process is almost impossible. However, with the advent of a configurator solution, it has become a reality - increasing customer satisfaction at lower cost. There are also links to podcasts and longer thought papers.

We'll check back on these partner and peer blogs occasionally, and continue to add them to our lists here on the iTKO LISA SOA Soapbox.

VMware – Sixth in Our Series on Software Community Blogs

VMware, one of our partners, has the VMNT blog which provides virtualization news from VMware and the community of virtualization users, including the VMware Communities and VMTN, the VMware Technology Network. This blog also lists 17 other VMware blogs, each with a specific focus.  There is the Console blog that is VMware's Executive Blog. It has contributions from a number of VMware executives. The Develop Center Blog supports their developer community.  There is the WMware PhD blog that provides education on VMware products.

Ed Albanese,
Product Management, 
VMware Application Virtualization Technologies writes the VMware Application Virtualization blog.  The VI Team Blog provides implementation tips. The VIX API Blog discusses the VMware VIX API which allows you to automate VM and guest operations on VMware Server, Workstation and ESX.

The blog, VMware: Virtual Reality, carries the subtitle, Setting the Record Straight One Post at a Time.  It responds to industry claims. There is a nice review of Why Choose VMware that provides their competitive differentiation is a single clear post.  The one prior to it addresses claims by Microsoft. There are others such as VROOM! by the VMware performance team. It is nice to see a focused set of blogs that cover a range of issues.

The VMware blogs also list 22 WMware employee blogs and provide a longer list of blogs on Virtualization. The VWware employee blogs are individual efforts, a good way to advance the advocacy of Virtualization for their community of practitioners with real examples.

Who Tests and Validates the Mashups? The Cave Man?

Joe McKendrick recently commented in his Fast Forward blog, Mashups: So Easy a Caveman Can Write Them? He makes a good case of the continued involvement of IT despite the claims of some mashup supporters. Joe quotes Ovum analyst, Tony Baer, “...no matter how visual mashup tools are, you still need developers or power users at some point of the lifecycle, whether it be to vet objects or sources than can be safely mashed up without violating some corporate policy, or to deal with some complexities of JavaScript under the hood.”

Joe concludes with the implication that even when the ease of use gets to the point where a caveman can do them, there is still a need for IT to play a supporting role in the foreseeable future. I agree since someone needs to test and validate the multiple data sources when business practices are at stake.

Sun Microsystems – Fifth in Our Series on Software Community Blogs

One of our partners, Sun Microsystems, has an excellent collection of blogs. Sun's News Blog has a list of some of these blogs around Sun including Jonathan Schwartz’s blog & other senior executives.  Jonathan is Sun's President and COO and he was one of the first senior executives to start a personal blog. He recently shared a memo he sent to Sun’s leader about the role of technology in helping with the current economic crisis, Innovation Loves a Crisis. He writes that many businesses will be most open to change now and he wants Sun to help its customers achieve greater innovation and boost productivity in response to economic challenges that many face.

Others bloggers include Mike Dillon, Senior Vice President, General Counsel and Corporate Secretary and Dave Douglas, Vice President, Eco Responsibility.  It is nice to see the head of Legal write a blog, primarily covering legal issues as many lawyers area bit shy to blog. Dave Douglas writes about technology and environment.

There is also the blog, On the Record, which covers breaking news and the latest information from Sun and Reviews Interactive blog that discusses Sun product reviews in the press.  This is a great mix of specifically focuses blogs.

Sun is famous for its blog guidelines that lead with this quote – “Many of us at Sun are doing work that could change the world. Contributing to online communities by blogging, wiki posting, participating in forums, etc., is a good way to do this. You are encouraged to tell the world about your work, without asking permission first, but we expect you to read and follow the advice in this note.”  There is much more detail, and their policy of sharing information has served as a model for many other firms.  See our sidebar category, Blog Reviews, for the other posts in this series.

Business and IT Alignment Regains Top Spot for CIO Concerns

The TotalCIO blog highlighted some of the results of the  recent Society for Information Management  survey of top CIOs. IT and business alignment, the Number 1 issue for four of the past 5 years, reclaimed the top spot after dropping to Number 2 last year. The IT talent issues that dominated year ago — attracting IT professionals and retaining IT professionals, both ranked Number 1 last year– slipped: retaining IT pros fell to the 8 spot, attracting talent to No. 4. IT strategic planning, 8th a year ago, is now up in the 3 spot this year, suggesting that hard times call for better planning.

The survey of 291 companies — 350 company executives — was done in June. In the TotalCIO blog post Linda Tucci wrote that, “it would be natural to conclude that economic realities have intruded on the profession.” However they quoted SIM’s Jerry Luftman, author of the survey, who cautioned against reading too much into the changes, noting that in the broad survey “there is no sense the sky is falling.”

It is nice to know that [at the time of this survey...] there is not panic at the CIO level and great to see a stronger concern with IT and business alignment and strategic planning. Of course, given some of the current grim news on financial shakeups and mismanagment, a little panic may intrude on the CIO's prospects, depending on their affiliation. However, this straw poll shows that many are thinking long term, and this should further open the door for better understanding the alignment of business rules with their validation in existing and planned software.

Thanks to Linda for giving us a preview of the result that will be fully released in November at the annual Society for Information Management meeting.

Skytap Provides Cloud Access and Licensing for ISV Solutions; ITKO LISA is an Initial Partner

Skytap, Inc., a leading provider of cloud-based virtual lab solutions, today announced its Independent Software Vendor (ISV) Alliance program to enable on-demand distribution of commercial software solutions through its Virtual Lab platform. Skytap's initial ISV partners include ITKO LISA. Skytap's Virtual Lab automation platform, combined with ITKO LISA, will enable customers to dynamically spin up entire IT environments for service-oriented testing or training labs.

I'm here at STARwest in hot Anaheim, CA, and a lot of the people I'm talking to are mighty curious about how Virtual Test Labs happen in the Cloud. For us, the ability to Virtualize a service-based application, and provision it with Skytap's virtual hardware, represents a great way to get the validation you need without the provisioning nightmare.

 

We think this is a great application for where the cloud is going - just like SOA, it is going to need better ways to test and validate the infrastructure (even if virtual) for expected business outcomes. Joe McKendrick mentioned my last post on this in his Service-Oriented blog on ZDnet: "SOA soon part of the Cloud: Hold that thought." The great thing is, if providers and customers are already thinking about these quality issues up front, the adoption of the Cloud model will catch up (and become part of) the wave of SOA.

Skytap allows companies to run their existing applications and virtual machines unchanged to take advantage of cloud economics without a major application rewrite. The company's Web services API and one-click VPN functionality allow organizations to create a "hybrid" IT model whereby cloud resources can be used and managed as an extension of existing onsite IT environments. With the introduction of the Skytap ISV alliance program, customers can now also access software from leading ISVs within the Skytap environment and gain the benefits of a pay-as-you-go model for the ISV software.

Jim Mackay, our CMO, said, "The cost model for using LISA Virtualize as a test and validation environment in the cloud makes perfect sense. In addition to offering Skytap customers complete SOA testing capabilities, LISA Virtualize abstracts away the actual system under test from dependencies on partner systems and third party resources, which means huge cost savings, as well as a much more rapid way to stand up a complex SOA environment."

Validation of cloud services makes them reliable for business. Virtualization of cloud services makes them available throughout the lifecycle. It's like two waves coming together for a bigger, stronger wave. Who can argue with that?