The ITKO Blog: Service Virtualization and Application Lifecycle Optimization - CA Technologies

Oracle Blogs - Third in a Series on Software Community Blogs

This post on Oracle Blogs is the third in our awareness series on the community blogs of enterprise software firms whose customers we work with on a daily basis. Oracle has a blog community portal - It says, “Welcome to the Oracle blogging community, where Oracle executives, employees, and non-employees alike exchange views about customer requirements and best practices for using Oracle and industry-standard technologies.”  There is a long list of employee and non-employee blogs listed by topic.  Employee blog topics include: Database App Development, Database Management and Performance, Middleware & SOA. The non-employees blogs also have blogs under these topics. There are more topics in both cases. It seems like there are more non-employee blogs and customer activity. Technorati reports 2,523 blog reactions to the Oracle blog community as a whole so it is well read and commented on.

Under the Middleware & SOA topic I found the David Chappell Blog. David is vice president and chief technologist for SOA at Oracle. Chappell has over 20 years of experience in the software industry and is the author of Enterprise Service Bus (O'Reilly, 2004).  I liked his post, ROI by the Ton -- Going Green with SOA, EDA, RIA and Web 2.0, where David discusses the business value of SOA.  He quotes a David Linthicum blog post on "Is SOA Green." We like David and his Real World SOA blog is on our blog list. David wrote, “the more efficient a business is, the fewer resources, natural and otherwise it will consume. Thus, the better the business is automated, and business processes optimized, the more efficient it will be. The primary of objective of an SOA is both efficiency and agility through architectural rejuvenation."

Like David Chappell, we agree with David Linthicum that by limiting "consumption of resources" you are reducing the denominator in the value equation -- so strong automation and reuse directly maximize this return on SOA investment.  He provides an excellent case example from a Verizon implementation of a new Fraud Detection application. As a conventional benefit, the implementation is designed to help reduce power consumption by 99.5%. The new implementation also has 0.5% of its original code - and minimizing the code in deployment to just what is needed reduces the support and maintenance cost of any application drastically over time.

Another blog is SOA Station: Mind the Gap written by Peter O’Brien, a software developer at Oracle in Dublin. He started the blog on his own a year ago and noticed a big bump in traffic when it was added to the Oracle blog community.  Peter focuses on financial services applications. Matt Wright's Blog covers discussions, examples, tips and trick on Service Oriented Architecture from BPEL through to Web Services Management. Matt also works for Oracle. He provides a very comprehensive and interesting post on Using Analytics to Modify In-Flight Processes.

Clemens Utschig writes clemens' soa@oracle blog. Clemens works within the SOA Product Management Team @ Oracle Headquarters. Before moving to the states, he worked for Oracle Consulting Services in Austria, troubleshooting and managing critical j2ee/soa projects within EMEA. Richard Naszcyniec's Blog is written a Consulting Technical Director with Oracle Professional Services where he specializes in system architecture and technology. This gives you a small sampling of the range of the backgrounds, conutires, and professional focus of the Oracle bloggers. The community also includes many consultants outside Oracle who work in the Oracle environment. All in all, a wealth of useful information to mine.

The People Side of SOA

In her post in Elemental Links, Does your SOA lack social skills?, Brenda Michelson recently shared a useful conversation with Hub Vandervoort, CTO of Progress Software.  She wrote that, “Hub is talking to organizations about their awareness and ability to form and retain relationships for the betterment of all parties. Why? Because at the core of SOA is federation. And federation requires cooperation.”  We could not agree more.  In her report on their conversation she covered five core areas: contracts, governance, trust & commitment, and empowerment.

In summary, SOA operates best is best if no single party controls the flow of work and information but everyone is clearly aware of their responsibilities. SOA is about partnerships, not hierarchy.  When you move from hierarchy to federation, then governance is based on trust and commitment.  Transparency supports trust. This leads to the last of the success factors, empowerment. I like the quote form Hub that Brenda offers:

“The real shift [for IT] is not to do what you’ve been doing in the past faster, but shift from being delivery mechanism to empowerment mechanism -- truly democratized computing in a Web 2.0 sense. The end-user is always going to operate on the path of least resistance. Find tools that enable him to create his own solution… mashable desktops, data access through services…”

Brenda closes with a useful set of questions to ask ourselves as we implement SOA based on these factors. I have just given the head lines, you should go to her post for the details.

We recently posted on some of these concerns in Hair of the SOA Dog that Bit You. Because of the mix of services in SOA there is the need for more cooperation. Therefore, when you “get more teams to start consuming the service for different business processes, and bringing more interdependent services into the fold. It doesn't take long for some real problems to arise.”  Hub’s guidelines for the people side of SOA are useful here. 

Technology can help also. In SOA inevitably teams run into the barrier of dependency on services that are either not yet created, or aren't available (such as constrained live mainframes and services that aren't yet created). Then, no matter how good the governance is, the people part can get stalled. Virtualization of these unavailable services will allow the people part to move forward again.

BPM testing & validation: How & Why?

BPM (Business Process Management) practices are becoming ubiquitous in the enterprise environment, just like SOA as a strategy for assembling the technologies that support business. Quite simply, the goal of any advanced enterprise application team is to get to a point where the business can manage its process, as it is executed in the software they and their customers use every day.

Enterprises are selecting from two options to enable their teams to define and assemble business processes:

1. Working with the BPM frameworks provided by their selected integration or platform vendors. These may include tools from TIBCO, SoftwareAG/webMethods, SAP, Oracle/BEA, Progress, IBM and Microsoft.

2. Using a "pure-play" BPM tool atop their architecture to model processes. There are several solutions in this category, including Appian, BEA Fuego, Cordys, Global360, Lombardi, Metasystem, Pegasystems, Savvion and SOA Software.

In either case, the company is hoping to achieve a high level of reliability in the orchestrated business processes, to instill trust that the intended business outcome will actually happen. This trust requires making the leap into validating not just the logic of the business process, but all of the underlying technologies and components that must support BPM.

We've seen a lot of activity in this space, especially over the past year, as companies move from pure integration to BPM orchestration initiatives. Our recommendations for ensuring quality and reliable outcomes usually revolve around orchestrating BPM testing and validation both from a "top-down" business process view, and a "bottom-up" technology component view.

Let's take a look at a typical business process for an Order Management application and simplify it to three levels. You have the BPM team that is defining the overall process, the Integration team that is assembling workflows that support the process, and the individual delivery teams that are programming or offering Component-level services that are assembled within the workflows (that support the BPM-defined processes).

The component-level teams provide tests that bubble up to the Integration team as they assemble the services into a Workflow that conducts one aspect of the BPM process, for instance a service that pulls in the Product Data into the workflow of "Place Order." Then the Integration team conducts their own tests of "Place Order," which become part of the validation process for broader testing at the BPM level of "Order Acquisition."

"Top Down" view: The guys at the top of the stack who are using BPM tools cannot possibly be expected to understand the issues and interactions that are happening at the Integration and Component levels. But they do need to know when an issue arises in their business process as it changes, and be able to report it to the teams who are responsible for delivering the underlying logic. They need to take the effort to Validate their business processes, and trust that the underlying delivery teams have done so at their own level.

"Bottom Up" view: The key here is to sign up each team for delivering a set of tests as they contribute technology components or Services into the Workflows that are assembled by Integration teams. Integration teams can take those component tests as "test harnesses" that describe what each component should do, and then tie them together as they are doing integration to provide proof that the expected workflow is valid when it is assembled from underlying components.

The key here, no matter what tools you use, is to validate business processes with tests that incorporate all of the validations that were conducted at the contributing integration and technology layers. This is the only way that you can trust that the expected business outcomes will happen, and trace the root cause of an issue at the underlying layers where the actual business logic and transactions occur.

Of course we have written and developed quite a bit around BPM testing & validation in this regard. We invite your comments on the challenges or techniques you have discovered on your own journey toward BPM in today's IT environment.

TIBCO Community Blogs – Second in a Series on Software Community Blogs

This post on TIBCO Community Blogs is the second in our series on the community blogs of leading integration and SOA platform software firms.  The TIBCO Community has a blog site. You need to register to gain more access. It looks new and there is only one blog so far, on TUCON 2008, but this blog has many authors, including people outside TIBCO such two of the bloggers on our lists, Dana Garnder and Joe McKendrick. SOA TIBCO is the most popular tag. The most recent post is When Real-Time Isn't Fast Enough: Notes from TUCON 2008 was written during the event.

It discussed a number of topics including an addition to the run time SOA governance, Service Performance Manager, which “cleverly bundles some of its Business Events complex event processing technology to tell you whether you're meeting the Service Level Agreements, at least from a performance standpoint. It takes the assumption that service levels are a classic complex event processing problem, especially given the kinds of highly scaled and highly distributed networks that TIBCO customers tend to have.”

At the same event, in another TUCON 08 blog post by Dana Gardner wrote that CEO Vivek Ranadive used his keynote presentation to describe the need for an "event cloud" to support the demands on the next generation of enterprise infrastructure. He said that “an elevated level of cloud management would allow for complex business events and activities to occur in "real-time" at the huge scale demanded of modern business processes. He said TIBCO's goal remains the same as it has been for years, to the bring the right information to the right places at the right times. Only now is that vision nearing fruition, and the combination of SOA and cloud computing will make it happen.” Ranadive also spoke against having data trapped in databases, preferring a pending era of data portability.

An alternate place to check out is the IT Toolbox group of TIBCO EIA users. It is a real user board - geared toward the bit twisters trying to solve tactical issues, but there are at least a dozen or so active threads going on every day, on everything from legacy enablement to managing BPM inside BusinessWorks.

This was certainly much of what we are finding TIBCO customers doing every day - how do I turn my technology "silos" into something much more flexible that runs on the information bus (the "TIB" in their name), and how do I validate and prove that it will continue to meet changing needs? It was great to join them at this last TUCON and here's to the creation and promotion of the "TIBCOmmunity" of users.

Podcast - Solving the Hardest Challenges with eBizQ

I had a chance to catch up with managing editor Peter Schooff and the team from eBizQ this week at the Gartner AADI summit in sunny (and sometimes stormy) Orlando. Between LISA 4.5 coming out with Virtualization for SOA performance and load testing labs, and the SOA software partnership announcement, there was a lot of ground to cover. Peter captured part of our talk here as a podcast:

http://www.ebizq.net/blogs/news_security/2008/06/solving_the_toughest_soa_chall.php

Hair of the SOA Dog that Bit You

This week we are at Gartner's AADI (Application Architecture Development & Integration) conference, and talking to a lot of managing architects and developers from all kinds of industries. And what we are seeing is fundamentally a recovery from the hangover of initial attempts at SOA strategies.

The story is fundamentally the same: "We were trying to modernize our legacy apps, and then we needed to interact with some third party applications, and an acquisition was involved, and we outsourced some development and/or testing tasks... so we tried building some services on top of that..."

Initially, it all works out pretty well. You take some requirements, put web services in front of some legacy apps, and tie those to a web interface and voila - a new SOA app. The initial results look good, the consumer can accomplish their business tasks. Then, get more teams to start consuming the service for different business processes, and bringing more interdependent services into the fold. It doesn't take long for some real problems to arise.

Moving to SOA does create more flexibility and agility, but it also creates a lot of fragility -- in that all of the services we use have underlying business systems that contain the real workflow, and therefore teams don't have enough access to the systems they need to properly test and develop their components as part of the overall workflow. This can cause huge quality issues later in the lifecycle, as well as a much slower time-to-market and higher costs than expected.

Obviously, hearing the good word from the analysts will help. But so will a little old advice about the  "hair of the dog" cure. If going services-oriented has got you down, the answer is -- more services? Well, exactly. The caveat is, making that foray into SOA is going to be flawed, if it doesn't dispose of the assumption that things will work together as expected.

Re-committing to services can actually turn this situation around. This can only cure the SOA hangover if:

1. We are setting clear Policy expectations for both the provider of the service to meet expectations, and the consumer of that service to define exactly what their intended uses of that service are as verifiable tests.

2. We are providing a system of record for enforcing these service policies, and how they are used to meet a business need. For instance, we recently worked with SOA Software around just this piece of the puzzle. Their governance framework stores tests as a way to manage the enforcement arm of the design/build/change process in their platform. There are many other leading governance and development process tools that can store tests as verifiable enforcement points in this way.

3. We Virtualize the dependencies of services. When we start splitting up teams and the services we deliver, we expect that we will get more agility. However, inevitably these teams run into the barrier of dependency on services that are either not yet created, or aren't available (such as constrained live mainframes and services that aren't yet created). By basically capturing and simulating Services and their underlying layers, these teams can move their development into a more parallel development and testing process at a component level, much earlier in the lifecycle. Time to market and delivered quality, as well as lower environmental costs arise out of this level of virtualization.

Just an initial musing from this show - the quality of discussion has been really good here so far. Look for some more thoughts from our sessions here.

SAP Community Network Blogs – First in a Series of Software Community Blogs

As a point of reference, the challenges and solutions we encounter in trying to manage, test and virtualize major vendor solutions have a source in the dialogue of the user, services, and developer communities that support it. Most of these firms have blog communities so we decided to take a survey of them over the next few weeks, starting with SAP.

SAP Community Network Blogs have been in operation for some time. It covers many topics. It is easy to become a member of the SAP Community Network and that gives you more access. You do not have to be an SAP employee to have a blog, only a community member so there are bloggers who work in the SAP space but are not employees. The community site is well organized with blogs by date, by topic, top blog posts, active bloggers, etc. There are clear guidelines on what you can do.

Many topics are listed including: Business Rules Management, Business Solutions, Composition Environment (CE), Emerging Technologies, Enterprise Performance Management, Enterprise SOA, Governance, Risk and Compliance, and SAP Business Workflow. Technorati reports 5,407 blog reactions to the SAP blog community as a whole, so it is well read and have plenty of coverage in the blogosphere.

I took a look at the Enterprise SOA area. One post, Enterprise SOA on SDN and BPX – The Good Stuff Just Got Better  by Mark Yolton of SAP, commented on how they recently reorganized the Enterprise Service Oriented Architecture (enterprise SOA) pages, information, and tools in the Enterprise SOA section of the community. Mark Yolton is Senior Vice President of the SAP Developer Network (SDN) and Business Process Expert (BPX) communities. He said that the Enterprise Services Community (ESC) area has grown and expanded with new members, groups, and successes,” which is good news to us at iTKO. 

Mark goes on to write, “We know that service-oriented architecture (SOA) is about breaking monolithic systems into components to access valuable business logic as standards-based services. Services can be combined into new services that support enterprise-wide business processes and can be reused to maximize efficiency and reduce total cost of ownership. However, effectively managing your IT with SOA principles means your SOA blueprint needs to reflect business semantics. To achieve this, you need a comprehensive governance process and design methodology that safeguards service quality, stability, and standards compliance.”   These are all principles we support.

Another post, How to approach the transformation to enterprise SOA, by Bjoern Treutel of SAP covers their methodology. A post by Daniel Stollenwerk describes the Enterprise SOA Development Handbook. There is a wealth of material here and if you are leveraging SAP in your enterprise (and that's a huge community indeed...) this community is certainly worth exploring.

Enterprise 2.0 and Testing, Validation, Virtualization

Had a good long conversation with Bill Ives of the FastForward blogs a little while ago. Apart from sharing a lot of information on what is happening with LISA's capabilities these days, he took a little time to enlighten me (and also blog himself) on what Enterprise 2.0 is supposed to mean, and how SOA quality and service virtualization fits into that concept.

I like the mindset - E2.0 isn't a technology, however it encompasses most of the advances we've seen over the last 10 years - SaaS, RIAs, mashups, and the underpinnings like SOA and virtualization. But beyond that it is really about a shift in organizational structure around information. Better leverage and control over information, better collaboration and sharing, more innovation, more agility.

Anyway, interesting topic and worth a read, to see how all of this IT change is being driven by the business need for Enterprise 2.0-ready applications.

Can Virtual Environments take Performance & Load Testing?

We've talked a lot in previous posts about how the practice and technology of Virtualization really has legs -- it keeps moving forward, from hardware virtualization, virtual test beds, to virtual endpoints, to actually simulating the behavior of the software itself, which we're calling Service-Oriented Virtualization (or "SOV" if you need a TLA for it).

Now we are seeing the Performance Lab getting into the action on this practice. For interconnected apps like SOA and serious enterprise integrations, the guys with the load testing firepower have tools like LoadRunner and SilkTest in their lab, but they get left out of the process until very near the end, when an interface is available. SOV can break that dependency of waiting for "all the moons to align" before they can get a test window.

The initial uses of SOV were to allow the development and testing team to regain agility much earlier in the lifecycle - so they could do their needed functional and regression testing against Virtual Services instead of constrained live applications -- the essential services, databases and mainframes in the environment.

But that same virtualization is perhaps even more valuable in the performance lab, if you can apply serious load testing to it. The constraints of having a realistic environment and test data to test and develop against is holding these teams back from finding performance issues much earlier - so we can gauge SLAs (service levels) at the component level. And in SOA - where you are dealing with services and underlying systems that are distributed and constantly changing, replicating that whole environment is incredibly costly and time consuming.

With a virtual service environment, the performance team tests the component they are working on with their existing load testing tools, and virtualize the rest of the system dependencies and data away. Rather than add hardware and bandwidth, just virtualize all of that, then see if it is indeed the hardware, or more likely, something in their component or its response to variable, changing data that is causing the bottleneck.

We had a good conversation with analyst Theresa Lanowitz from voke - no stranger to advising companies on the application lifecycle and ensuring quality - about this concept. She's going to be co-hosting an upcoming webinar with iTKO's chief geek John Michelsen, and InfoWorld's Test Center editor Doug Dineley on this topic on May 28: http://www.itko.com/site/resources/vsewebinar052808.jsp

Hope you can join us for this event, if not, we've written a paper on performance testing in a virtual environment, and we'll continue to talk about this practice here.

iTKO LISA Soapbox Overview

Welcome to the updated iTKO LISA SOA Testing, Validation and Virtualization Soapbox! John Michelsen (founder & chief geek of iTKO) and other iTKO experts (see side bar list on the right) use this blog as a sounding board for issues, events and trends within enterprise IT. We cover how enterprises are managing the risk of increased change and complexity in IT. As business applications evolve, through techniques such as SOA governance, BPM (business process management), rich Internet applications, and highly distributed integration of shared services and legacy technologies, our methods and tools need to help us keep up with this increasing rate of change and complexity.

Though this blog is coming from a software provider, tools are just a piece of the puzzle. Achieving success across the teams that design, build and change our enterprise applications requires a higher level of trust across teams. We lived a life as a custom development firm in a service-oriented world before we were even calling it SOA.

We invite you to look into our past posts and not just our latest content. Many of the past posts remain current as we aim to discuss issues and not simply provide news. Browse our categories and use our search field. So we hope you will find some insights and ideas here, and we welcome your comments and questions on these topics.

John Michelsen

John has over fifteen years of experience as a technical leader at all organization levels, designing, developing, and managing large-scale, object-oriented solutions in traditional and network architectures. He is the chief architect of iTKO's LISA automated testing product and a leading industry advocate for software quality.

Before forming iTKO, Michelsen was Director of Development at Trilogy Inc., and VP of Development at AGENCY.COM. He has been titled Chief Technical Architect at companies like Raima, Sabre, and Xerox while performing as a consultant. Through work with clients like Cendant Financial, Microsoft, American Airlines, Union Pacific and Nielsen Market Research, John has deployed solutions using technologies from the mainframe to the handheld device.

Jason English

In a market crowded with "silver bullet" solution claims, Jason ensures that iTKO's message and solutions are tightly focused to reach the customers who need them most -- enterprises who want to offer their constituents value by ensuring flawless development execution and quality.

Jason joined iTKO in January 2004, bringing more than 15 years of experience in executing marketing plans, re-engineering business processes and meeting customer requirements for companies such as IBM, EDS, Delphi, TaylorMade, Sun, Motorola and Sprint. As Director of eMarketing and Executive Producer, in2action Consulting at i2 Technologies, he was responsible for i2's outbound messaging during a period of extreme growth, as well as marketing services and working directly with clients to build easy-to-learn front ends to B2B systems. Prior to that, he managed customer experience as an Information Architect at Agency.com. Jason scored and designed several internationally released computer games in addition to conventional
print advertising and television commercials.

Chris Kraus

Chris is an expert testing and architecture strategist, with an 18 year background in computer software development, product management, and sales support for enterprise software.  As Product Manager for iTKO's LISA suite, Chris applies both project management and development experience to refine LISA features to best suit customer quality needs throughout the software delivery lifecycle.

Chris was previously a Retail and Manufacturing Industry Manager at the enterprise software platform firm webMethods, overseeing requirements, customer presales and training for the $16M annual group. At supply chain software provider i2 Technologies he worked in i2's infrastructure group, with responsibility for the release of business process, workflow and monitoring engines. Prior to i2 at Software AG, Chris specialized in cross platform product installation and administration.  As a software engineer, project manager, and solution architect, Chris has a great breadth of industry knowledge from working with companies like Citi, TD Ameritrade, Lenovo, Tandy, Rubbermaid, TI, and TxDoT to ensure quality application delivery lifecycles. 

Jim Mackay

Jim has more than 20 years of experience in delivering products and technology for enterprise computing. He has held key positions in leading technology companies including SVP Solutions at webMethods, CTO and SVP Product Marketing at i2 Technologies, Numetrix (now Oracle), and Marcam (now Invensys).

Jim is very active in the technology community, delivering presentations at key events as well as publishing numerous articles. He is an early investor in, and sits on the board of advisors of a number of promising software companies. He also holds two patents related to integration.

Shridhar Mittal

Shridhar joined iTKO as CEO in September 2005. He has an accomplished executive career leading global teams in marketing, sales, implementation, and business development of software applications. Prior to iTKO, he held several key management roles during a 10-year tenure at i2, including CMO, SVP and GM of the High-Tech Business Unit, and VP of eBusiness solutions. His deep understanding of sales management, product management, development and end customer business needs were essential in helping i2 grow from a startup to a world-class enterprise software company. He started his career as an applications engineer for Ford Motor Company.

Shridhar holds software and process engineering patents, and earned Ph.D. and M.S. degrees in Industrial Engineering from Purdue University and SUNY Buffalo, respectively.