The ITKO Blog: Service Virtualization and Application Lifecycle Optimization - CA Technologies

Intermediating ESBs for SOA: The Final Webinar?

OK, sorry to mix metaphors, but it is time to put this can of worms to bed (???). If you've been following this blog you couldn't help but notice the many posts on the apparent ESB vs. SOA controversy. All of this started with an innocent little interview John did for SearchSOA - where the topic of Intermediating multiple ESBs came up. This article created a tornado of posts from some heavy hitters on SOA topics over the last month.

Given all of that, we decided to have our own little webinar on the topic Monday - and we were glad to find dozens of you Chief Architects and senior IT & Dev Managers attending - and asking some great questions about how to rationalize ESBs with their existing technologies, and enable your SOA initiatives going forward.

You can watch an archive of this informal, and informative webinar, with iTKO's John Michelsen, and Chris Kraus here: http://www.itko.com/site/resources/esbwebinar.jsp. Chris demonstrates a little software how-to in the middle of the session for those to like to see the screens.

Hey, are ESBs evil? We don't think so - we've always considered them to be another valuable building block in SOA for many of our customers. The key to success here is making sure you are using ESB platforms, even if they are of different flavors - for doing what they do best in the integration process. An ESB does not make an out-of-the-box SOA. However with good design and architecture principles, and strong validation and Governance, you can mitigate the risks of working with these technologies and bring a lot of your legacy and transactional technologies into the SOA fold with much more ease.

Virtualization of ESB messaging behaviors (and the underlying apps and mainframes they expose) is another way to address these ESB intermediation constraints between teams - say if I'm on the webMethods (SAG) team and need access to the TIBCO team's messaging framework. Or if I'm using Oracle/BEA Aqualogic and need to interact with services exposed through the SAP XI messaging tools. Our counterpart on the other team may want to limit our access to these systems - especially if they are supporting live business requirements. The ability to virtualize away those inter-team dependencies, and have a realistic, stable Virtual ESB earlier in my design/develop/test lifecycles can take away much of the cost and delivery timing constraints inherent in intermediation.

Is this the final time we'll visit this controversy? Maybe not - as long as there is complexity and change in SOA, which is the case for virtually every decent sized enterprise IT shop, there will be a need to rationalize multiple technologies. After all, we have to keep doing business today, despite mergers, compliance changes and new competitive pressures that make us keep bringing in new technologies.

We invite you to sit in on the session and make up your own mind -- and welcome your comments on the content.

SOA Pushes IT Leadership beyond Traditional Enterprise Architect Role

Dana Gardner recently reported on an interesting Open Group panel in his post, SOA places broad demands on IT leadership beyond traditional enterprise architect role.  At the 19th Annual Open Group Enterprise Architect’s Practitioners Conference in Chicago, Dana moderated a panel on the role and impact of skills and experience for EAs in both the public sector and the private sector.  Joe McKendrick was one of the members and he also wrote about it, At Open Group: business and government face similar SOA growth pains.

The panel members conveyed that the complexity and interdependencies of SOA are pushing IT leadership more toward a business strategy role. As one panelist said, “The enterprise architect is really more focused on trying to bring the organization together around a business strategy or mission. And, certainly, they understand the tooling and how to translate the mission, vision, and strategy into an SOA architecture.” And “what we are really looking for are people who can then move into the business space, who have a lot more of the softer skills, things like influencing.” You can find the complete transcript of the session through Dana’s post.

Technical skills remain a necessary but no longer sufficient requirement for IT leaders in the new SOA space. I certainly agree with the premise here. To support this requirement, we are constantly striving to make the enabling technologies of SOA more transparent, so that architecture is more of a business process and Policy activity than a programming one.  For instance, enabling the "auto-discovery" of a service and its downstream dependencies, and providing that as a view without all of the technical poking and prodding usually needed to get a diagnosis.

As you go through testing and validation of all the interconnected pieces of SOA, it is important to keep the business side and impacts in mind.  As one of the panelists said, “we are software architects, but we are really trying to solve the business problem.” It is nice to see a group of IT industry leaders taking this position.

More on Cloud Computing from Dion Hinchcliffe

Dion Hinchcliffe recently provide a comprehensive overview of the status of cloud computing for the enterprise in his post, Enterprise cloud computing gathers steam. It is a good article for spinning your knowledge up on what this trend means today.

He starts with the comment, “The days when organizations carefully cultivated vast data centers consisting of an endless sea of hardware and software are not over, at least not yet. However, the groundwork for their eventual transformation and downsizing is rapidly being laid in the form of something increasingly known as “cloud computing … The promise of cloud computing has captured the industry’s imagination this year for two big reasons. The first is the growing realization that cloud computing can successfully be used to strategically cut costs and drive innovation. And the second is that current offerings are getting very close to being ready for prime-time use in enterprise environments.” 

Dion writes how cloud computing can address the two main challenges facing IT today, cutting costs and supporting innovation, issue that are often at odds with each other.  He goes on to list a series of benefits around the cloud such as reduced capital expenses, scalability and performance, security, and service orientation.  Dion then predicts that ultimate future of cloud computing will be with an increasing emphasis on switchable, federated services and less on proprietary, centralized services.  At iTKO we have long been fans of the Federated approach to IT and SOA Governance.

I think that Dion is making some very valid points. However, the benefits of the Cloud are also balanced by the risks of ensuring quality in the lifecycle, and potential costs and dependency. For this discussion, we're more concerned with the applications and business workflows we are supporting, and not necessarily the value of shared computing resources, which are more self-evident.

Quality and Validation in the Lifecycle of the Cloud: The use of data from multiple sources through the cloud raises the issues of testing and validation in spades.  This goes beyond security and raw performance/uptime concerns. As with SOA, continuous testing and validation of all these combined data sources from the cloud migrates the risk of invalid data getting mashed up in your work flows and your business rules going off track. In addition, when you have a malleable set of data and services, it makes finding a "control group" for your own development far more difficult -- how can you test against a changing baseline of data? Virtualization of realistic cloud behaviors and data lets the team get through the lifecycle, to the point of implementation testing in isolation of all that chaos.

Incremental Cloud Costs? The benefit of incremental and subscription-based software in the form of SaaS is very appealing compared to the big bang implementation. However, once you actually achieve a business dependency on the Cloud, the odds can shift in the favor of the supplier. If every transaction or access point represents an incremental cost, what happens if those costs are exceeding estimates, or if your transaction costs for testing and developing against the cloud exceed the cost of ownership in today's model? Here again, we need to consider how to pay for the transactions that make money (through a paying customer), while reducing the costs of transactions that cost us money (during our own dev, test and monitoring lifecycles). Behavioral virtualization of Cloud services and data addresses this risk, and we'll address this in further detail in relation to Cloud Computing.

I would add these issues to Dion’s list and to the list of concerns that IT departments need to address as they consider a move to the cloud.   

Agile Commons – "A Community to Support Software Agility"

I have added a new blog to the list of blogs we read. Agile Commons is run by Rally Software as a means to share information about Agile development.  Rally provides Agile lifecycle management solutions. As their site states, “Rally unites Agile project management with tracking of requirements, tests and defects, so everyone has a real-time picture of your project’s features and quality, priorities, roadblocks and risks.” There is a lot of great content to support Agile development is available through their site and through Agile Commons.

The Agile Commons blog is a collective effort of the community and you can request an invitation to join.  There are plenty of comprehensive and useful posts. Luke Hohman just did a three part series on Why Prioritizing Your Product Backlog for ROI Doesn’t Work. Luke starts with the premise that pundits tell developers to prioritize their backlogs to generate the best possible ROI. But he has not found an Agile teams that does this. Then he explains why using ROI for prioritization does not work in three comprehensive posts.  He basically feels that it is hard to get good data and is thus a fool’s errand.

Instead of ROI, Luke suggests gathering stakeholder preferences, making sure efforts align with corporate strategy, and look for initiatives that drive profit. These do seem more concrete than ROI.  There are many other excellent bloggers contributing to this conversation. Building a knowledgeable community of Agile Coaches and experts is a good strategy for Rally.  Luke’s post and a number of the others, such as Rachel Westin and Evan Campbell,  are related to topics explored at the Agile 2008 conference, August 4-8 in Toronto.

Through the Agile Commons blog you also gain access to the Rally Community, Agile University and Agilistas – a collaborative community dedicated to evangelizing and advancing Agile practice. All of these resources are worth checking out. 

Blowing Smoke? 3 Tests for the Cloud's Future

We couldn't help but notice how fast Cloud Computing has gone to the top of the hype cycle over the last month. You could say the Cloud is "the internet" in general and it has been there all along, and in fact it may have been referenced 10 or more years ago. This time it is new and improved, and it is really getting attention as a philosophy that is broader than SOA - you can't click twice in your IT news feed without sniffing this version of the Cloud.

Alas, this is probably the time that the big SaaS and internet platform providers like eBay and Amazon, Google and Salesforce, really come of age. Joe McKendrick recently put out some good coverage on this on his FastForward blog, "Cloud Computing: Uh Oh, Now It’s Getting Serious".

To us, this is not a mind-blowing concept -- it is a natural move for services, and applications to "live" in the cloud. However, there are 3 tests that Cloud Computing will need to overcome if it is indeed going to become the air we breathe:

1. There is no "free beer" for consumption of the Cloud.
Just like we saw a few years ago in SOA, there a perception that Cloud resources will be freely available for use in designing and building your architectures. But that is not a sustainable model. And no it's not going to be supported by advertising (on your iPhone for Facebook, etc.). If someone is offering an application that we rely on, they will need to ensure that they cover those support costs, and that the consumers are using the applications in ways that can be supported without unduly impacting other users of the cloud. Strong Validation of new offering and intended uses puts responsibility for reliability on both the producer, and consumer of Cloud resources. If there is healthy competition in the Cloud Computing market, the most reliable applications will win the most subscription dollars or usage fees.

2. There is still dependency, in fact, dependency may increase.
So if you are relying on the Cloud to provide data, transactions and other services, how can you test and validate your own business workflows and continue to evolve, if the Cloud may either not be available or ready to take your team's test and development transactions? Lack of ability to validate your own business applications against Cloud resources creates a huge bottleneck to productivity and agility. This is where behavioral Virtualization of the Cloud becomes of primary importance. If you could capture and simulate your expected interactions with the Cloud, that would eliminate a lot of the dependency and bring back the agility you expected.

3.  Fad resistance. (Uh, what were we talking about again?)
When the Cloud hype dissipates, how will this IT approach be driven by business, and not by the "coolness" of the technology? The world's leading companies will be taking more cautious wing-flaps into the Cloud - after all, they are still working on integrating and consolidating their most critical live applications for end customers. Vendors will continue to attempt to assert dominance over the definition of Cloud Computing - you might call it a shared Grid for a variety of apps, or a suite of apps running as a SaaS on a hosted provider's servers. Interoperability definitions and standards will be in flux. Nobody wants to be left behind when the definition and practice continues to change, and it certainly will continue to evolve.

We have been asked about this a lot lately, in terms of how to test, validate and virtualize Cloud Computing environments. And what we've found is that most of the same concepts that we saw in SOA apply to the Cloud - if you are going to rely on these services and apps for business, the risks have to be mitigated with continuous testing, and you need to isolate your own teams from dependency on the Cloud by virtualizing those behaviors. In this way, you can reap the benefits of ubiquitous availability of Cloud resources (services and data) as a platform for your apps, while still being able to productively design, test and develop (and keep working) when the cloud isn't available.

Anyway, this is just an initial take on the topic. We will be there at SYS-CON's first "cloud computing expo" in San Jose in November, and talking much more about how cutting edge companies are making the cloud work for business in the future.

Overcoming the “If Onlies” of Application Development

Forrester's Carey Schwaber recently wrote a nice post on the typical obstacles that application developers face, The "If Onlies" of App Dev. For example, if only “requirements were complete” or “there were enough time to test at the end of the life cycle" or "we weren’t constrained by legacy" among others.  Using a transportation metaphor, Carey then suggests that changing the vehicle through approaches such as SOA and Agile processes can negate the problems raised by the 'if onlies.'  I especially like her insertion of the image of Homer Simpson’s dream car as a note of irony.

She asks her readers about their 'if onlies.' It is a good question for people who create products that support application developers. It is also a good question for the architects, testers and developers delivering functionality for end consumers. We are constantly to talking with clients about their 'if onlies' to discover new ways we can change their means of transportation. It made me think: what are the 'if onlies' our customers are saying about their own process?

Testing:

• If only we had the [time/budget/resources/tools] to test... or "If only management cared more about quality before the project due date"
• If only the [QA team didn't see UI testing at the end as application testing] OR [development team didn't see unit testing as actual test coverage], we would have better delivered quality.
• If only I could use my component tests in regression and then load testing phases it would save us a lot of trouble.

Validation:

• If only I could invoke a behavior somewhere in my application, like at the UI or web service, and then verify it somewhere else in a database or other system...
• If only BPEL provided enough information to validate my business process (sorry it won't)...
• If only I could get business stakeholders like analysts, partners and customers involved in validating my applications.

Virtualization:

• If only I had access to the mainframe for more than 2 hours a week
• If only the transaction or information Service my application leverages didn't charge me fees even for my test transactions while I'm developing
• If only we didn't "go Agile" I wouldn't be waiting on the other small team to finish developing their own component before I can continue developing and testing mine...
• If only I could have my own test environment including our SAP
  financials, Oracle order management, multiple databases & CORBA,
  etc. that I can't just virtualize with hardware methods, without
  basically doubling our team size and budget to support that instance!

Thanks to Carey for this image to consider when facing the challenges of application development. I think if more AD vendors prioritized their own development efforts around the "if onlies" of their customers... there'd be less of them.

Appian Attempts to Move Up the BPM Food Chain

Sandy Kemsley at Column2 recently reported that Appian, one of the "pure-play" BPM tool providers is attempting to move up the food chain by taking on $10million in Venture capital. One of the objectives for this funding is to enhance their Appian Anywhere SaaS BPM platform. They also want to take on the big players in this space like IBM and Oracle so they need more marketing clout.

I was interested to read several things. Their CEO Matt Calkins said that they’re creating a separate Appian SaaS sales force. It includes a sales model that compensates them based on projected 3-year revenues. This makes it easier to compare the on-demand and enterprise sales teams.

More importantly, Sandy reported that “they plan to shift to a SaaS “metabolism” for Appian Anywhere, allowing for frequent code releases in the on-demand platform, while only occasionally freezing that code base for the Enterprise edition. This keeps the two versions in sync, but provides a functionality advantage to the Appian Anywhere customers, who will see the new features earlier.”

I was also pleased to see that Sandy included in her links, one of our posts on BPM testing & validation: How & Why? She said that “often, companies test BPM in a pretty ad hoc manner; this post discusses how to get some discipline into that activity.”  In this post Chris Kraus and I went into a method that companies can better handle the types of changes that BPM providers like Appian bring and keep testing and validation of business processes under control, especially with more frequent code releases that SaaS models like Appian Anywhere offer.