The ITKO Blog: Application Lifecycle Optimization

Fast, Cheap & Good? Virtually Breaking the Limits of Software Development

We had a well-attended ITKO webinar with analyst firm voke on Lifecycle Virtualization (see an archive here) with almost 200 live attendees yesterday - clearly a lot of interest in these new developments, and thanks to all who attended.

In the first half, voke's Theresa Lanowitz described her research on Lifecycle Virtualization (LV) as an emerging technology space that encompasses several major vendors and niche players in Service Virtualization, virtual lab management (VLM), cloud platforms, network and device virtualization. She predicts an LV market will come to fruition and demonstrate value for IT by 2013 -- not so far off for something so new. But then again, remember how recently Cloud was a new concept?

As everyone knows by now, you can't simply cost-cut your way to differentiation and leadership in these competitive times. Hardware and desktop virtualization made a big one-time impact on IT costs by reducing the number of Intel boxes, but now what? Is there opportunity for another field of Virtualization that could be focused around meeting customer needs?

For most industries today, your ability to successfully deliver software is your ability to deliver business value for customers. Or, as Theresa put it more simply: "Software IS the business." Software touches everything, from sales, to manufacturing and delivery, to back end accounting and strategic planning. The ripped-from-the-headlines software failures she mentioned yesterday in banks, telcos, retail are moribund, and making headlines every day.

A couple of the more pointed questions we got during the webinar really made me think about how far we've come since the limits of the Rule of 3 in software development:

"Fast, Good, or Cheap Software:
-- You can't have all 3!"

In fact, simply making strides against this rule was a main goal of ITKO's in the very early days (around 2003-2006) when we were selling quality assurance software for complex architectures like J2EE and SOA. Hey it's not cheap to get skilled programmers and testers for these applications, so do automated component testing earlier, save some time and money and avoid costly project failures. There was just one missing piece though: How could we move this lifecycle forward faster as software became more and more distributed, while not having access or control over so many systems we need?

Service Virtualization is already becoming a proven and essential practice, with solutions from leading vendors -- and it will fundamentally reset the value equation of software development. Call SV part of Lifecycle Virtualization if you like, but it definitely fits the bill of replacing physical and manually copied development infrastructure with automated, virtual infrastructure.

This new virtual environment, whether on-premise or in the Cloud, is flexible enough to allow software development and testing activities to happen in parallel, at much lower cost and extreme efficiency. Yes we can now do Better, Faster, Cheaper - all of which enable the enterprise to deliver software that perfectly meets customer needs.

Thanks for your questions yesterday, and we invite you to recommend yesterday's webinar archive replay now on our site, and download a complimentary licensed copy of voke's latest Market Snapshot overview on Lifecycle Virtualization if you have a colleague who might benefit from this info.

Next up on the ITKO Blog: Who Manages Cloud Chaos??

Webinar 1/18: What is the Lifecycle Virtualization Market, and Why Service Virtualization Matters

You've seen a rapid barrage of market news in just the last 12 months about Service Virtualization (or SV) technology lately. CA buys ITKO for LISA's Service Virtualization capabilities, and many more acquisitions and announcements from major players in this space happening. Why now?

We've been talking about this practice of SV for almost 5 years now at ITKO, but clearly all these companies don't arrive at these conclusions unless there is a market movement afoot with customers. Indeed, Service Virtualization is part of a bigger shift toward more agile development using a new form of virtualization in shared environments such as private and public DevTest Clouds that will allow increased parallel development, with on-demand capacity for performance and earlier regression testing, thereby speeding up the overall application development lifecycle.

One analyst we respect on these matters, Theresa Lanowitz of voke, has a term for this new space that floats above the conventional requirements, development and test tools of the ALM portfolio - she calls it "Lifecycle Virtualization."

Here's a chance to learn about Lifecycle Virtualization and this new generation of technology encompassing Service Virtualization capabilities, and much more. Join Theresa and ITKO/CA "Chief Geek" and founder John Michelsen for this live webinar Wednesday, Jan. 18th:

Webinar: Lifecycle Virtualization - A New Practice of Optimizing Application Development, featuring voke and ITKO

January 18, 2012 1:00pm-2:00pm Central

Theresa has been talking to both end customers and software vendors in this space for many years as an IT analyst following emerging trends in application development. Here's her take on Lifecycle Virtualization from her latest blog "Part of An Overall Lifecycle Virtualization Solution":

"Lifecycle virtualization is defined as the use of technologies such as virtual lab management, service virtualization, defect virtualization, device virtualization, virtualized cloud platforms, etc, to enhance the application or product lifecycle through reducing defects, lowering costs, speeding time-to-market, and increasing customer satisfaction."

We hope you attend this webinar and find out how the application development industry is evolving to meet the difficult challenges of complex, interconnected systems and increasing customer demands for high performance, flawless service. Call it Service Virtualization, Lifecycle Virtualization, or whatever you like -- customers are realizing the value of much faster delivery with less risk today, while saving millions in infrastructure costs.

Also as a primer, we suggest you pick up the latest voke Research Market Overview paper on Lifecycle Virtualization (available for a limited time) which provides a good Overview of this new technology space.

ITKO retrospective with John & Shridhar in D Magazine

ITKO's own John Michelsen and Shridhar Mittal recently appeared at a Dallas-area business forum to share their unique experience of building a little private software firm into a global software player, and writer Glenn Hunter from D Magazine's CEO business publication was there. Read this interesting personality piece here:

D CEO Magazine: The Triumph of LISA
iTKO scores with a revolutionary software product, thanks to the geek who invented it and a CEO who knew how to grow.

It has been an incredible ride -  less than 6 years from initial funding to reshaping the application development market with Service Virtualization innovations, and then becoming a strategic part of a leading software company like CA Technologies. But this story is far from over...

[link:] http://www.dmagazine.com/Home/D_CEO/2012/January_February/iTKO_the_Triumph_of_Lisa_Software.aspx

Forum: What was the biggest development for Cloud in 2011?

Participating again in the eBizQ forum on Cloud - and starting the process of looking back on a very eventful year for ITKO (and CA Technologies) in 2011, and picking our horses for what will happen in 2012.

The most obvious thing we saw in Cloud was a continued increase in consumerization and public level awareness, thanks to the usual suspects of Amazon, Apple, Google & Microsoft. You would be hard pressed to complete a download of consumer media or software these days without an invitation to claim your own Cloud spot for storing information or streaming your playlists.

But what about big business? How effectively did major companies in other industries embrace the Cloud? I just wrote a little piece on this here: http://www.ebizq.net/blogs/ebizq_forum/2011/12/what-was-the-biggest-development-in-the-cloud-for-2011.php

The biggest development for the cloud this year was, well... development IN the cloud. The categorization of Cloud as "another place to host your apps" provided some incremental efficiencies, yes, but that's a pretty commoditized activity. We always knew the cloud could do more!

Setting up elastic, on-demand dev and test environments in cloud actually turns out to be the best first use case of cloud in the enterprise. As developers need to create new functionality and verify its performance and quality on top of layers of existing stuff, they all inevitably demand their own environments.

We've seen many enterprises that have dev and test environments outnumbering production environments by 7-to-1 or more - and with inconsistent accuracy, as there is a great deal of manual setup and maintenance involved in each of those instances. Let's harness all that excess computing power and pool it inside the company as Private clouds -- and for higher capacity or seasonal needs, instantly provision ready environments in Public clouds. [more >]

Hopefully that makes sense to our readers who live in the application development and testing world -- you are in the vanguard of Cloud simply because these activities are the most natural first step, and there are immediate infrastructure efficiencies to be gained right off the bat.

As for the rest of our predictions and a look back, stay tuned...

Mock & Roll will Never Die

If you already have built a "mock system" in order to advance development, why would you need something better? I recently had a good conversation with our East architecture lead Ken Ahrens (see more of his work here) after a great CA World event for ITKO was winding down in Las Vegas. We had a lot of questions in our sessions about the development shop that has already "rolled their own" system of stubs and mocks - and what that meant for Service Virtualization practices.

When a company says they already have a mock system, we normally view that as a very positive sign - and tell them as much.  Often we have to convince customers that incomplete and unavailable systems are development problems that need to be solved, before we can even talk about the idea of Service Virtualization capabilities. Whereas the customer with a mock system already understands this is a real problem that needs to be solved.

I usually comment at this point about how we’ve replaced several mock systems at other customers (banks, utilities, telcos, etc...) with LISA's service virtualization. SV really is the productization of the stub and mock space, with enough sophistication to support more complex logic and performance scenarios. Similar to how you wouldn’t write your own database nowadays, it doesn’t make sense to build your own mock system when there is an industry standard solution for this available.

Some questions that usually come up:

• Does your mock system meet 100% of your needs and is it used on every single project?
• How expensive is the mock system to build and maintain?
• How fast do you need new mock services built, and are they ready on time?

If a development shop is mocking a single binary transaction -- that is a simple enough problem to solve with a bit of code. But as soon as multiple systems talk to each other, and multiple business scenarios need to be validated - the manual stub approach quickly starts to trip over its own weight, and the process of mocking and massaging test data and environments can drastically slow down delivery.

On one recent visit to an East coast utility, the customer team decided that LISA began to win (from a cost perspective) after only 10 rather finite services were mocked.  One of our favorite banks established a "center of excellence" for enabling teams with Service Virtualization instead of stubs, and measured a full return in about 6 months - then did something even more interesting by rapidly fixing issues found in production by virtualizing production behavior into the dev environment.  A successful customer like this doesn't just buy software, they take the adoption of the practice seriously - and it's usually the guys who have experienced the pain of rolling their own mocks that see the potential benefits the fastest.

In short – if you already do mock services we think it’s a good thing, so tell us about your story!

 

Giving Caesar His Due at Gartner AADI 2011- New Agile and Composite Dev Reports Available

We're heading to Las Vegas again this week to Caesars Palace - at the Gartner AADI (Application Architecture, Development & Integration Summit) 2011 where we are appearing as a Platinum Sponsor, and joining our colleagues from CA Technologies to talk about the ties between ensuring application performance and increasing the agility of application development.

This is the topic of our latest joint Newsletter with Gartner that just came out today (complimentary view/download):

ITKO Insights, with Gartner Research, Issue 2 -
Winter 2011 - Ensuring Agile Composite Application Development


This issue features a new report called "The Three C's of Agile" from two Gartner analysts that cover the App Dev space very thoroughly, David Norton and Tom Murphy. Get your copy of this research here, as well as a couple new papers and articles from ITKO providing further insight into Enterprise Agile and Service Virtualization best practices.

That's one of the biggest themes we're hearing from end customers as they work with the leading analyst firms - "Enterprise Agile" is the next big thing, and it's coming to fruition now. We all know IT budgets have been cut - but today's enterprise cannot cut its way to profitability in an uber-competitive market. Faster delivery of new services that customers demand requires an extremely strong plan for developing and delivery quality to market at a much greater pace.

If you are going to Gartner AADI, please stop by the ITKO booth at the Solution Expo, and reserve some time for our Session on Tuesday with our LISA product manager Chris Kraus: ""Optimizing the Application Lifecycle with Agile Collaboration and Cloud DevTest Labs" Tuesday, November 29, 3:30-4:30 PM, Octavius 16. 

To successfully deliver critical customer-facing services faster, enterprises must first leverage Cloud to optimize software lifecycles. As companies move applications to Cloud, they find these environments intertwined with dependency constraints on other off-cloud systems, limiting the elasticity and overall usefulness of Cloud environments. This session discusses how multiple software teams increase agility and collaboration using virtual infrastructure in DevTest clouds. Time-to-market is accelerated when lab infrastructure and data scenarios are in place on-demand, enabling teams to “shift quality left” and eliminate issues early.

 

 

 

Get the voke Market Snapshot on Lifecycle Virtualization

At CA World today hearing from voke analyst Theresa Lanowitz introducing her latest independent research on the practice of Lifecycle Virtualization. Theresa defines this as a new space that encompasses the existing concepts of Virtual Lab Management, coupled with newer capabilities such as Service Virtualization, Device and Network virtualization that eliminate constraints and costs from software development tasks and enable faster time-to-market.

We've licensed this new report for a limited time, and you can download it here: voke Market Snapshot report: Lifecycle Virtualization (.PDF download, 313K)

The report is especially interesting because it is the first document we've seen that defines a new space of virtualization as it enables faster, unconstrained development and test of today's complex, highly distributed applications. Lifecycle Virtualization encompasses many of the new technologies and practices that enable development environments to work beyond the existing, well-known definition of Application Lifecycle Management (or ALM).

Here's an overview of the research from voke's press release:

The report highlights the importance of Lifecycle Virtualization, which leverages virtualization technologies for development and QA teams to enhance the application or product lifecycle. Alongside desktop and server virtualization, lifecycle virtualization serves as the third pillar in a truly virtual environment. In fact, Lanowitz believes lifecycle virtualization has the power and potential to deliver more impact to the market than both server and desktop virtualization, by finally solving age-old and classic challenges of software development and delivery.

Lifecycle virtualization includes the following solutions such as virtual lab management (VLM), virtualized cloud platforms, service virtualization, defect virtualization, and device virtualization.

We hope you find the free report useful in your planning and research, and look for more articles with voke as this firm continues to survey customers employing this new technology and the practices around it.

Forum: What Don't Companies Get about Cloud?

Recently participated in a forum thread from our friends at eBizQ: "What do companies still not get about Cloud Computing?"

A lot of this uncertainty stems from the old stalwarts of "Is the Cloud secure? Is it scalable?" -- and I think the panelists here debunk those fears pretty well. There are many well-secured ways to take advantage of Cloud, with almost unlimited scalability -- a lot more effort has been put into that at the big public cloud providers than most companies' own web and firewall strategies.

Now, let's get down to SLAs, or service level agreements in the public Cloud - many of which are starting to improve dramatically, while most still pretty much disavow responsibility for interruptions and downtime. Caveat emptor.

That said, Cloud is already happening, and is still going to happen - there is no way to escape the business appeal of provisioning IT resources and software in an on-demand basis. We are seeing many of the world's biggest customers first pooling resources within their own enterprise and rapidly obtaining a Private cloud, while thinking of Public cloud only for the highest capacity or "edges" of their applications.

Here was my take on this topic - read more at http://www.ebizq.net/blogs/ebizq_forum/2011/08/what-do-companies-still-not-get-about-cloud-computing.php:

"Yeah we gotta do it. But who owns it, and who buys it?" Cloud as a term can cover so many departments and responsibilities within an enterprise.

Is it general corporate IT infrastructure (stuff like shared drives, backups, security & email)? Or is it the company's revenue-driving edge where business functionality is developed and delivered to partners and customers (commerce and differentiated service offerings)?

Often we are seeing larger companies with a more innovative bent hiring a "Cloud Czar" type executive or guru -- with a responsibility to overlay multiple organizations and map out a Cloud strategy. It's not always optimal, but at least then you have someone looking at the bigger picture.

Anyway, we recently had a great little meeting of the minds with some of our customers on this particular Cloud topic as well as the "Czars" and front-line job skills and talent pool of the future in our industry, so look for more interesting material on that coming soon.

Take the voke Survey: What is Agile to You?

"To err is human, to err faster is Agile."

Just got a request to take an Agile Survey from our friends at market analyst firm voke, inc:

We are in the process of conducting a survey on Agile. We will use the data from this survey to create a voke Market Snapshot on Agile, the report will be vendor neutral. I know your customers have very strong opinions about Agile and welcome the participation of as many as possible.

So - here is that survey link: https://www.surveymonkey.com/s/LNCVQ6H

I took the survey myself and it made me rethink exactly what Agile meant to me - hence the quote at the top. Funny, this comes up almost exactly a year after this blogger commented on Mark Masterson's killer video about Cloud enabling you to fail faster.

Like many terms in the tech world, "Cloud" and "Agile" often get misappropriated for somebody's marketing agenda. But both terms have the intrinsic value of mobility that the business side of the enterprise craves from IT: smaller incremental efforts and costs, and deferred larger decisions, so they aren't going away anytime soon.

I'm quite curious about seeing the results and comments our readers might add to the mix of this survey and the resulting report. ITKO customers generally tend toward the leading edge of what we'd call Enterprise Agile, but we will wager only 50% of them actually refer to their own efforts by the term "Agile." Let's find out!

News Today: CA Technologies to Acquire Privately Held ITKO for $330 Million in Cash

Things are heating up here in Plano, TX. And I'm not talking about another 100-degree summer day: ITKO has signed a definitive agreement to be acquired by CA Technologies.

Read the full press release here: "CA Technologies to Acquire Privately Held ITKO for $330 Million in Cash"

This is great news for our customers, partners and employees. At ITKO we have long believed maintaining a vendor-neutral approach is a requirement of any large-scale, heterogeneous enterprise IT environment, and CA Technologies shares our vision of supporting “any application, any technology.”

After hundreds of customer engagements over the last 5 years, we've learned our LISA suite, and our capabilities for Service Virtualization are most effective when used to eliminate the most difficult and costly constraints, and solve the world's most complex development, testing and integration challenges. With CA, we are now part of an organization with the reach and capabilities to truly solve the biggest problems enterprise IT faces.

Conveniently, we have seen many of our large enterprise customers also running CA's Application Performance Management (APM) solutions such as Wily Introscope and Net QoS to assure higher service and performance levels amid changing conditions. These joint customers are tackling large scale modernization and migration projects, and seeking new ways to move their applications to the Cloud with less risk.

You can still call us ITKO, and our team will continue to support customers with the service and strategic approach we're known for. But the overall picture of what customers can do to optimize the entire IT life cycle will get a whole lot bigger.

There are many more synergies and advantages to be realized from today's announcement, so look for future posts here on this blog (itko category: "CA") as well as reading pertinent articles on CA's very useful blog "Service Assurance Daily" over the coming weeks.

Webinar w/ Infosys 6/15: Solving the Hardest DevTest Challenges with Service Virtualization

We're looking forward to our third major webinar with Infosys, in which we wrap up our joint series of events on accelerating testing and collaboration for now -- with one covering the best practices of Service Virtualization.

Many of the world's leading companies trust their delivery and testing business to Infosys, and therefore they are particularly qualified to know about how to achieve higher levels of efficiency across large, geographically distributed services teams, through eliminating constraints and isolating the right conditions for validating that functionality gets delivered as expected.

Webinar with Infosys and ITKO June 15^th, 1:00PM CDT:
"Solving the Toughest Development and Testing Problems with Service Virtualization"

Join Infosys' Aparna Sharma and iTKO's John Michelsen for this Service Virtualization best practices and technology review webinar. This session for ITKO and Infosys customers and subscribers, will cover:

• How to break down silos that exist in complex business processes to enable end to end business process testing
• Parallel development and testing strategies
• How to provide software dev & test teams with a more production-like environment with Service Virtualization, and
• Performance and Regression Testing against Virtual Labs

Hope you can join us for this compelling online event - it's your best chance to hear from two thought leaders in software delivery. Bring your questions!

Impact2011 Preview Video - LISA for WebSphere, SOA, BPM DevTest

Here's the follow up to last week's post on "iTKO Opens the Throttle for Agile App Dev at IBM IMPACT 2011 - Platinum Sponsor" -- which outlines 4 key sessions to attend at IBM's Impact2011 conference (April 10-15, Las Vegas - register here). In this new Chalk Talk video, John outlines some of the reasons IBM technology customers will learn something at the event:

 

Key approaches John covered in this latest Chalk Talk video - relevant to anyone in enterprise IT but especially tailored for IBM users:

1. Service Virtualization
2. Collaborative Defect Resolution
3. Continuous, End-to-End Testing and Validation

The reality of enterprise applications today is that complexity and rate of change will always increase. So if your company is doing IBM-enabled SOA, BPM, and seeking aspects such as data management and higher performance from applications, it will behoove you to explore ways to increase your agility and open the throttle with some of this new thinking.

On the Forums: Is Cloud Computing Now a Business Necessity?

How widespread is the use of Cloud now, and is it a business imperative, or just another deployment option? We recently commented on this on our favorite forum at eBizQ:

 http://www.ebizq.net/blogs/ebizq_forum/2011/01/is-cloud-computing-now-a-business-necessity.php

I thought Lori MacVittie's comment that "... the "business" doesn't need cloud computing. It needs applications." was very appropriate. True, business isn't usually driving cloud adoption. The IT side of the business has the most immediate gains to realize, having to offer more computing capacity while responding to budget pressures.

However we are seeing two current trends that are very real. Some major companies have identified Cloud as more than a different hosting option, or another word for leveraging somebody's SaaS applications. They are mandating that any business activity, including not just using functionality, but building and testing the applications themselves, must happen in the Cloud.

On top of that, you have what Alain Rap calls "cloud creep," which is the inevitable march of individuals within companies going ahead and using Cloud to get around administrative and procurement issues. Here's what John said about it:

It'll be like Palmolive. You're already soaking in it!

I would say right now about 20% of Global 500 companies identify full Cloud-enablement as an important strategic initiative. But what about the rest?

It may not be defined now as a necessity, but for many companies it is already becoming a reality - you have individual developers and business unit heads, tired of a multi-month IT cost justification for provisioning new servers and elastic capacity to test and run their apps... and many of them are just going out with their credit cards and purchasing some Cloud capacity or leveraging business services on their expense accounts.

What do you think?

Virtualization or Cloud? Both or Neither? Why or Why Not?

Recently commented on eBizQ's Forums about a question that on first read seemed rather silly: Is it better to use Virtualization for some business apps rather than the Cloud? Here's what I said about it:

"It's hard to picture how it is better to use one than then other, when both could be perfectly applicable. I would expect virtualization to be a commonly accepted precursor to leveraging cloud-based environments for applications."

The discussion here, especially in posts by Ian and Lori, does a great job of making sure we aren't lulled into thinking of Virtualization and Cloud as either/or propositions. You are talking about two totally different things. But let's play devil's advocate and discuss how they are not very different.

1. Near-Ubiquity. Virtualization, as in hardware/desktop VMs and system images, is now par for the course. And we expect Cloud Computing as defined here to also become a major part of every enterprise IT strategy - in fact, we think it truly arrived this year. Therefore, both are present in most major IT shops already.

2. Complimentary Value Proposition. Obviously, you have compelling reasons to do both - I like Satoru's quip on "selfishly" utilizing several different Virtual images on an Amazon EC2 Cloud environment - perfect for quickly launching or freezing several different versions of his software lab depending on their reliability or versioning needs. To an extent, you can use both for similar value reasons: cost efficiency, more optimal use of computing resources (and/or capacity), system flexibility or elasticity... again, different things but similar value props.

3. Constrained by External Factors. Lastly we have discovered that Virtualization and Cloud are most similar in what they CAN'T optimize conventionally. Almost the same behavioral rules apply here - if it's a system or server you have access to and control over, it is easy to image as a VM, and also logical and cost-effective to deploy it in a Cloud. If you are dealing with an Over-Utilized (high traffic and size) system, or one that is Out-of-Scope (external to your dept. or a major service outside your company's authority), BOTH hardware virtualization and Cloud quickly become ineffective or constrained. Here's what I commented on that:

"There will still be some over-utilized business app resources that are also resistant to both - it doesn't make sense to try to replicate a 2TB mainframe as a VM, or stick it in a public cloud..."

A solution to the above constraints for both, is a technology we call Service Virtualization - this is now a very mature solution space and something we've been discussing for more than 3 years  here, since we introduced LISA Virtualize product features. Virtual Services provide a companion to VMs, and simulate the behavior and performance of those Out-of-Scope items, so when and if you do deploy both in a Cloud, you truly get an elastic, constraintless environment for software development and testing purposes.

iTKO's 3 IT Predictions for 2011, and How 2010 Played Out...

Another year of accelerating change in IT has swept by, and as we clear out our inventory of leftover holiday cookies and cakes, we also like to collect some of our predictions for the upcoming year, while seeing how this past year went in retrospective. (Please note, these are in no particular order, and they are just the opinions of some iTKO execs we collected by passing the hat for thoughts - use at your own risk!)

Predictions for 2011: Are we entering a "Nirvana Age" for software development? We are starting to think so - more on this concept later, but here are 3 trends we predict for the coming year:

1. An Agile Resurgence is happening. Companies are increasing their commitment to Agile methodologies - but the big difference this time is that we're not talking about a few development teams adopting XP/Scrum etc. - Agile Resurgence will be the mandate Agility across the overall IT and software development lifecycle, so you will see it cover an entire delivery organization. Companies will measure their overall success in meeting customer requirements in terms of improved Agility - speed and efficiency across design, development, testing, support and services as a whole.
2. Enterprise Cloud will take hold. By the end of the year, we expect serious enterprise Cloud adoption to accelerate rapidly beyond merely using it to improve infrastructure and hosting capacity. A whole new class of business process applications that leverage cloud will appear, along with new Platform as a Service (PaaS) capabilities that make cloud an environment for collaboration and software development itself. We've been talking about DevTest Cloud for most of 2010, and we are seeing major enterprises take it as a part of establishing their enterprise-wide cloud initiatives.
3. Shakeout in Testing space is still not done: We have seen that many companies have delayed or deferred replacing their existing UI-based testing and load testing tools and focused mostly on development tools with testing as an afterthought. That is going to change bigtime, as companies can no longer afford to have exponentially increased test labor costs, and risk of failure accelerating faster than software can be delivered. Customers will demand both deeper testing coverage across complex multi-tier apps, as well as new Cloud- and service-based delivery models for testing that can offer more flexibility and labor efficiency.

Retrospectives for 2010: Did we call it? Well, this time last year we were busy releasing LISA 5 Suite, so we didn't stop to make 2010 predictions we can grade ourselves against. But here's some observations we couldn't help but make.

1. The Cloud took off like we thought it would. We're not a market research firm, but we couldn't help but notice an explosion in Cloud strategy and implementation. In our own customer base, the number of inquiries and interest of "how do I move to Cloud?" and "Should I use a Private Cloud?" increased by 3-5X over just the last 6 months. We sponsored a couple Cloud-related events and webinars, and found a huge increase in both attendance and sophisication of this audience.
   
   
2. Service Virtualization became a line item on the IT budgets of market leaders. We saw an increasing number of project requirements and RFPs here and in Europe specifically calling for more than just "stubbing" of responders for testing and development. 2010 was a breakthrough year for the concept of virtual services: the simulation of behavior, performance and data characteristics of systems that are constrained or unavailable throughout the development lifecycle, and customers want to know how to resolve these big constraints to delivering software. A big part of this awareness was assisted by our partnerships with HP and IBM, allowing customers of their testing and ALM product suites to simulate constraints in their environments with capabilities from iTKO's LISA Virtualize.
3. A weak economy did NOT take any pressure off IT to deliver more functionality, faster, maybe it was even worse due to it... While companies likely spend 2-3x the time scrutinizing their budgetary plans for IT, and hiring was usually frozen, companies could not afford to slow down one bit. Increased competition for customer dollars, rapid consolidation and acquisition in many markets, and more regulation all created big IT projects, often without a commensurate increase in budgets or resources.
4. VMWare successfully defended its near monopoly on large enterprise IT virtualization; it was predicted by many they would fail (not us!) thanks to competitive or open-source pressures.
    
   
5. SOA Governance fell flat.  This was definitely last year's fruitcake, regifted. It still seems like such a great idea - the runtime part of governance enables companies to do some cool stuff which kept some of the players afloat, but we didn't hear many inquiries about SOA Gov all year long.

 

From SOA to SO-What (or, SOA Is Dead Is Dead)

As you can tell we've been all Cloud and Virtualization, and not much SOA on this blog lately. And there's a good reason. Just commented in what may be the last of the eBizQ expert forums on why SOA (which stands for Service-Oriented Architecture for those who forgot about it) may or may not have failed in the past as an architecural approach to delivering business applications.

The Forums Editor Peter Schooff asks "What reason do you give for the initial failure of SOA?" citing a recent update from Burton about SOA "rising from the ashes." Well that is one way to look at it if you also stated that "SOA is Dead" a couple years ago. (Of course, we understand both statements were not made in all seriousness and they were circulated through the pundits with healthy irony...)

Here's what I said about it:

We've gone from SOA, to SO-What. I agree that it never really failed, and it is quietly contributing real value today. It might have gotten a bad reputation just because a few early adopter companies bought a particular vendor's SOA "bill of goods" or ran down a JBOWS (Just a bunch of web services) route, and found everything didn't fit together as expected, projects ran over, etc...

That rep of a few people getting burned (perhaps by not fully embracing a SOA approach) was the only real issue with SOA. 3-4 years ago we were finding some IT leaders stating "I'm NOT doing SOA - DON'T mention it again in this room, we are doing continuous integration," but despite avoiding the term they were indeed moving down a SOA path. Now, major enterprise IT does not need to make this distinction.

I thought Michael Poulin and Tarak Modi made good brief points too. Hey, if you think SOA was a failure, chances are you weren't doing it right - you weren't actually allowing the software to be delivered in a services-oriented way, you were accepting some grand vendor lock in instead of allowing for some platform independence, you were trying to dictate an all-or-nothing strategy with partners trying to connect to you - the list of ways you can fail at SOA were quite long.

And that's before we even address the issues of maintaining continuous quality and validation in the face of constant change, something that SOA makes much more complex. Or talking about how some of those service dependencies can become huge constraints to the software lifecycle that we need to virtualize in novel new ways.

We enjoy having a soundingboard for this stuff, as we watch each new IT strategy or technology get pulled by the pundits from hope, to disillusionment, and if it works, back to hope again. There is real value being proven every day by companies who let sound architecture and focused discipline drive their SOA initiatives, even if we don't talk about it that much anymore.

(Picture: I took this in 2006 at Prague Castle, a Kafka-esque skull riding on a man figure)

 

John - Talking Cloud Development & Test on TMCnet

Here's a clip from my trip last week to Sys-Con's Cloud Expo 2010 West - I had the chance to catch up with TMCnet Executive Editor Erin Harrison for this interview about DevTest Cloud, and the trends we are seeing at this event.

Enjoy:

 

Here's to a continued - and inevitable - maturing of this sector of the Cloud market. Much of the remaining cost and constraint of IT is in properly equipping very expansive and widely distributed dev and test teams with ready environments that support needed scenarios.

 

Notes from Cloud Expo 2010 - This will fly...

Just got back from Cloud Expo 2010 in Santa Clara, CA, and this year's event was very much a coming of age for Cloud. Attendance and expertise levels were high. Two years ago when I audited a Cloud conference, the subject matter and vendors present seemed to be catering to an MIS or webmaster-level audience - largely a sideshow to the SOA/Virtualization audience concerned with moving servers off the racks of their data centers to save some incremental network and hardware costs.

iTKO's "Chief Geek" John Michelsen talked Tuesday morning on "The DevTest Cloud: An Essential Platform for Applications" and it was standing room only, with plenty of good audience interaction - a lot of developers and testers in the audience from a show of hands.

If you want to see the content from this session, and a little more specific to LISA DCM, I suggest you attend the iTKO webinar Wednesday, Nov. 10th, 1pm CDT: What Makes a DevTest Cloud Fly? (And what is the value?)

We are seeing a maturing of the space. Hardly anyone here is trying to "define Cloud" anymore. Not as many irrational fears about "is the Cloud secure? Can it perform?" Even in infrastructure terms, the Cloud offerings here are more specialized, for instance, leveraging private Cloud storage for content management uses, or offering industry-specific services atop cloud infrastructure. From my perspective talking to attendees on the Expo floor, people are looking for ways to develop business functionality to take advantage of Cloud, and not the other way around. Also it was good to see more reporters and analysts stopping by.

John also participated in the CTO Round Table that evening, and he will cover that discussion in an upcoming post.

Seeing the BPM Process is Believing

I recently commented on this eBizQ forum on "What is the single greatest benefit of BPM?" here:

"The biggest benefit to me is the simplistic view: that BPM represents the process that actually gets executed, in a visual model that the business can actually understand.

The "dirty little secret" of integration heretofore was that while we made neat little whiteboard sketches or UML diagrams, the business process itself would be implemented and coded behind the scenes by technicians in a way that probably didn't resemble the process at all.

The inability of business stakeholders to be involved in real process design - because it used to be an exclusively technical endeavor - is exactly what created disconnects between requirements and delivery. So in my opinion, the visual model of an executable business process itself is the most important benefit of BPM."

This got me thinking further about how both the business expert and the software dev/test organization should get a literal big-picture view of any analogous technical activity, rather than work from two different blueprints or interfaces.

Why was a computer with a mouse better, when at the time an expert user could certainly work much faster using keyboard shortcuts? Why is a multi-touch phone better, when discrete keys clearly require less dexterity and produce fewer input mistakes? Sometimes the long-term capabilities of a more interactive UI supercede any immediate productivity hit.

Thinking of testing or development activities in a BPM view will involve some tradeoffs in efficiency over specialized interfaces, but may net better results as delivery will be much closer to requirements. In the long term, new methods for collaboration can arise from the presence of a shared visual interface as well.

We've certainly taken this to heart in thinking through shared aspects of test and dev interactivity and making our LISA Pathfinder product have a more collaborative, process-oriented UI. While both testers and coders need to drill down into specialized tools, there is something to be said for a common view that all teams can use to speed up interaction as service-based software and cloud applications become more complex and interconnected.

2nd Paper in Infosys & iTKO Series: Service Virtualization for Modern Applications

I just got back from a select QA Leadership workshop in Palo Alto hosted by our partners at Infosys, where I presented perhaps one of the world's shortest talks for one of the world's most complicated tasks: Optimizing the entire application lifecycle in 30 minutes or less!

Obviously I could only scratch the surface, but there's something to be said for the focus that comes out of brevity, and it led to great questions and discussion with joint customers and Infosys leaders like Aparna Sharma (whom I've presented webinars with before) - people who have overcome some of the biggest challenges in software testing.

Certainly a major focus of my talk and an ensuing discussion was the practice of Service Virtualization - the simulation of systems and services that are either constrained or unavailable for use in testing and development throughout the software lifecycle. Service Virtualization is key to eliminating dependencies and allowing multiple teams to work around their constraints and conduct their activities in parallel.

It was a great day -- and on that topic of virtualization, we also have the release of the final version of this brief paper we co-authored with Infosys for the second of a series of articles about best practices for IT, Dev and QA initiatives for modern, service-based applications.

You can download the new paper for free here: Whitepaper with Infosys and iTKO: Service Virtualization for Modern Applications

Here's the abstract: Disruptive innovations such as the internet have repeatedly forced businesses to change quickly or be relegated to history. Every such disruption asks a simple question of service functions in an organization - can they evolve quickly enough? Modern approaches such as service oriented architectures (SOA), along with other similar distributed application architectures, carry great promise and can provide an effective model for lean, agile and componentized IT that can adapt quickly. SOA is one of the most widely adopted enterprise architectural methodologies, but its implementation is anything but simple. Since classical testing methodologies and tools do not fit modern applications well, we have evolved newer more rigorous approaches and products to address increased software complexity and change. Service Virtualization is one of the latest ideas that can provide a faster time to market for software, with higher quality and lesser risk.

This new whitepaper, jointly written by Infosys' Gaurish Hattangadi and iTKO's Rajeev Gupta, focuses on the use of virtualization of constraints as it applies to the problems of developing and testing modern SOA applications. Let us know what you think as you explore virtualizing services in your own software lifecycle. http://www.itko.com/resources/servicevirtualizationmodernapps.jsp.

2010 - The year we met DevTest Clouds...

Today we released a new version of the whitepaper: "The DevTest Cloud: An Essential Platform for Application Development" -- and aside from looking really good (kudos to the designers), it contains a whole lot of new thinking about how we should best bring enterprise software development into the Cloud age.

If your enterprise has a Cloud initiative brewing, there is not a higher value lever to pull right now than optimizing development and testing activities in the Cloud. You should find out what DevTest Cloud platforms can do to bring the elasticity of Cloud to the software lifecycle: on-demand dev & test environments, with ready capacity at exponentially lower costs.

This is just a start, as we believe 2010 will be the year of the emergence of DevTest Cloud platforms. Leading vendors such as IBM, VMware, Citrix, Amazon and others are delivering elastic, on-demand infrastructures that are more than just places to host applications. We'll continue demonstrating how enterprises will be able to build their own DevTest Cloud platform that leverages these solutions.

Look for more blogs from us on this vital practice, as well as new videos, new press and articles, customer stories, and an appearance from John at the upcoming Cloud Computing Expo 2010.

You can get a copy of this paper, and find all the latest DevTest Cloud resources from iTKO here: http://itko.com/cloud.

New Analyst Product Report: LISA 5.0 Technical Audit - Ovum

Useful news out today about some available research -- iTKO recently participated in an independent product review of our latest LISA version 5 Suite with UK-based analyst firm Ovum (formerly Butler Group) principal analyst Rob Hailstone, The research was not sponsored by iTKO, however we have purchased rights to the report so you can download it free for a limited time at http://itko.com/ovum.

In the report, Hailstone provides this overall assessment of LISA's market:

"LISA Suite provides a comprehensive set of testing capabilities that will allow customers to improve the effectiveness of their QA function for SOA or other component-based development environments. The functional modules can be enabled individually to satisfy point-solution testing requirements, but make most sense when deployed together as an integrated suite.

LISA Suite is targeted at large organizations serious about the use of SOA, but limited deployments at the department level provide a common entry level. Organizations deploying applications into a public or private cloud will benefit from the end-to-end visibility and root-cause analysis provided. LISA Suite is less appropriate for smaller organizations or those using SOA technology for simple integration."

For many years, LISA was often considered by market analysts only as a "SOA Testing" tool, which is a useful definition, but not necessarily one that represents the scope of many of today's development and performance challenges for teams dealing with complex, distributed architectures. This document points out it's also not just about SOA, as many enterprises are more concerned with BPM, Virtualization and Cloud, which are not tied to a specific architectural style or approach.

Why did we license the report? Well, after a comprehensive functionality comparison and walkthrough process, we think the report Rob produced provides a pithy overview of LISA 5 software. You can download your own copy and decide for yourself...

Mind the Development Gap in Virtualization and Cloud Approaches

We recently had the chance to catch up with Eric Knipp, who happened to be in the area (yes, we have astute local Gartner analysts in Dallas, and no cowboy hats on in the room). He made an interesting point: that the way we describe Service Virtualization was really oriented toward the "Design Time" aspect of a development environment.

That is definitely a "development gap" that needs to be minded. When I read and hear articles about new development processes, it seems like there is a wide gap between the developer's code itself (addressed by Agile, scrum, source control, some ALM) that extends all the way through integration and testing, and continues until Deployment (where the software assets exist in Runtime).

As an industry, we look at Virtualization and Cloud as deployment frontiers. That's cool, but it represents a small portion of what could be done with the two technologies working together to eliminate that development gap, before we step off into it.  

We are increasingly finding that as development becomes more distributed across small teams and separate domains, the number of pre-production development servers and test beds balloons. Whether that deployed app is on a regular web server, or a Cloud-hosted VM, it still seems like we are leaving a bigger proliferation there. It might not seem that big at first, but few real enterprises haven't already started leveraging these distributed computing resources now.

If Cloud Computing and Virtualization are all about replicating or replacing production servers, then why would a top banking customer replicate more than 1100 services and systems with Virtual Services instead of letting them keep making copies of Virtual Machines (VMs) on their network, and later, in their private cloud solution?

Short of longing for the old days of monolithic, centrally controlled systems with 1 production environment and a couple test and staging environments... business won't let IT go back to slow releases - our agility is also creating a new burden in pre-production. Of course, this applies for larger enterprises - there are many instance of new companies who are building functionality "for the Cloud" that is limited in scope and integration requirements, starts fresh, and thus, does not have as much of a burden here.

So here's my question again: If you had to count up the instances of pre-production vs. production "environments" and servers, would the pre-prod outnumber the production by 10-1? 100-1? Be honest and think about all those dev and test servers you don't know enough about to shut down... and think about all the partners that must have their own as well.

And, mind the gap before you step off that platform!

Those Guys need LISA!

I frequently joke with my team about finding our next customers in major business publications. Every time I read a headline like the following, I immediately think: “Those guys need LISA!"

Yahoo Tech News: "Apple, AT&T sites buckle as iPhone 4 pre-orders begin"

The fact is, all the agile developers, functional testers and load testing servers they can fire up are not going to help them here.  This is NOT a load testing issue, it’s a virtualization of services issue!

If we met the guys working on all this critical stuff, you would certainly not hear that companies like Apple or AT&T don’t have enough money to buy servers.  In fact, many major companies actually tell me that they have more servers registered in their asset tracking system than they have employees!  These guys will be no different...

So how could these teams have used LISA to prevent this conundrum?

1. Service Virtualize each others' pieces of functionality that contribute to the site experience, so that each team can develop and functionally test their bits, without having to wait for the system to come together later, during integration.
2. Service Virtualize the dependent systems (inventory, customer ordering, account and number provisioning, etc.) so that performance tests at high levels can be performed ahead of time, and each service can size and qualify their environment for the load expected when it goes live.

Some day in the future, they will even want to have intelligent ways to fail over in production during peak load hours, but let's not put the cart before the horse. One of my banking customers puts it eloquently when he tells his component teams to reach a certain performance level on their own: "If your app isn't performing fast enough in your own virtual environment, why should you bother giving it to me to test in my environment?"

OK I’ll stop rambling.  Now somebody let me know when the coast is clear, so I can order my next phone ...  ;)

Pooling Major Software Costs in Clouds Across Large Enterprises: Will it work?

Found an interesting article from Bob Evans in a recent InformationWeek - major financial firms are working up a cloud consortium to try and negotiate down some of their IT costs by pooling resources. Of course, we are familiar with this concept in government requisitioning, group payer negotiations for healthcare... but how will it work for IT?

"Global CIO: Global Banks Form Consortium To Counter HP, IBM, & Oracle"

True, if anyone has the clout to push the big software vendors to offer them economies of scale, it's banks like Bank of America and Deutsche Bank, and if this were going to happen anywhere, it would be in the Cloud. However, while there may be commonalities between large banks, in terms of the types of workflows and compliance issues they need to support, the enterprise software packages they are likely to be running are very seldom standard one size fits all applications.

I think like many seemingly disruptive changes, this strategy will have more utility for a new class of "shared utility" types of Cloud-based software that represents a small percentage of functionality that can be shared across global banks. This will be like a "toe in the water" to gauge the temperature for a bigger plunge later.

For more strategic operations, the inherent uniqueness of financial systems in different countries, and even global competition, will still require custom purchasing and large service contracts from the software giants, and the SIs that support them. I'll certainly be watching this story in the coming few months to see how it plays out.

eBizQ Podcast: Should you take a "Big Bang" Cloud Approach?

I recently had the chance to catch up with eBizQ's Forum Editor, Peter Schooff -- and record a podcast about iTKO's perspective on Cloud Application development. To us, Cloud Apps are the ultimate form of modern, distributed enterprise applications - they leverage everything we've done from service-orientation, virtualization, business process management and SaaS to date.

I think Peter asks some very salient questions - take a listen, read the full transcript of the interview (cool feature of their site), or download the mp3 Podcast here:

ebizQ Podcast: Should You Take a 'Big Bang' Approach to the Cloud? iTKO Explains

Cloud Apps are the apps themselves - not the hardware (or virtual hardware) they are hosted on - which are unique from the well known practice of Cloud Computing, which addresses and commoditizes computing capacity in deployment. We are especially excited about how developers and testers will be able to use Cloud App platforms to revolutionize their efficiency and reduce costs throughout the software lifecycle, so I hope that comes across in my interview.

More About Dev & Test Cloud Environments from the Whiteboard

We're continuing our series of iTKO Chalk Talks about moving Development and Testing environments to the Cloud. In our previous post, John discussed the Challenges of Dev & Test cloud environments. Since there are extremely significant cost savings -- on the order of 95%+ cheaper -- to be gained by provisioning this Pre-Production capacity in Public and Private Clouds (or, "Hybrid Cloud" Environments) it behooves any enterprise IT shop that uses and integrates software to pool resources and use Cloud Computing approaches in this way.

In this Dev & Test Cloud Chalk Talk installment, we'll talk about why and how companies are moving Pre-production aspects of the software lifecycle to cloud environments:

What about the hidden costs of Dev & Test Clouds? Cloud vendors and providers can point out the significant cost advantages of using Private and Public clouds for a given computing resource above, but they leave out continuing costs like MIPS charges, transaction fees, data maintenance, and many other cost aspects that cannot be reduced by moving Pre-production to Cloud environments. We will cover this aspect next.

Cloudy Chalk Talk 2: Dev & Test Clouds in Pre-Production

To continue our series on Cloud Application Chalk Talks, our "chief geek" John will cover Development and Test Clouds. After the move toward commoditizing some hardware and capacity in Public clouds, we find that the next logical step is using Clouds to level off Pre-Production costs throughout the software lifecycle.

Capacity in Production is fairly easy to predict, but it is hard to predict in Pre-production. You can plan to make capacity for spikes and volume and provision your infrastructure if you are an eCommerce vendor getting ready for the next Christmas rush. But what if you could eliminate 80% of the unpredictability of the Pre-Production environment that hundreds of skilled developers and testers use, and therefore not need to buy all that capacity up front? Private/Public Clouds are a great way for our partners to pool together computing resources into Private Clouds, use Public Clouds for overages, and try to regulate their infrastructure costs (compare oh, $.06-.12 a CPU hour of private/public capacity vs. $2.00 for each team using dedicated systems - and you see the economics).

So what happens next? We'll cover how companies are dealing with those "spikes" in capacity and cost, as well as the hidden costs that you don't usually hear about in the standard arguments for employing Cloud Computing and Virtualization in pre-production.

Chalk Talks on Cloud Applications - Part 1: What is a Cloud App?

Over the last 3 years, I have been asked quite frequently how our vision of service virtualization and validation applies to Cloud Computing. Indeed, we have waited a while to inject our own views in this arena (though we posted a solid whitepaper on Cloud and several recent Cloud commentaries on this blog). So far, we have often found that the Cloud discussion seems to be all about deployment, and not enough about the software lifecycle. Well, it is time to address that side of the equation.

So here's the first installment of a new Chalk Talk series I've recorded on Cloud Applications: "What is a Cloud App?"

Think about the difference between Cloud Applications - the software you run on a highly distributed, elastic architecture, versus the deployment model of Cloud Computing itself. They go hand-in-hand. Everything we've done in the software lifecycle up to this point - from legacy enablement and reuse, to service-orientation, to continuous integration, validation, and collaboration across multiple teams, has helped make Cloud Applications possible today.

We hope you find this series useful and invite your comments and suggestions for topics you would like to hear.

What is a Cloud App?

We at iTKO are not claiming to coin the term “Cloud App” -- in fact a number of vendors out there have already been referring to cloud apps – for instance, vApps from VMware and others. But I wanted to frame the discussion around what cloud applications really are, and the challenges and benefits that will shape what cloud apps could be.

I do like that we can participate in this dialogue. We’ve been a part of many of the intermediate steps of making customers ready to have a cloud application over the last decade. The more distributed, service-based and loosely coupled applications have become, and the more they are connected through commoditized middleware and messaging, the more viable they therefore become as cloud applications.

Cloud apps must have these attributes, because without them, you would have a hard time delivering the value of Cloud to the business -- being able to interconnect various components from different vendors in different ways, and being able to do so in an on-demand basis.

For instance if I had 4 or 5 different service vendors underneath my application, I could elect to use Vendor A for a time, and then remotely switch to Vendor B for a similar service, based on price, availability, contractual reasons, what have you. This is an important concept, and not a new one. When we didn’t have Cloud applications and the ability to make that switch easily and remotely, the cost of switching was too high and thus, rarely happened.

The IT community tried to address this with SOA, to offer a “plug and play” capability for services, but the infrastructure didn’t allow for it. We did get service-oriented benefits around more modular programming, better reuse, and most importantly the distribution of business logic between teams, and a better way to get teams to iterate on development of components more quickly, without having to create a new application.

However, the ability to put the entire infrastructure in a service mindset really couldn’t have happened without a cloud infrastructure. So, Cloud Apps are just the next step in a journey you are already on. If you are building apps that are web-based, using web services, N-tier, with connected legacy systems, and service-based access to third-party systems or SaaS providers, you are already on your way to having a Cloud App.

The only difference is, you’ll now think of deployment as a service-based activity as well, and you will have a greater ability to switch from one provider to another, when it comes to acquiring and accessing those services.

I hope that explains what we mean by Cloud Applications, and you can expect to see more discussions on this topic here. Read more new content from iTKO on Cloud Apps:

• iTKO.com: iTKO LISA Solutions for Cloud Application Validation
• iTKO.com: iTKO LISA Solutions for Development & Test Clouds
• Blog: Chalk Talks on Cloud Applications - Part 1: What is a Cloud App?
• Blog: Cloudy Chalk Talk 2: Dev & Test Clouds in Pre-Production
• Blog: More About Dev & Test Cloud Environments from the Whiteboard
• Blog: Hidden Costs Off-the-Cloud: iTKO ChalkTalk

Why Stubs Don’t Work: Part 1 – Developers & Consumers

I’m frequently asked what the difference is between what we call a Virtual Service, and what developers have done for years, which is “stubbing” and/or “mocking out” a particular service. There are a number of ways I can address this, depending on who I’m talking to.

 First, for the Developer, I’m going down the path that Virtualization is now taking on the task of industrializing or productizing the practice of stubbing itself. Some 20 years ago, when I first started development, I wrote my own database, wrote my own development tools, my own middleware to get components to talk to each other – in fact, I used to code all kinds of things that today, I would never even dream of building anymore.

The requirements for proper databases and middleware became advanced, so that it was no longer feasible for developers to roll their own. Instead, those requirements became a category of software, and vendors started delivering solutions for that space, thus we saw the rise of a database market, and a middleware market, and so on. We see that the requirements for simulating components have become too steep to roll your own. So in that light, we see Service Virtualization as essentially the “space” that is coming out of the requirements of stubbing and mocking.

The second answer I would give, more often to the Consumer of a stub or mock is just how brittle these stubs or mocks have to be in order to be efficient. The whole reason why a team would construct their own stub is summarized as: “I don’t need a product for that, because it is something I can make quickly and easily by myself.”

Well if the goal is to do something quick and easy, the best you will be able to do with this artifact, is do a “round trip” to the stub, to prove connectivity of a particular application to that stub. In reality, there is so much more detail needed by the consumer of a service, that calling it “something you can mock up quickly” means you don’t understand the true nature of the problem.

Let me explain: If every customer response for a stub has the exact same profile, and the exact same address, account balance, etc. all with hard coded values and dates, then that may help the consumer of that stub with ONE data scenario. But, what about customers with high account balances? What about old invoices? What about transactions that occurred yesterday – and will they work the same tomorrow when that date becomes 2 days ago?

The problem with a stub that has static data, is that it won’t support the real variety of scenarios that are needed for a real world application. And you can’t just randomize the data – that would be even worse!  Consumers need predictability and control over the data they’re seeing. And they need to integrate that data not only from the simulated application, but from all the other services that might be associated with that particular application they are building.

A simple example: Say you have customer lookup in one service, while you have account management in another service. So if you have Account IDs and Product Codes and Balances that are not aligned properly between the two systems, you will have an application that fails, either at the consumer level, or somewhere in the downstream systems.

When you think about data required to support development in this scenario, stubbing a system simply doesn’t hold up. Considering all the things you would have to do to make that stub more real, you would need a pretty sophisticated framework to build those stubs. So you might start out building stubs, thinking “it’ll only take a few hours,” but you end up taxing your whole project plan with the hidden cost of building the first generation of a stub, and then modifying it over and over again. And by the way, they don’t even work very effectively.

This is part one of John's discussion on "Why Stubs Don't Work." Read part 2 of the blog, where he will discuss Performance Lab uses of stubs, and the "Tipping Point" for moving from stubs to Virtual Services.

Question: Are Pre-Prod Environments Balooning while Production Infrastructure Shrinks?

I recently posed a question to eBizQ and got some good answers, both reasonable and unexpected perspectives. See my eBizQ SOA forum question here: Are Servers Bloating in Pre-Production, but Shrinking in Production due to SOA?

K. Scott Morrison and Joe McKendrick discussed how Virtualization is a tester's best friend in reducing this phenomenon - and yes we have seen virtualization of hardware reducing some infrastructure needs. What about the ones you can't virtualize? I vote the best answer (received by 5:00PM CST) goes to Avi Rosenthal, whose comments highlight why this phenomenon is happening:

"In distributed Service Based, Virtualized and Cloud based environments the number of other environment servers is skyrocketing vs. the number of Production servers. This is true as well for virtualized server instances. The reason for that phenomenon is that Production environments are more controlled because no one can allow uncontrolled Production. The risks of non controlled Development or Test environments are less obvious. As the number of instances grows the number of servers in the non-controlled environments grows more rapidly than the number of servers included in the controlled environment. However, logically the number could be reduced at least to a ratio of 1:3 or 1:4 and not 1:20 if the non-Production environments will be managed and virtualized similar to the Production environment."

Really good point Avi.

However HP's Kelly Emo then came up with the gist of an issue I wanted to address further: the huge rift between measuring costs and effort for Production vs. Pre-production environments. It seems these activities are usually owned by two completely different departments and cost centers within the enterprise as she notes:

"In a majority of customers I speak with, their organization, culture and funding models are no where near having this happen... different silos are managing hardware and software resources and different SLAs are associated with them. In fact, in some cases, SOX compliance keeps organizations from being able to mix and mingle their networks and data between pre-production and operations. That said, I see one place where this can happen.. in the cloud..."

That's a next-generation view. Indeed the cloud is a big piece of the puzzle. However, many of the companies who get excited about the value of deployment savings using VMs and Cloud-based infrastructure to replace servers, simply do not have a reason to care about the number of groups maintaining and consuming dev, test and integration environments.

 The same goes vice-versa for pre-production - when you break up development into smaller, agile dev & test teams, those teams each provision and set up their own hardware and software environments in a largely unregulated fashion - and often find out that there are many dependencies they can't copy or replicate via conventional means. This leaves a lot of cost and effort unaccounted for in the wash of the software lifecycle, or as a cost of per-employee infrastructure or support.

You will see a lot more on this topic from us - while we aren't in the business of virtualizing servers and hardware, we are particularly curious about why companies aren't recognizing the value they get by optimizing Production vs. optimizing their infrastructure at every step along pre-production. Is a ballooning number of servers a side-effect of distributed, agile approaches, or is the symptom we are reporting simply a matter of perception? We'll keep you posted as we do more research in this area.

We're thankful for IT change and complexity in the 00's.

The challenges of change and complexity are what we are thankful for this season here at iTKO - because without them, we frankly wouldn't have much to talk about. Likely, if you are reading this blog, I wager that these two factors might be keeping you up at night. But the source of this change and complexity might not be what you expect.

Simplicity and agility in development, is quite literally becoming the cause of change and complexity in IT, especially in ensuring that a reliably functioning software environment is delivered on time. We have been covering this theme for years - but for some reason it has really turned on the light bulb for IT listeners this year, when pressures to deliver more functionality with less resources have been at their greatest. 

Enterprises tried to reduce release cycles and costs by focusing on one part of the lifecycle: development. They've brought in new development tools, techniques, methodologies, to make software development more agile, done in smaller, reusable units, more distributed, more iterative, with ever-shortening release cycles.

Companies have spent billions, in fact, on buying platforms to enable SOA, BPM, ESB, and legacy modernization approaches. It created a new problem however... development is getting faster, but the time and cost savings is more than eaten up by QA and maintenance -- and the overall release timeline is getting longer. This imbalance has continued to grow, and organizations can not afford to keep going down this path.

To resolve it, you have to focus on test AND delivery, to wit: How do I optimize the larger lifecycle to keep up with Complexity and Change in my software? We can't afford the support nightmare of software failures in production, but if we throw resources and infrastructure at testing to keep up with all the changes, our costs and timelines, will balloon.

This is why companies are turning to Service Virtualization as a way to alleviate the huge cost of having a valid system environment available for testing, validation and performance purposes. We are thankful to be here, working to solve this problem, at a time of so much change and complexity.

Hear what some other musing friends feel thankful for on the eBizQ forum here. I personally have also been thankful to see less top 10 predictions, and more things to be thankful for, as we finally head out of what has been a tough decade for many businesses. Here's to a very successful 10's!

Thoughts from Gartner AADI 2009 Summit - SOA's still on.

I recently returned from the Gartner AADI conference in Las Vegas, and had the chance to enjoy dozens of good conversations with IT execs who were there primarily to explore strategies for the coming year. Some thoughts from the conference and sessions I was able to catch:

• Budget optimism... Overall it seemed that there was a guarded optimism for re-invigorated architecture and integration projects and budgets in 2010, many people said they were part of a new group or new reporting structure specifically tasked with defining a strategy.
• SOA Won't Die - in fact any lead architect or CIO I caught up with says they are quietly going about service-orientation and getting real value from it, even if it's not sexy anymore. After the age of boundless service proliferation earlier in the decade, it seems that much of the talk is now about authority - who owns and manages these services, and how are they compensated for doing so. Recently a lot of the "SOA elite" posted some good discussion on this topic (check out this eBizQ SOA forum here).
• Heterogeneity is king. The optimism for integration is tempered by a big dose of skepticism about "one stop shop" vendor claims - even more so than you normally hear from the analysts. Companies are dealing with the realities of multiple acquired systems from different vendors, and looking for ways to modernize their applications.
• People understand the distinction of service virtualization. Another pleasant surprise was the understanding shown for hardware/network virtualization vs. the practice of Service or behavioral virtualization of constrained assets that simply can't be captured or dropped on a VM. I believe the analyst community has talked to application layer customers about how these assets can be brought to bear earlier into the lifecycle, even if this practice doesn't exactly fit under defined research fields like software quality, app dev, virtualization and IT ops.
• Cloud Computing still uncommon in large-scale enterprises - There was a great deal of content about Cloud-based approaches at the summit, and it is certainly important to get some forward insight. In practical terms however I did not see these $500M+ company IT execs mention Cloud in their basket of must-do items for 2010, other than a notable exception of specifically wanting trading partners to use the Cloud more to ease cost and access issues. (This reminds me of public transportation, "it would be great if everyone else would do it to free up the road...") Certainly a lot of curiosity and predictions around this. I am sure at a less enterprise software heavy conference you would see more cloud practitioners and this will continue to grow.

In any case it was a productive time at Caesar's, and we will discuss some topics we touched on this week at a later time.

What's the next big obsession for Enterprise IT Geeks?

As an occasional pundit and commentator on IT trends, I've had a lot of chances to watch the "next big obsession" come and go for enterprise IT shops. The one commonality we've seen, is that if the obsession successfully takes hold in business, then it will become so ubiquitous that it becomes kinda boring to the IT guys on the cutting edge.

So goes the case for SOA... We've tracked the "Is SOA Dead?" thread over the last couple years and it was fun, because it points out just how common SOA has become part of the enterprise IT landscape, delivering value so quietly that we almost take it for granted now. Just like "eCommerce" used to be a big deal, then it just became the nature of doing business over the Internet.

So one of these questions I posed to Peter Schooff of eBizQ was this very thing - and he posted it to their Forum - What are Enterprise IT Geeks Obsessed with Today? I was looking for stories, and not the next dot on an analyst wave or cycle diagram. He posted the question and some fun answers there.

Clearly you can't escape Cloud Computing right now as the grand champion of obsession (Make mine Private or Public?). Mobile platforms and smartphones - riding high again. RIAs, particularly REST-type web apps are garnering a lot of geek attention. But what never changes is that these innovative obsessions aren't just about money - they are inevitably tied to doing things better, more efficiently than before. The cost savings and revenue upside are the results of success, but not necessarily the motivator.

I particularly liked Sandeep Gupta's comment in the forum that "IT geeks do not change...they are always passionate about the technology space/environment that they work in.....obsession brings aspiration and passion...leading to effeciency and growth" and that to me describes the accelerating nature of innovation that is going on here.

Also great - Brenda Michelson's comment (via Twitter) about "Hopefully getting things done!" Well put. And Phil Ayers laments that IT folks he runs into "just don't have time to be geeky enough and keep up with, let alone apply, the vogue technologies and methodologies." Yeah, there is probably a little too much talk about the next big thing and not enough application going on. So until next time...

What are Modern Applications Anyway, and Why Do We Care?

You may have noticed lately that a new terminology has surfaced in the way we talk about testing, validating and virtualizing software architectures of a certain type: Modern Applications. Most recently it was used in our release on the HP / iTKO agreement, and you can also see it now on our website. So why is this distinction important?

Understandably, the first thing that may spring to mind with "Modern Apps" is an architecture akin to a Mondrian compartmentalized painting, or maybe some kind of new wave synth music. However, the use of "Modern" makes a lot of sense in encompassing several commonalities inherent in today's enterprise software, without requiring adherence to a specific architectural approach such as SOA.

In short, "Modern Applications" are today's class of distributed, multi-tier, object-based, flexibly assembled software architectures, used to support complex business workflows. Modern applications may include all of the above technologies or architectures (SOA, REST, BPM, Virtualization, MDM, Cloud Computing, SaaS, what have you). While that sounds very general, that defines the kind of software we see getting put in place in most leading enterprises today. We are seeing analyst traction in this sense too, for the natural result of "legacy modernization" should, by rights, lead to "modern applications."

A couple days ago we had a live webinar with Perficient's chief technologist and SOA blogger Eric Roch - and we started with SOA but quickly migrated to other integration platforms, the use of Service Virtualization and Data Management which are very different from SOA, and there are many other aspects of Modern Apps that are not relevant to SOA. (You can watch a WebEx archive of the 9/26 webinar with iTKO and Perficient here.) Eric recently wrote about this in his blog: "Do We Need more than SOA for a Modern Application Architecture?" as well:

"While SOA is certainly a big part of the solution, a total modern architecture also includes leveraging business process management (BPM), data management and Web 2.0 technologies. SOA alone does not implicitly improve governance, IT strategy or business alignment, and top-down, strategic SOA has been largely disappointing."

We definitely agree. What is relevant is that this category of applications are pulled into an environment where they are built, leveraged and shared among multiple development teams and business partners - Modern Applications are not "stovepipe" architectures designed and built by a single team with a rigid set of protocols. Modern Apps are the kinds of flexibly assembled, agile software structures that modern businesses demand, in order to be competitive.

SOA, even if the term fades away, would be considered modern as a model for designing and building applications. However SOA alone leaves out a lot of other strategic and technology advances we are seeing in today's enterprise. We hope this explains our use of this admittedly broad term adequately. Your thoughts on this topic welcome!

Hitting the right QA Notes with SD Times' Feinman

Recently I had the chance to catch up SD Times' Jeff Feinman for an interview about the latest trends we're seeing in software QA. In the resulting article, "Hitting the Right Notes in QA", Jeff surveyed practically every software testing vendor in the field, from unit-and-code coverage firms to the functional UI types of tool providers.

 The article talks about the importance of metrics and measurement tools that support testing - comparing software testing in today's complex environments to attempting to play a symphony, saying it would be "like a symphony orchestra getting up on a stage without a single sheet of music and trying to play Bach or Beethoven."

The golden rule of software testing - the earlier you find bugs, the less costly they are to fix - was mentioned as the driving force for measuring the value of testing. Cost-per-bug found in development is indeed much cheaper.

Agreed - however there is also the aspect of automation closer to integration time which must happen in these complex environments that must happen - as there are simply too many possible combinations and points of failure in today's distributed and "loosely coupled" environments like SOA. How many aspects of enterprise software are really built from scratch, and how many are reused or re-purposed from existing components.

I wager that going forward in QA, the big challenge in testing is no longer at the code level, or even the user interface -- that isn't where the big chaos is happening, and there are already a number of processes and tools that do that successfully now.

All in all appreciated the perspectives in this industry-wide QA survey.

How Soon are BPM and SOA Merging?

I just joined in another fast forum on eBizQ with a good question posed by uber-SOA blogger Joe McKendrick:

"There has always been a huge cultural divide between the business folks, who felt that they own BPM, versus the IT folks, who own the architecture or the technology architecture, which would be SOA. So do you think BPM and SOA are going to merge, and if so, how soon?"

Judging by the rapid-fire responses alone, I can tell this strikes a nerve with lots of the experts. I can certainly understand the concern about yet more redefinition and lumping together of terms that aren't really related. However, I don't see this confusion as a problem in practice as much as others might.

In our experience the most successful SOA implementations are being driven at a high level by a BPM process, even if we aren't talking about directly connecting a process engine with a SOA registry for instance. It isn't going to happen overnight, but the SOA implementations without business process orientation will die out and be replaced by business-process driven SOA, which the business can justify budgetary considerations for.

Anyway, yet another good conversation and I invite you to take a look.

Virtualizing Over-Utilized Systems: Part 2 - Under vs. Over Defined

Here's an introduction to the concept of "Over-Utilization" in terms of Virtualization of software and services. This and 4 more posts in this series will cover concepts detailed in a recent whitepaper by Ken Ahrens and Jason English, "The Next Frontier for Virtualization: Over-Utilized Systems", downloadable on iTKO's site.

Are we Virtualizing the Path of Least Resistance?

Hardware virtualization technology is now par for the course in enterprise IT infrastructures. Enterprises can realize rapid cost savings through the use of hardware virtualization to reduce server proliferation, consolidate desktop environments, and run numerous apps with a smaller combined hardware footprint. The key value espoused by leading virtualization vendors (such as VMware, Microsoft, Citrix, etc.) is making better use of “under-utilized” servers that do not require dedicated system resources.

But what are we supposed to do about the kinds of systems that are not good candidates for hardware virtualization?  What about the critical over-utilized applications that create bottlenecks among teams that need access throughout the software development and testing lifecycle?  Hardware virtualization has a much less compelling value proposition in this scenario. 

A strategy to apply in this case is the use of Service Virtualization (SV) to reduce multiple team dependencies on over-utilized systems, allowing them to work in parallel. Let's take a look at the utilization patterns encountered in enterprise IT environments, and which type of Virtualization (Hardware virtualization, Service virtualization, or a combination of both) should be applied to receive the greatest value.

Defining Utilization Patterns

For simplicity, we’ll just categorize enterprise systems into one of 3 categories of utilization and figure out what this means from the standpoint of virtualization:

• Under-Utilized systems have low CPU/memory/disk usage, or otherwise there is significant capacity on the system to run additional applications.
  
  
• Over-Utilized systems may have high CPU/memory/disk usage (for instance a transaction mainframe or multi-terabyte system of record), or perhaps they are services outside of the control of the Enterprise, with usage limitations or per-use costs (such as third-party services, cloud computing, SAAS, etc.).
  
  
• Properly Utilized systems, which are working optimally, and can be left alone.

Causes of Under-Utilized Systems:

• Infrequently Used – Low usage does not necessarily equate with low business value (as a rarely used service could process very expensive orders), but it does usually mean that the system could get by with less processing power during those hours when it is not used.
• Excess Capacity – For teams that didn’t have the time or resources to do significant capacity planning tests, many architects hedge on the safe side of providing too much capacity rather than too little.  At some enterprises this creates a capacity glut of servers and bandwidth.
• More Efficient Software – Let's give developers and testers some credit, they can build very robust software, or perhaps optimize the software performance and use of system resources over time, to get their services to run very efficiently.
  

In many of the above cases of under-utilization, conventional Hardware Virtualization approaches have proven successful in eliminating excess capacity by consolidating servers and applications. Performance testing and benchmarking can help identify good targets for hardware virtualization, but do these under-utilized systems represent the greatest cost and risk to IT?

Causes of Over-Utilized Systems:

Why would systems end up over-utilized and unable to handle ongoing transactions, particularly in supporting software test and delivery teams?

• Common Bottlenecks – Literally this means that many teams depend upon access to the same constrained system or service.  Usually this is due to a design decision early on, but in shared environments, the usage pattern is hard to predict when a service is called upon by many customers, and other teams’ services in unplanned ways.
• Limited Capacity – More likely for services that have been in production for some time, it’s not hard to imagine that new technology teams get hardware upgrades, while older technology gets left in the dust. Or the production environment may be re-built, but pre-production servers needed by development and testing may not get the same upgrade due to budget constraints.
• Inefficient or Unscalable – Unfortunately, technologies and platforms may not provide the scalability initially advertised.

While we hope that responsible architecture, integration and deployment strategies will result in proper system utilization, it typically doesn’t work out that way in today’s heterogeneous, distributed application environments. There are simply too many variations in how the systems are connected and used, and too many components in the environment that aren’t under one team’s control.

Next post in this series: Outlining the Challenges created by Over-Utilization of key systems.

How the Recession Affects CIO Budgets

Interesting article in a recent Forbes magazine. In "IT After the Recession," Ed Sperling interviews Gartner's Mark McDonald about research conducted with a sample of corporate CIOs about the economic impact of the global downturn on their budgets. Not surprisingly, 90% of the answers did say they had reduced budgets this year.

The report points out a couple interesting trends that call for virtualization on a greater scale in my opinion. The first is in the management of server environments, an area that is only becoming more constrained. The survey notes that budgetary cuts are focused on reducing third-party and contractor support, or renegotiating support contracts.

That is a good sign compared to knee-jerk layoffs to meet budget goals, but it still will call for a lot more automation and reduction of costs wherever they are available. Hardware virtualization will remain a top priority for CIOs, given the need to "do more with less" with every technology asset, as well as the ballooning amount of storage and processor power required to handle complex workflows.

Second, in the article McDonald points out a discontinuous trend that calls for more innovation: "There traditionally was a relationship between transaction volume and revenues. As revenues grew, so did transaction volume. Starting about four-and-a-half months ago, CIOs began reporting that relationship is broken. Transactions continue to grow even though revenues are flat to down. In banking, for example, balance inquiries are one of the fastest-growing transaction types. It doesn't generate any revenue, but it does generate a lot of work."

That is a stunning report, and one that we have seen corroborated in banking/insurance, travel, defense and other industries - more transactions no longer means more revenue. Today, both customers and business partners expect 24/7 any where, real time data from the enterprise -- in fact a companies' ability to provide good real-time data such as order status, tracking, pricing, balance, etc. are key differentiators in customers' minds.

The same also applies to development and testing transaction volumes -- non-revenue generating  transaction fees, bandwidth, and data setup while working to build functionality, can increase much faster than the revenue the business functionality supports. In these cases, Service Virtualization is a valid approach, since you have systems that are either unavailable or not under the teams' control for Hardware Virtualization.

We hope as the CIOs start to employ these leaner practices during the downturn, they will come out stronger in the years to come.

Who needs a "Personal Cloud?" - Gualtieri

With all of the hulabaloo around Cloud Computing, you'd think the LAST thing we need is yet another kind of cloud, in addition to Public Clouds and Private Clouds (secured within a company's domain). Forrester's Mike Gualtieri, an application development analyst we have enjoyed talking to in the past, agrees in his blog "Cloudmania: Developers Need A Personal Cloud" that we don't need a new cloud for everything in principle, while proposing another cloud anyway.

"What developers could use is a Personal Cloud that would allow them to configure their local environment in multiple way and take it with them wherever they go. I know this sounds like virtualization and it is to some extent, but extend PC virtualization with cloud concepts and you get the Personal Cloud."

What is a "personal cloud" anyway? Is it the "Most Private Cloud?" In essence it is a way for developers to completely isolate themselves from dependencies, by having all of the services and resources they need to work and test in their own "personal cloud" environment right on their own computer. No connection to the Internet or a hosted service needed.

At this point you may ask yourself if this can even be called a Cloud, and if it is even relevant to the real world, much less plausible. Indeed, many of the comments on Mike's blog are challenging the feasibility of a "Personal Cloud" altogether. If you are talking about working with enterprise applications, you can't possibly replicate monolithic mainframes, systems of record, and simulate complex transaction and service environments right on the developer's laptop. Right?

In reality this kind of thing is happening today with the practice of Service Virtualization (you can read a paper on the concept here), when coupled with a localized Virtual Environment (or virtual "appliance") that allows a very genuine simulation of the developers' dependencies to run locally. Even if the moniker "Personal Cloud" doesn't stick, we think he is onto something.

Mike mentions Moore's law: exponentially increased processing power and capacity on the developer's desktop at lower prices. That makes what seemed to be an impossibility 5-10 years ago, a very real proposition today. As developers have more local computing power at the ready, they will be increasingly able to model sophisticated test and development environments within a VM, provided they are detailed, robust, and can perform close enough to reality to be relevant.

Anyway a good original blog and this idea is worth watching as it develops. While there are compelling reasons why Public and Private Cloud deployment are here to stay, don't be surprised when developers ask for the ability to truly work "off the grid" sometimes to move their lifecycles into parallel and eliminate the costs and constraints of access to the environment.

SOA Market Projected at $10 Billion or more by 2015

So much for the decline of SOA. Joe McKendrick reported on a Wintergreen Research projection that SOA will grow by 17% to rise from the current $3.3 billion to $10.3 billion by 2015. This is very consistent with the Research & Markets market 2008 estimate that putting the SOA infrastructure market as growing from $2 billion in 2007 to more than $9 billion by 2014.

We have been discussing the growth in SOA that we see from our perspective for some time. It is nice to see more validation of this perspective. And this is a conservative estimate.

Joe notes that AMR Research sees a much larger market ahead. They estimate the SOA market will grow from the current spending level of $28 billion to $52 billion by 2014. AMR analyst Ian Finley explained that their estimate is based on 600 companies globally, and covered a broader range of software and services than other estimates. I think this broader range of services is appropriate as SOA is dependent on many components, including governance, security, testing, validation, and visualization.

To focus only on SOA infrastructure does not provide the complete picture. There is a much larger pie in terms of the strategy, delivery and maintenance effort required to make SOA happen. It is like only counting sports ticket sales and not including the concessions, gear, TV rights, etc. Of course, this fact is not lost on the leading consultancies and SI partners, who have responded quickly with "SOA CoEs" (Centers of Excellence), coalescing this expertise for their SOA practices.

Joe concludes his post with a quote from the Wintergreen report that conveys some of the current and potential pervasiveness of SOA. "SOA reaches into every industry and every segment of the economy and represents a fundamental change in the way automated process is delivered to replace manual process. Service enabling offerings are a response to the fundamental change in IT, where enterprise competitive advantage is gained from having IT flexibility."

This is a game changer, and we are still only in the first half. Now, if we are going to call "SOA" something else by 2015, that's another matter...

Dan Woods on Brick Walls Of Virtualization

Brick Walls Of Virtualization is the title of a recent post by Dan Woods at Forbes.com. The subtitle is, "Without automation, creating and managing system images will slow adoption." I could not agree more.

The "brick walls" refer to the burdens of virtualization that come along with the benefits. On the benefits side Dan mentions the cost savings of less physical servers; increased flexibility to provision systems or change configurations; and better access to resources in the cloud. This latter benefit outsources complexity and shifts capital expenditures to operational expenditures.

You would be hard-pressed to find a large company not already leveraging some mix of hardware and software virtualization. Especially in a down economy, it has obvious value.

However, Dan then brings up the burdens of brick walls. The good news is that the number of physical servers may drop due to support cost and energy pressures. And, the potential bad news: the number of virtualized servers and devices can skyrocket without the physical constraints, and each of these virtualized devices still needs to be managed. We are seeing this all of the time in our major enterprise client engagements - virtual instances can proliferate quite rapidly if not managed carefully.

Here's another, deeper challenge of virtualization in enterprise IT: for all of those virtual servers, you still have dependencies or "wires hanging off" of those, leading to big iron mainframes, SaaS applications, 3rd party services, what have you. These dependencies are resistant to hardware virtualization, so they need to be simulated with behavioral virtualization.

I personally like to compare this phenomenon to unmarked leftovers in a shared refrigerator... [whose food is this, and without opening it up, can I figure out how long it has been here?]

Dan adds that a "whole new class of IT systems-management software and techniques is needed to create a virtual data center operating system." Part of the increased requirements for systems development and administration involves testing and continuous validation, and part of this involves better management of both VMs and Virtual Services, a topic we will discuss in further detail on this blog.

There is much more in Dan's post and I recommend going to Forbes.com to read it. I liked his concluding remark, "The upshot is that IT, which is dedicated to automating the rest of the business, must start spending more time automating itself." Well said.

Comparing the State of Consumer Web and Enterprise Applications

Here is an interesting post by Jeremy Chone in Bits and Buzz on Enterprise Web vs Consumer Web [2.0]: Top Six Differences. I was struck by this comment on one differences and thought of similar things we've seen in the field. Especially the fifth factor.  Here it is in Jeremy’s words.

"5) Integration (Loose vs Strict) ... In the consumer space, application integration happens very organically. Application providers (e.g., Twitter) or application container providers (i.e., Facebook) define  interfaces with which everybody can integrate. The result is that thousands if not hundreds of thousands of applications will potentially integrate with some of these interfaces. Consequently, interfaces tend to be simpler, lighter (e.g., REST), and often even client-centric (e.g., OpenSocial and iGoogle). In other words, consumer application integration is about breadth rather than depth. In the enterprise space, on the other hand, application integration has to be thoughtfully designed and managed and often has an impact in all layers of the application (i.e., presentation, logic, and data). Heavy and expensive technology, such as ETL, SOA, JMS, and Portal, are often an effective way to establish strong integration between applications."

Many of the current enterprise application efforts have been influenced by the consumer web (and you can call it 3rd party services, Cloud, SaaS, what have you).  However, success depends on recognizing the challenges and differences, as much as the opportunities and similarities. Much of the consumer web involves preparing for the unknown such as traffic volumes, as well as integration from multiple and unanticipated applications that you may not have control over, or be able to enforce an expected service level on.

While more is often known about the architecture in the enterprise world, the level complexity can go way up due to much deeper layers of system-to-system interaction.  There is also less tolerance for lapses in performance and data integrity.  This is where the edge between Consumer Web and enterprise applications gets particularly jagged, and where we are seeing the most innovation.

For instance, the need for software testing to be truly stateful and asynchronous as you cross over the edge between the enterprise and consumer web is a necessary thing (and quite a mouthful) -- if we are going to consume the best of both in our distributed apps.

How do you validate a transaction, if the response may move around among several third parties and decision points, and maintain its state when (and if) an appropriate response occurs? This kind of leap is going to take a lot of automation and innovation to allow the two to achieve scale.

Gartner Sees Expanding Role for SOA

I like the way this post, Gartner Says SOA Is Evolving Beyond Its Traditional Roots, starts out: "Despite the economic downturn, Gartner analysts said they do not anticipate a dramatic drop in SOA adoption."

It goes on to quote Jess Thompson, a Research VP we've enjoyed talking to at Gartner: 
"SOA underpins and enables initiatives aimed at cost reduction, such as SaaS or BPM, and we have noticed a growing interest around SOA from vertical industries like government and healthcare, which have been waiting for the technology and best practices to mature."

The post goes on to acknowledge that there are horror stories about SOA projects failing that have created a perception that the era of SOA is over. However, they add that the lack proper SOA governance is, and will be through 2010, the most common reason for SOA failures. This is something we have discussed a bit on this blog (see Avoiding SOA Disillusionment).

They conclude that quite contrary to going away SOA is actually emerging from the Trough of Disillusionment within Gartner's Hype Cycle and climbing the Slope of Enlightenment. It is good to see this trend in the field. Indeed, their previous commentary on SOA talked about fewer enterprises starting new SOA projects in 2008 - which is true - when you consider that any expansive IT department is already leveraging elements of an SOA model and it is quietly being used to create value.

Perfecting IT Strategies in a Down Economy

In today's tough economy, you can't even read an article that doesn't start out: "In today's tough economy..." This one is no different. Fortunately when it comes to IT, the message does not need to be accompanied with knee-jerk cost-cutting measures.

Last month Todd Biske offered some solid advice on what to do in today's down economy in his post, "Don’t Go On an IT Diet, Change Your Behavior". Needless to say, his post remains relevant. I especially liked his comments:

“The whole point of Enterprise Architecture, SOA, and many of these other strategic IT initiatives is to allow IT to be more agile- to respond more quickly to changes in the business objectives. Guess what? We’re in the middle of a big unprecedented change in our lifetime. My guess is that the best survivors of this meltdown will be organizations that don’t go on a starvation diet, but instead simply recognize that their priorities and goals have changed and execute without significant disruption to the way they utilize IT.”

Todd also wrote that cost cutting efforts to respond to the economy should not mean that organizations drop their SOA initiatives, or other long-term initiatives, especially when most of these efforts are trying to reduce costs cost by eliminating redundancy in the enterprise. It would be ironic as these efforts are exactly what are needed today.

SOA efforts should be more closely managed and held to their ROI objectives in any economy - otherwise, they will not have a chance to create the value you expected. It may be that closer review will actually improve the number of SOA efforts that are successful, as people will be more highly motivated to get them right. There is no time for poor execution now but there is a lot of room for innovation. As Jonathan Schwartz, the Sun COO, wrote in "Innovation Loves a Crisis," now is the time to make changes that will make you most adaptive to the new economy and best positioned to come out of the downturn on top.

McKinsey also supported this strategic thinking in their report, "Leading through uncertainty": “The task of business leaders must be to overcome the paralysis that dooms any organization, and to begin shaping the future.” Transforming to SOA - not because it's "cool" but because we have real value expectations, is one way to become more flexible and cost effective at the same time.

Accompanying that, we are also sure to mention Service Virtualization as a key means of increasing the availability of SOA assets in the lifecycle, at a small fraction of the environmental cost. It is money well spent not just in terms of cost savings, but more so in coming out of a lean time with a more agile and efficiently delivering dev and test organization.

ebizQ Discussion on SOA & Cloud Computing - Everyone's got a point...

The eBizQ site recently hosted a discussion is based on JP Morgenthal's feature article, Differing Views on SOA & Cloud Computing, and the question, which seemed to cause a bit of disagreement between the old "gray hair" and new "newbie" schools of IT, is: SOA & Cloud Computing new and unique or a continuation of the same old stuff?

Occasionally you come across a topic like this that touches off a flood of responses from the community. There are a range of insights from a "who's who" of SOA and IT pundits on this particular topic worth looking at on the ebizQ discussion site: SOA & Cloud Computing: New & Unique or Continuation of the Same Old Stuff?

Here was my response:

The future risk of the cloud is the current risk of SOA. I agree with the above statements about the hype factor of cloud computing, however there's still one more challenge that both strategies will share -- dependency. In SOA engagements, we encouraged the importance of good architecture and governance, but often it was just too easy for companies to hook up a couple web services to a legacy system for a quick ROI.

Dependency didn't rear its ugly head in SOA until the services started to be used by multiple customers, each with their own needs, and then those services were connected with the services of business partners, and used in a variety of unintended ways. A lack of access to dependent services and systems throughout the development lifecycle created a constraint on the agility we expected from SOA.

Similarly, the true cost and complexity of using cloud-based solutions will become evident only at the point when companies rely on them for critical needs. The low cost of entry of the Cloud's easy provisioning and pay-as-you-use can end up costing more than expected once the solution reaches business volume. If I pay-per-use for a cloud-based solution, I am establishing recurring costs, and increasing dependencies in my testing, development and release of new functionality that need to be addressed. Hopefully this shared concern of dependency will encourage enterprises not to sacrifice good architecture, governance, and virtualization practices for short term ROI gains.

We see many companies getting past the hype of SOA to generate real value today by applying the approach with more discipline, and many of the same lessons will apply to cloud computing.

Can You Lend Me A Shovel So I Can Start My SOA?

Joe McKendrick (aka, Joe the SOA guy), put on his economist hat and recently commented on stimulus ideas for the downturned economy in his post, Shovel-ready SOA. He writes, "perhaps we should look at service orienting as a shovel-ready 'stimulus' program for corporate growth. Because it works both ways — the current rough-and-tumble economy may be good for SOA, and SOA may be good for the economy."

Good idea Joe. As we wrote earlier, on one hand SOA is not dead or going away. We see success stories quietly happening all the time. On the other hand, like the shovel, SOA isn't glamorous anymore, but it's being used to create real value. Not only that, but we recently were musing on how SOA virtualization can help cut carbon emissions while it is saving money, hitting another aspect of the current stimulus suggestions.

Joe takes this discussion of SOA and the economy a step further as he comments that it can also help in "revitalizing business growth as well ... Consider the advantages properly implemented SOA can deliver: Increase confidence through greater transparency of applications and data, especially at a time when markets have been quaking from fear of what they don’t know; help businesses collaborate better through more rapid integration… help organizations expand or contract pieces of the business, as needed, without fuss and muss."

In tough times, nothing boosts your chances to survive, and be positioned to thrive, like enhanced confidence, collaboration, and agility through SOA efforts. We'll just have to roll up our sleeves.

Of course, before you start digging, a little upfront architecture and planning goes a long way -- you don't want to hit a wire or gas line! OK, enough, we'll put away the shovel metaphor for now. Each of these themes can be further enabled by the confidence, speed, and agility that automated testing, ongoing validation and Service Virtualization can bring to SOA.

More on SOA & Cloud Governance from HP via Dana Gardner

Dana Gardner recently posted on Enterprises can leverage cloud models and manage transition risks using service governance, says HP.

Dana had a conversation on the role of cloud computing, services oriented architecture (SOA) and governance in exploring cloud opportunities with Rebecca Lawson, Director of Service Management and Cloud Solutions at HP; Scott McClellan, Vice President and Chief Technologist of Scalable Computing and Infrastructure in HP’s Technology Solutions Group, and Norman Lindsey, Chief Architect for Flexible Computing Services at EDS, an HP company. Dana provides highlights in his post and you can also read a complete transcript of the discussion.

The panel pointed out that virtualization and the associate cost savings are not dependent on cloud computing, but they are an enabler of it. The panel focused more on the cloud than SOA, but they had a number of useful thoughts. I liked the panel’s comment that -- “In the longer term, the more overarching impact of cloud comes when your IT department can deliver value back to the business, rather than just taking cost out."

That is more than a novel idea - for the past few years we've talked too much about driving cost out of IT, instead of driving value into the business.

Some examples? "Using aspects of social networking and other aspects of cloud computing, and the fact that cloud is delivered over ubiquitous media, the Internet, to increase 'wallet share,' increase market share, maybe bring higher margin to a business, and build ecosystems, and drive user communities for a business."

This allows cloud computing to be strategic and not simply tactical. As cloud computing becomes more strategic, it becomes even more important to have a solid governance model to make the decision on whether to move from the traditional deployment model to a cloud-delivery model. In the process of getting to a service-centric IT governance model, organizations are going to have to deal with the governance model for deploying new services (see also – Dana’s post on IDC research shows enterprise SOA adoption deepens based on certain critical practices).

These governance models can better sort through the risk/reward trade-offs on cloud computing. Of course a good testing and validation strategy takes out some of the risk side of the equation. The panel added that tougher economic conditions would heighten the acceleration of cloud computing, and not just because of the opportunity to save cost (see also Direct Insights Analysts identify IT winners and losers in global economic downturn). The panel said that opportunities for the cloud to bring value are high when it can drive user communities with users and consumers of whatever it is your business produces, using techniques of social networking and other web-based capabilities. In these strategic cases that directly touch your customers, having continuous validation of services is especially critical.

For SOA Quality - there's just no free beer (Live from Gartner AADI)

Just exited a breakout session from Gartner's Tom Murphy, who is one of the main analysts covering the Testing side of our marketspace, as it edges into Agile development and ALM. His talk, "Application Testing and Quality in the Age of SOA" revisited a theme we haven't touched on in a while here: that in order to motivate and ensure good Reuse, you need Compensation.

Look at it this way: If I am a producer of a service - unless I'm a ramen-eating college student who just likes producing services for the glory of making the next cool widget, I am going to need some motivation to make that service as readily consumable and usable by my end customers as possible. So it falls to the Producer of a service to build quality into their service from the beginning.

Tom mentioned that SOA "offers a platform with a shifting foundation" so a Center of Excellence around SOA testing and quality is needed. The reward for good quality (and useful design as well) is Reuse, and that is compensated with real or "company store" budgetary dollars.

In SOA, quality is not as granular as checking the code for structural bugs or clicking on a UI to test at the end. It requires discipline and specialized tools that directly invoke and verify the services so they meet defined policies - structural, behavioral, performance, and that this process is carried out continuously throughout development and into runtime. That's a lot of responsibility if you have a well-used or critical service.

The part we often forget is the responsibility of the Consumer in this equation - If I am consuming a service, and its underlying architecture, there is no "free beer" here. I should be expected to reward the Producer that makes a service that has the most utility to my business. And in addition, I need to declare exactly how I am intending to use the service so I am not going to impact the promised service levels of other Consumers.

Check out this post from a couple years ago - "Consumption is not Free" is still one of our favorite themes and it's good to hear it again here at Gartner AADI.