The Product Development Agenda Blog

Cloud computing often fails before it ever begins and long before we ever achieve our first operational deployment. Looking into developmental nature of unsuccessful enterprise-level cloud computing initiatives, several conclusions an be drawn about the causal effects of their failures: it wasn’t the immaturity of the cloud computing, but that of its adoption. Poor planning and management plays a larger role in the long term viability adopting the cloud computing paradigm than any other characteristic. 

In order to reduce the chances of failure, these four principle activities should be part of your cloud computing initiative:

1. C-level alignment of business drivers: 

There is something magical about clouds, from the ominous power of a summer time thunder storm to the “silver lining” seen in those spring show storms. Most of us don’t know exactly how they work, but we all know what they do.  In cloud computing, it is no different.  So, while we may not know exactly what’s going on in all those distributed systems, we do need to understand what it’s suppose to provide.

Most organizations are beginning to believe that cloud computing should supply cost effective, scalable, and secure delivery capabilities. As such, many of the business drivers make assumptions that become an implicit part of the financial and operational model. In order to successfully meet corporate and customer expectations, these drivers need to become an explicitly managed part of the process. 

Cost models should not be left to back of the napkin modeling. If the organization needs to take out cost, then get it down on paper in the beginning. Since the cloud is often about converting capital expenditures into operational expenditures, the more explicit you are about the reduction, the better. Don’t forget resources in the modeling either, while you might reduce your current IT staff (less systems onsite could me less onsite resources), you will most likely need other skills to manage outsourced cloud activities.

Organizations are often looking to the cloud as a new source of revenue. Are your customers buying the cloud or are they buying the services that are in the cloud? It is a question that needs to be addressed up front. In most cases, the cloud is the cost means to the revenue ends, unless you are going to be an Amazon or Op Source. Make sure that you aren’t over estimating the power of cloud to actually create revenue by explicitly calling out these assumptions.

Here is the secret decoder ring that you can use to translated corporate discussion (please keep it between us): When a CFO says Cloud Computing, what they are implying is – I want lower cost$. When a CIO says Cloud Computing, what they are implying is – I want $omebody else to do that $tuff. When you hear a CEO say Cloud Computing, what they are implying is – Please, let there be a $ilver lining in that cloud.

2. System-level impact of cloud computing: 

It is not enough to understand that an application can be deployed in a cloud computing environment; whether it be an IaaS, PaaS, or SaaS based system. Systems fail to perform according to expectations for a host of reasons and understanding behavioral characteristics from a systems engineering point of view is a necessary means through which this risk can reduced.

In order to successful launch an enterprise cloud computing initiative, organizations should carefully consider some of the following area:

>> Architecture - Does the application architectural style (service-oriented, process-oriented, and/or data-oriented) match the cloud computing orientation.

>> Product/Software life cycle development - Software development (creating and testing) is principally done through a local machine. Leveraging cloud computing resources for development might require changes to IT policies in order to use remote resources.

>> Monitoring and Reporting - Many organizations still heavily rely on home grown/proprietary monitoring and reporting tools. What changes need to be made in order for your NOC to continue with their monitoring operations?

>> Business Continuity (BC)/Disaster Recover (DR) - Is the application operationally taking advantage of continuity of operations characteristics supplied by the cloud provider? In the event of a catastrophic failure of the cloud (e.g., lack of additional resources, provisioning failures, access disruption, etc.), can services be continued through tertiary resources provisioning services (e.g., private data center)?

>> Partnerships/Vendor Integration - Do you outsource some or all of your development and operations? How will these providers gain access to cloud-based resources? What happens if the partnership is terminated? Who owns the data, metadata, and other IP created in the cloud?

>> Security (Physical and Logical) - If you are using a PaaS provider (e.g. Force.com, Google Python, etc.), how do you know their frameworks are vulnerability free (e.g.SQL injections, buffer overflows, race conditions, insecure file operations, etc.)? What level of security monitoring and reporting can you conduct in the cloud? Can your cloud provider supply audit reports for physical/logical security tests?

>> Intrusion Detection/Notification/Mitigation - Attacks are inevitable, it is just a question of when. How will your current IDS practices need to change in our to support cloud-based operations?

>> Privacy -  All data is not the same in a global multinational regulatory driven world. Just between the US and EU, there are significant differences in what data can be stored and where one can store it. Is your data architecture designed for this type of global distribution?

>> Regulatory Compliance and Standards Assurance - Are there any governing compliance requirements? How does SOX, HIPAA, or even SAS 70 impact your cloud computing practices?

>> Cloud Computing Switching and End of Life - If you needed to migrate from your cloud computing provider, could you? Is the technology base of your PaaS provide portable to other service provides (language, frameworks, meta-data, etc.)? Will you have to redevelop your provisioning scripts?

3. Migration and Modernization Planning 

It is not uncommon to see some level of migration planning in today’s cloud computing modernization program. Most migration activities make varying assumptions as to the viability of their products/services on a cloud environment. These assumptions are often erroneous and in many cases applications will need to be modernized before they can  be successfully deployed. For example, applications that assume they can persist data to their local drive will fail to recover between Amazon Machine Instance (AMI) sessions. 

A robust proof of concept (POC) period for every business critical applications should become part of the migration and modernization planning activities. The POC needs to test all assumption that are critical to the success of the project, from ensuring systems can be deployed to their desired cost/revenue characteristics.

4. Cloud Computing Performance Metrics

Beginning able to effectively manage cloud computing ecosystems requires the adaptation and augmentation of measures and metrics associated with development, operations, financial, and continuity of operations activities. Adding adoption metrics, such as percentage of cloud computing process/data work streams, and comparative metrics, such as operational cost comparison, is essential for proving out the cloud computing business model.

Moving to the cloud isn’t easy, but taking a few precautionary steps can dramatically improve the likelihood of your success. Gainly alignment, thinking systematically, migration/modernization planning, and measures/metrics are just a few that necessary areas that, if aggressively addressed, will keep you away from those failure land mines and on a much safer road to success.  

Tying the costs of outsourcing to the achievement of outcomes that support real business objectives sounds like nirvana.  You can assign a value to a given activity and it can help you better evaluate the ROI that you're getting too.

Not surprisingly in this economy, outsourcers are trying every angle they can to get business and many are tired of just reducing rates.  So dont' be surprised if they start making promises about delivering against outcomes and outputs.  But don't just believe the hype.

The real question that you have to get to is how committed are they to really delivering on outcomes and how much are they just trying to suck you into a sales conversation, only and perhaps purposefully, to shift the conversation back to traditional outsourcing engagement models.

Do your own due diligence and understand how committed they truly are to this kind of engagement.  Here are a few questions to ask

• What they've changed organizationally to enable them to deliver against outcomes instead of providing bodies?
• What % of their business is coming from outcome or other performance-based arrangements
• Especially for vendors that have a heavy offshore component, how are they dealing with the philisophical and cultural shifts required to really deliver.

What you may find is that their's not much beneath the veneer.  In the meetings that our CEO Gordon Brooks and I have had with journalists and analysts, we've of course gotten very good feedback on our approach to outcome certainty (especially in BusinessWeek's NEXT blog). 

But what's really struck people, like AMR's Dana Stiffler, is the extent that we've embraced this approach.  Today about 20% of Symphony's engagements are outcome or output-based with another 40% of so utilizing other performance-based mechanisms like revenue sharing, fixed margin and SLA's.  According to Stiffler, she hasn't heard of anyone else in the software product development outsourcing space really embrace outcome-orientation at all and even in the big Indian IT shops its a tiny percentage of their business.

Now why don't other firms adopt performance-based contracts as aggressively -- because it's hard.  It's hard to track the metrics that matter.  It's hard to change the way that your employees think about thier work to align with delivering outcomes.  And most of all it's hard to change the risk profile that your company is used to when taking on these committments.

Over the next weeks and months you're going to be seeing a lot of discussion from Symphony about "Outcome Certainty".  So let's start out with a definition and talk about why we think it's an strategic direction for the Company and, more importantly, why we think it's a great idea for our clients.

What Does Outcome Certainty Mean?

Outcome Certainty is about getting the results you require to meet your business goals. It’s about achieving goals, driving innovation and bringing predictability to costs, schedules and quality of the outsourced software product engineering process. Traditional outsourcing revolved around contracting for inputs (resources, rates), but were abstracted from results. Outcome certainty puts the focus back where it belongs – on getting the work done.

At Symphony Services, we know that especially in the economic times that we're in, but really in any economy, results are the only real measure of success.  So we're trying to change the emphasis of our client engagements in the outsourced software product engineering market to focus on helping our clients achieve engineering outcomes tied to business objectives instead of simple resources and rates.

At Symphony Services, engineering outcome certainty means we stand behind the commitments we make, and the work we do. We are delivering what our clients need -- the predictability (costs, schedule, quality) required to execute on their business strategy. And we put our money where our mouth is. Our risk-reward partnership models and focus on metrics and transparency ensure complete alignment with your business and R&D objectives.

Like I said, you'll be hearing a lot more about outcome certainty from Symphony. So let me leave you with a few words about it from our CEO, Gordon Brooks.

Symphony Services CEO Gordon Brooks talks about the company's focus on "Engineering Outcome Certainty" -- that is aligning outsourced software product engineering contract with meeting clients' business objectives, rather than just contracting for resources and rates.

Well many software and Internet companies have had their heads between their legs the last 8-10 months as credit markets tightened, the recession worsened and the future became more cloudy.  Well there's signs that there's a light at the end of the tunnel and it's not an oncoming train. 

TechCrunch's Sarah Lacey talked about a coming IPO boom, noting that IPO registrations are up, but it isn't only for the uber-platforms like social networking giants Facebook and LinkedIN, but a broader rally that includes smaller companies like OpenTable.

In talking to clients and prospects we're also seeing a thaw in the market and companies are gaining confidence and visibility into the future, bringing projects back on line that have been on hold for a while.  Are you seeing the same positive signs at your company?

Well for many start up companies the answer is easy.  There is likely only one product that so often the focus is on what function gets hit - sales, marketing, R&D or administration.  Likely the answer is "all of the above".  But today many companies have multiple products - some times a handful, sometimes hundreds.  This is where the conversation gets more interesting and more difficult.  This is when you have to shift (although not dismiss) the focus from functions to products.  Which products do you invest in?  Which do you phase out?  The choices are not always clear, but what is clear that you need a consistent process to continuously evaluate your application portfolio.

It would be nice to say that every company has a structured process to evaluate their "software assets", but alas that is not the case. After talking with hundreds of people in all sizes and types of companies, I've found that the decisions of how, when and what to do with products differs widely from company to company.  

Some companies use a framework based on the age-old Boston Consulting Group matrix, others have their own system.  And there can be great differences in what attributes you evaluate and the weight that each attribute bears in the final decision making process.  But at the very least, you have to start with creating a process that the entire organization buys into.

All parties have the same goal of improving company performance. However, the main stakeholders in each company differ on how to achieve that level of improvement and what programs deserve higher priority. What the CTO believes may be quite different than, say, the view of Engineering, or what the BU leaders and Finance are looking at.

That is why creating a well documented and communicated plan is critical for evaluating products and making decisions. This will reduce, not eliminate, the subjective factors that go into decisions.

Next time we'll look at some of the common elements to a successful product portfolio framework and look at what options are available to you once the evaluation is complete.

I was catching up on my Facebook posts and read IDC analyst Michael Fauscette's latest blog post where he wonders whether the emergence of the tech mega-vendor driven economy mean the end of innovation?  Mike makes some excellent points in the article, discussing the roles of continous and discontinous innovation, but there were a few things that I think I'd like to amplify and some that he missed out on that I'd like to add.

1. I agree that many large companies struggle with developing innovations and that most companies focus on feature/function improvements that are just a function of corporate inertia.  But I also think that this discounts a very important problem: many software companies don't have a good innovation management process. This is the probably the single biggest reason for failure to deliver impactful innovations and the most underrated aspects from an organizational perspective. If you don't have a working funnel to evaluate new ideas and winnow them down to the few that get prototyped and implemented, you are left with a situation where you are hoping that the best ideas (or any ideas of merit) actually make it to your customers.  Worse yet, you lose the ability to track value of ideas generated and implemented, calculate an iROI (Innovation Return on Investment) are build institutional knowledge of what works and what doesn't to inform future investment decisions. BTW, at Symphony Services we have another way of evaluating the return on innovations from a revenue perspective called the Vitality Index, as opposed to looking at cost driven evaluations like R&D  as a % of sales, # of patents, etc..
2. I also think we need to explicitly recognize that while start ups are usually the source of many of the discontinous innovations, many start ups are founded by employees who worked at bigger companies, and came up with good ideas that were not acted upon by the big companies in which they worked. So they leave, create their own company and try to bring them to market.  Even some of the most progressive companies like Google, who promote the fact that they want their employees to spend 20% of their work time on coming up with new ideas, leave thousands of "inventions" (it's not an innovation unless it generates revenue or reduces expenses) on the table each year (sometimes for good reason) and people still leave to develop their own ideas.  This is actually a very beneficial thing in my view and keep a healthy innovation ecosystem alive and well.
3. Michael also points out that due to the premise of delivering better, more predictable returns for their shareholders, the big mega-vendors don't make the investments in R&D to generate these new innovations.  I don't think that's actually true.  Microsoft spends billions, but seems to get very little in return.  Many times I guess they're just spending on the wrong things.   For the large companies, I think it's generally cheaper -- and easier -- to let others invest in new ideas and then just acquire the companies that are successful.

I have been asked a lot about the where/what/why/when/how one should start when addressing the need to adopt cloud computing within an organization. This is one of those questions that has many right answers and few wrong ones. The only wrong answer that I would address, for right now, is the “not addressing it at all” argument. Given the transformational nature of cloud computing (it has an impact on software, operations, sales, marketing, etc.), it is important to at least raise the question for discussion. With that wrong direction aside, let’s look at one right direction that will help management gain a perspective on at least the “what.”

Your leadership could benefit from understanding how our products/services map into the cloud computing space. A simple way to do this is to take an inventory of our offerings and map it to a Cloud Computing Reference Model (CCRM or C2RM). 

A quick side note - while there are no formal C2RMs, the NIST (National Institute of Standards and Technology) is promoting a version that seems to make sense many practitioners in this field. This particular version categorizes the cloud along subscribers/supplies, SaaS, App-comp/PaaS/IaaS, and Physical Infrastructure. While I will discuss this model in more detail later, for now it looks pretty complete, with the exception of addressing internal vs. external cloud computing characteristics.

Using this model, your organization can allocate each solution to a service level. You also want to cross categorize it as a function of external cloud (e.g., Amazon, Force, etc.) and internal cloud (e.g., IBM, etc.). For example, using the external cloud for QA testing of your products often makes sense (External Cloud.Amazon EC2.Your Product). For many organizations, you probably won’t think about getting in the Platform as a Service space (Internal Cloud.Platform as a Service.Some Scriptable Platform), but using external cloud PaaS providers, like Force.com, should be considered (External Cloud.Platform as a Service.Force). But for which application(s) and how they should integrate with external/internal supplies, is an important question to be flushed out in later analyses.

Keep in mind, as you go through this process, which is one of many, the initial allocation should be based on high level business propositions (revenue/margin increase, cost reduction), that will later be vetted more completely. This kind of asset mapping is one of the first parts of developing a comprehensive top down driven cloud computing strategy. Understanding where you fit in the reference model and how it will further promote our business interests is essential if we want to make the cloud more than just a passing fad. Make sense?

I always liked this military term.  For those who have been living in a bunker for the past 10 years, Force Multiplier refers to a factor that dramatically increases the combat-effectiveness of a given military force.

In the world of software testing, a strong and well executed test automation program can act as a force multiplier for not only your QA organization, but the entire R&D team.  With an aggressive test automation program you can:

• Reduce test cycles by at least 40%
• Slash testing budgets by removing the time and effort from manual testing
• Vastly expand code coverage

Even just on a pure QA testing budget perspective, Symphony Services is driving ROI's of 2-3 months, saving in one case about $100K/month in testing costs with our test automation services. 

So it all sounds great, right?  Yet when we engage with clients and prospects we find that while some have some level of automation, it's often quite low, with less than 25% of all test cases automated.  And of course many others haven't implemented automation at all.  Why is this?  I've got a couple of ideas:

1. Lack of Automation Expertise: This is of course the most obvious reason why people don't use it aggressively.  But that shouldn't mean that you should eschew test automation as a strategy.  There are plenty of companies like Symphony Services who have the right expertise, and automation frameworks to help -- although not as talented ;)
2. Wrong Automation Strategy: This of course pre-supposed that there's a strategy in the first place and not a decision to just start automating cases.  This problem manifests itself through Wrong selection of test cases, inadequate maintenance considerations, insufficient ROI models lead to inefficient test automation with low ROI yield.
   
3. High Upfront Automation Costs: That is if you're starting from scratch.  Highly maintainable scripts require careful planning, development of reusable framework & components.  If you don't have these handy, this increases upfront investments and reduces time to value.  Additionally, you need to drive towards lights-out execution to really ratchet up the benefits, but the ability to do that needs to be designed in at the beginning.
4. No Attention to Script Maintenance: This is what usually kills an automation project.  Teams begin an automation project, but then can't keep up with the script maintenance.  So the scripts get out of date, the benefits begin to melt away and the project seems like a failure and is abandoned.  Common approaches such as Record & Replay yeild fragile automation scripts and don't work.  You need to access re-usable libraries and modular, self-sufficient scripts.

So what's holding you back from implementing test automation?  Of course if I'm all wet, let me know that too.

Please write back and let me know your experiences with test automation.

QA Test Automation is a Force Multiplier http://tinyurl.com/nab2f5

For anyone that's going to be in New York City on June 23rd, be sure to check out a very interesting panel on Cloud Computing that will feature Shankar Hegde, Chief Architect at Symphony Services, hosted by the NYC Chapter of the International Association of Software Architects (IASA).  Details below.

Meeting Topic: A panel discussion on "What Cloud Services means to Architects in the Industry".

The panel will focus on the business and architecture considerations of “the Cloud”. Panelists will discuss the various cloud based services such as Amazon Web Services, Google AppEngine, The Azure Services Platform, SalesForce, Rackspace Mosso, etc. Each panelist will get five minutes to introduce themselves and make some opening remarks. Then it will be a free-form discussion/Q&A after that.

Panelists:

• Andrew Comas. Vice President Architecture, Cordys. Cordys is the workflow engine behind Google Appengine.
• Vivek Sharma. Works for EngineYard, a PaaS provider for the Rails community.
• Shankar Hegde. Senior Client Partner & Chief Architect, Strategic Consulting, Symphony Services, Waltham, MA. He has compared various services and been advising companies on cloud computing.
• Abhijat Vatsyayan. Bristol-Meyers Squibb. Has been working with AWS (Amazon) and Google app engine
• Bill Zack. Architect Evangelist, Microsoft Developer and Platform Evangelism. A Microsoft Cloud Believer.