Welcome to the Product Development Agenda Blog

The Product Development Agenda will cover a variety of topics that interest executives at product and services firms -- from technology trends such as SaaS and SOA, business trends and, of course, best practices across the product development lifecycle.  A full RSS feed is available or you can subscribe via e-mail below.

Subscribe by Email

Your email:

The Product Development Agenda Blog

Current Articles | RSS Feed RSS Feed

SOA DoE - It Can Save Your System

Posted by Dr. Jerry Smith on Mon, Mar 02, 2009
 | Submit to Digg digg it | Submit to Reddit reddit | Add to delicious delicious | Submit to StumbleUpon StumbleUpon 

I recently reviewed an article "Test SOA for the unexpected" by Rich Seeley that got me thinking about SOA testing frameworks.

 


The design of experimentation (DoE) is probably one of the most significant testing issues one faces in SOA and is more complicated that most other archetypes. Take, for example, three simple services that can be choreographed in any order, each capable of interacting with each other. How many tests do you need to ensure 100% functional coverage? You're right if you said 6 (3x2x1). This is a classic factorial problem, in this case with 3 service. Now, let's say your system has 100 services, so how many test would you need? Again, your right if you said 100!, or 9.33x10^157, and it would take you over 3x10^150 yrs to test completely, if you ran continuously a test every second of every day.


Traditional testing in these kinds of systems are impractical given the number of total tests needed to understand/certify the functional/non-functional behavior of the system. So what do you do in these circumstances? Well, that is where a good , well thought out, SOA design of experiment (DoE) comes into play. DoEs are all about reducing the number of experiments without unnecessarily diminishing the value of the test. Such DoE are call Partial Factorial DoE. A good SOA DoE for the 100 services, for example, could reduce it down to 10! or 5040 tests, a significant reduction in effort and cost. 

 

So, the next time you are thinking about testing your complete SOA environment, ask whether or not you have the right type of testing framework (DoE) and whether or not you really need to do all those tests in order to show that your systems is actually working correctly or not. 

 

Dr. Jerry A. Smith 

Tags: ,

COMMENTS

Currently, there are no comments. Be the first to post one!
Post Comment
Name
 *
Email
 *
Website (optional)
Comment
 *

Allowed tags: <a> link, <b> bold, <i> italics

Receive email when someone replies.