Wring ROI out of your UDDI Registry/Repository by plugging in SOA Validation…

We’ve been talking about validating SOA Governance approaches for three years now, but surprisingly, we have found that very few enterprise IT shops of any serious scale are actually using them to their potential at this point. I had lunch yesterday with one of our wily gurus on this topic, Ken Ahrens, and he aptly noticed that the practice of SOA Governance just hasn’t kept up with the grand expectations we had of it. Why?

It may be that these companies haven’t seen any tactical value to that registry, which they picked up along with the rest of their SOA shopping spree. They are more concerned with the integration of their existing and new technology assets. They found that while they can put all of the Service descriptions and locations in one place, it didn’t add a whole lot of incremental ROI if the developers in that department already knew about the Services they were planning to use. You see, a lot of people like the concept of Governance and having Policies, but it’s not necessarily practical all by itself for the way businesses construct and leverage applications. This happens because SOA Governance without Validation doesn’t provide any assurance that it’s actually meeting the business requirements set forth in the Policies. In essence, it is like posting a speed limit, but not having the radar gun to ensure that drivers are obeying the rules of the road.

Think about the gap between what the BUSINESS is trying to do, and the actual IMPLEMENTATION of the technology that makes it happen. The further you are from the actual implementation logic, the more difficult true validation becomes:

As you can see here, the more layers of abstraction you have, the less likely all of these layers will work together when you hook them up, and the harder it can be to validate behaviors at other layers that may affect the one you are testing.

It can be a long road indeed to dig that deep. You might have a great idea of what you actually want to do, but that may not be grounded in a way that all of the teams are addressing it. This is becoming even harder in today’s economy, where you have a lot of partnerships, acquisitions and outsourcing – and therefore very little control over the day-to-day activity of your development teams that you are relying upon.

Being able to connect a UDDI registry, where you store everything in one place, with a strong Validation strategy can be a big advantage in this environment. But to make it work, that process has to be continuous. There are several types of Continuous Validation:

1. Checking continuously at Build Time. Every time you are checking in a new service, you are automatically validating that it works before it becomes available as an asset.

2. Continuously checking on a Scheduled Basis: We call this the “belt & suspenders” approach of making sure that your applications are still working as described, as you may not know if the service was changed by another party, or brought down by a performance issue or dependency problem.

3. Leveraging UDDI to make your BPM and Integration tools, and their associated tests, find the appropriate services for the workflow they are validating. With UDDI v3, it's even easier to read your endpoints, and that can be integrated with many different tools.

4. Reporting on Usage and Value: Figure out which services are popular, and are becoming key components of the SOA architecture. For organizations looking to "trim the fat", this gives the SOA team the knowledge of where to focus testing, capacity planning, and future integration and development labor.

All of the above are examples of how the Validation practice can add teeth to that SOA Governance effort. So we’d like to catch issues at Change Time, and validate each service then, but we also need the additional safety of checking structural, behavioral and performance factors at runtime, and reporting on the success of each effort to maximize efficiency. It’s not just about checking the WS-* protocols to make sure they are structurally correct.

Take for instance the idea of pure Web Services compliance. If you were to open up a registry and scan all of the services out there for WS-I and some flavor of WS-Security compliance, that would inevitably return with about 1% of the services being compatible. It turns out, the tools and processes that generate Web Services make a lot of custom decisions in constructing the WSDL and SOAP messages in order to be more robust and compatible for their own platform choices. Not to mention the underlying databases and apps those are talking to.

There are so many non-standard decisions we see every day in the field. So we need to clear our minds of the idea of simply thinking of “compliance” testing as the only way to achieve Validation for SOA Governance. We need to test and validate a very Heterogeneous SOA world to get the compelling value out of that SOA Registry that will make the CXO sit up and take notice.

For instance we were at a major telecommunications company that is using Systinet, which has a lot of great features. They were storing Services and Policies in there, but they weren’t really relying on that registry for everything they need, because those services couldn’t be meaningfully Validated at change time and runtime.

An example is the service that returns "true" for compliance checks, but doesn't actually do the work that it was supposed to perform. Depending solely upon the web service response (it came back true and didn't SOAP Fault, so it must be OK, right?) can lead to a false positive. We can perform deeper validation to ensure that the "true" response is backed up by an actual system of record or transaction layer that was exercised. The same values apply if you are using CentraSite, BEA ALER, or other leading Registry/Repositories to manage those services.

Validating Non-SOA-ready elements too? Outside of the registry, most companies connect several tools together that aren’t WS-I compliant, and they can talk quite effectively. This isn’t a new concept – in fact it has been around since CORBA. And it is still how most integration happens today. Now we need to automate the Validation to account for all of these flavors of service integration, and when and if those technologies are Service-enabled, pull them into the SOA Governance platform as a system of record. Of course, we offer a solution for doing this kind of UDDI validation in LISA.

So, if you already have a SOA registry, and you are frustrated with being able to get meaningful results out of it, don’t give up on it – there is lots of ROI in there! Take the next step, and don’t just test for compliance, but actually validate that those services meet your defined SOA policy. If you can define a policy, then define an accompanying test that actually validates and executes that policy, and use it at change time and runtime.

So you have a SOA Management dashboard. How do you drive it?

Lately we've done a lot of research and publishing on how Dev & QA teams can ensure better quality throughout the software development and release lifecycle. But what about the IT Operations guys? It seems we are encountering a disconnect between the Enterprise Architecture and Integration disciplines that are moving to SOA, and the people who need to monitor and maintain these systems in deployment. There are a lot of companies starting to focus on better SOA Governance, which includes the management of business processes and workflows, and the services and applications behind that. Great stuff happening on that side of the IT shop. However, often the actual running applications are still in the purview of IT Operations teams. Don't get me wrong, it is a great thing that we have a team that is manning Mission Control -- hopefully providing an impartial report and monitoring function that is crucial, when you have live customers depending upon these apps in deployment. And there are mature tools out there that provide this kind of live "dashboard" of how the implemented apps are performing, like TIBCO Hawk, HP OpenView, IBM Tivoli, CA/Wily, etc. So why would these IT Operations teams be interested in SOA testing and validation? Well, it's really about "causing the cause" of performance and latency issues in these systems. If you are trying to prove that live SOA infrastructures are ready to roll, it's not enough to sit and watch the dashboard for that "Check Engine" warning light when a condition is caused by too many customers using those apps. The point I'm making is to use automated Test Execution as a way to drive expected and unexpected behaviors into your SOA Monitoring dashboards. This can be done both at integration time in staging, and against live applications, if you can provide a test window of time to make this process happen. For instance, we recently worked with a large energy industry customer who wanted to prove that their live implementation was able to handle a realistic profile of varied transactions in their system. So instead of waiting for a rush, they manufactured one -- by executing automated test transactions into the system, and monitoring the performance results within their Operations dashboards. Then they could feed the same data out to their SOA Testing platforms to tell the tests if they performed according to service levels. We recently put out a little news on this kind of integration - it's a pretty exciting way to apply Test & Validation to the Monitoring applications that have been around and evolving for some time: http://www.itko.com/site/company/news_65.jsp To me, that is a better machine - and it takes away a lot of that disconnect between Development and Integration teams, and the Operations people who are ultimately answerable for customer service levels.

Feeling Crushed under the SOA Data requirement?

One of the most difficult obstacles to attaining enterprise-ready SOA is the sheer scale of the systems and data that need to be managed. To test the actual results of an SOA application, we need a very realistic set of data – both positive and negative – to input, and then get out of the environment under test.

True, we can map much of our interaction with other Services according to the metadata we set forth during architecture and design processes. But when you get past that ideal model of connecting the endpoints, you still have the nitty-gritty of a CRM mainframe, or an SAP or Oracle Financials enterprise system, and the administrative owners of that system, to contend with. The data and business logic embedded at these layers has been added to and customized over the course of several years.

So why can't we just have developers and testers work against the live SOA system?

Well, those system administrators might be reluctant to provide access to key business systems in deployment. Beyond that challenge, getting a bed of realistic test data in place can be more than difficult – and hardware virtualization doesn’t scale to replicate the terabytes of data such an implementation requires. Implementing a complete mirror image copy of the system to test requires another enterprise license and implementation team – far too costly in scope. In addition, managing SOA data in order to do successful service development, integration and testing can truly be a moving target. It's tough to maintain that context of an actual user moving through the system, without actually having access to every implemented layer.

The best practices for overcoming the data crunch isn't by any means an easy road, but it has to be done.

• It still starts with good architecture - mapping out realistic business workflows, and the metadata relationships that define them.
• Next, we need to capture as much of the data as we need to provide a realistic test environment for SOA. There isn't a way to replicate all of it, but we need to obtain enough to encompass most of the workflows we've defined. Virtualization of test beds, and the behaviors of apps as Virtual Services, can help you get to the point of reaching the 80/20 rule for the data you need most often.
• Finally, we need a strong SOA Governance approach to staging, promoting and deploying the application, which includes continuous testing and validation of the expected behaviors, and underlying data. No amount of simulation in development can account for all of the unforeseen consequences of changes in the deployed system.

Part of our approach is to automate that process of data capture and modeling with Virtual Services. By monitoring a given Service and all of the live and test traffic that is going into, and out of it with LISA, you can get a pretty rich data set. Often people tell me that it must represent only a scenario where the data is not very complex. Not so. Though admittedly, the more complex your data set is, the more elaboration and checking you need to do on that data set. And what if there are data errors within your Virtual Services? Well, in a sense that is what you are looking to uncover, right? However, it is important to note that there is no shortcut for Continuous Testing & Validation. When you move into staging and deployment, you need to move from that virtual data model to actually doing continuous integration testing. The point is - to get 80% of the testing you need done at those early stages accomplished by using a Virtual Service data model, then you can have a more dedicated testing effort with less conflict against that live data in deployment.

Obviously, there is much more to this process than I can cover in a post, but I hope you will seek out leaders and solution providers that can help you accomplish all three of the above goals.

Education is Essential – SOA Master Class

iTKO is honored to join industry leaders webMethods/Software AG, ZapThink, and ebizQ to present a tremendous community resource for SOA knowledge, the SOA Master Class Online (www.soamasterclass.com). Some of SOA’s best thinking can be found on this education and community forum site. You’ll find great video-based tutorials on SOA concepts, architectures, governance, business issues, and complements of yours truly, SOA quality. If you’ve spent the last several months trying in vain to put together a complete understanding of my thoughts on SOA Quality and the SOA Testing as a critical aspect of SOA Governance, you will be pleased to see what’s in the SOA Master Class. This is a completely free resource. Our goal is simple enough. We are committed to SOA’s success in the market. We know that as the talent level of SOA architects, developers and testers matures, the SOA market matures. Those who are enabling that success with great content, analysis, and technology will benefit as well. This is one of those true win-win scenarios. Please check out the site to read, watch, learn and participate in the dialogue on all things SOA.

A humble test may just make SOA Governance Policy someday...

The marketing guys are at it again - read David Thompson's recent discussion on SOA Testing. Actually, we are very glad this discourse is happening out there. SOA Testing has long been relegated to a project milestone somewhere after development and integration happens. Our CMO Jim Mackay just had to chime in. SOA Testing is a continuous part of SOA Governance, and not an event that unit tests a specific technology as a pre-release feelgood. True, you do need to test the Web Services (if that is a part of your SOA approach) to be compliant to WS-I, etc. And of course, you need to be able to test the Messaging layers for compliance with the platform as transactions move through the architecture. But these kinds of tests don't tell you if the business requirement is being missed. Or if the upstream and downstream dependencies are creating problems. You also need to test the database, and the front-end HTTP. And the EJB itself. And both the legacy CORBA objects and the new POJOs sitting at the implementation layer. If SOA is making that round trip, your testing should make the round trip as well. And testing should support the entire extended set of players collaborating on SOA, from the people writing the requirements, to the developers implementing them in SOA, and the QA teams verifying the functionality and performance. And SOA testing should be continuous, not just in development and integration, but in deployment, because an SOA by nature is never a static application. Remember the old School House Rock educational commercials? "I'm just a bill, yes I'm only a bill, and I'm sittin' here on Capitol Hill..." Well, the humble Test is just like that bill waiting to become a law (or an SOA Policy). In order to get there, it is going to need the support and nurturing of the SOA community at a company level, and at large. In other words, we promote the humble test as a actionable aspect of the vaunted concept of SOA Governance. To be part of Governance, the SOA Test must contribute more than a specific technology checkpoint at a specific point in time. A test must span the SOA application, continuously, and in so doing, it becomes a verifiable SOA Policy. More on this concept to follow - and your thoughts welcomed.

What is SOA Governance?

(This is the second in a series of "SOA Boot Camp" educational articles from iTKO founder/chief architect John Michelsen... enjoy! - Jason, ed.)

With all the hype around SOA and certainly around SOA governance, sometimes it’s hard to get a good clean definition of what it is, what it’s made of, and what is relevant.

SOA governance in our view (and in the view of Frank Kenney from Gartner), is made up of 3 components:

1= Registry/Repository, 2= Policy, and 3= Testing.

1. The Registry/Repository is the standardized way of storing and retrieving information about services and policies and the variety of other metadata around services that are required in order to make a widely distributed SOA solution work. So this is certainly a critical piece of a SOA deployment and readily available in the market at this point. It’s a fairly well understood problem, UDDI is the protocol or the standard around which the Repositories are working and at this point every major SOA platform and a few third party Registry Repositories are available for you. So that’s pretty much a well-understood space.

2. The second area which we should discuss is Policy.

Policy is the legislative arm of the government, if you will. In the legislative arm you need the ability to author policies in much the same way we would do in our society at different levels based on your location relative to the item that your trying to put policy upon.

For example, at the Federal level we have laws that are very broad in nature; at the State, County, City, and even Home-owners Association level for me, the policies continue to get more and more finite as well as more and more constraining.

In fact the Federal Government won’t have anything to say about where I can put my American flag in my yard but my Homeowners Policy is all about that. So given that, policy is how we author the expected behaviors and the standards upon which those behaviors must perform.

Simply put that comes down to two different things:

A--how you express functionality, what standards you use and what ways you provide that functionality, and ;

B--the functional integrity of those services themselves. So the policy space is much bigger than a single vendor’s solution is going to be.

We’re going to have to deal with the idea that unlike Registry/Repository, it’s not going to be a one-product solution. In fact, your policy needs may be much less on the “does it match WS-I interoperability” kind of policy, and much more on the functional integrity side; such as: “Do my top customers get sub-second response time?” and “Do I make sure that if I ever have production failures, that they escalate properly into the business?” Those are the kinds of policies that we need to pay attention to and your needs will dictate the kind of policy solutions you’ll need for your SOA Governance.

3. The third piece of Governance is Testing. Clearly we’ve had an SOA testing solution for years now with our LISA software, and this is the area where we will play most of the time, although a lot of what becomes written as policy needs enforcement in testing.

Generally speaking, Testing could be called the Executive Branch or the enforcement arm of the Governance Model.

So it answers the question simply: How do I know that the policies I’ve established are actually being upheld and on an on-going basis? Because as we all know, something that works today, especially in a SOA environment doesn’t necessitate it works tomorrow. So as we try to put in a Governance solution we’re going to have to think along these three lines.

Let’s take one step back and talk about why Governance is important to begin with in an SOA. The simple answer among many is trust. We use the government in our real world to establish a sense of safety and trust in that our behavior has certain guidelines and that other peoples’ behavior is within certain guidelines and that there are consequences for that. That’s what makes our society safe and breeds trust.

Within a SOA world we now have to deal with horizontal trust. I’ve always had to trust my manager -- but I didn’t ever have to trust my peer. Now within today’s SOA world, I do.

To create this sense of trust and to establish the enforcement of unexpected behavior, or at least behavior that’s outside of policy, we have to have a system in place to allow for that.

That is why Registry/Repository, Policy itself; and of course, Testing are an important part of any SOA Governance strategy.

- John Michelsen, Founder and Chief Architect, iTKO LISA.