Survey Results: What does the WS-Testing Community Think?

Almost a year ago, we released the first edition of our free WS-* only version of LISA, in LISA WS-Testing. Now as we near 10,000 downloads, we wanted to know how teams are progressing in their adoption of Web Services testing as a part of the greater SOA quality assurance process. We had hundreds of responses, split almost evenly between people in the business of making software, and users working for another vertical industry, and the group provided lots of great feedback and comments as well. Thanks to everyone who participated in the survey. Some of the industry high points of the results:

• Collaboration on Quality Outside QA - More than 60% of the users were not QA practitioners - showing that Architects and Developers are increasingly getting involved in testing.
• Test Your Own Services - A full 75% of LISA WS-Testing users say they are testing their own services. That is a significant increase over last year's estimate of just 40%, showing that almost all users of WS-Testing are at least in the early stages of building some services.
• Java vs. .NET Testers Steady - Same as last year, almost half (44%) of the testers are testing .NET services, while almost all are also seeking to test services on one or more forms of Java services/integration platforms (BEA Weblogic, IBM WebSphere and JBoss are still tied for the most users.) Since most surveys selected more than one platform, we can see that technology heterogeneity is here to stay.
• Load & Integration Testing is Key - More than 90% of the participants rated Load Testing and Integration testing as important features they would look for beyond functional and unit testing of Web Services. Web UI testing was also rated very high at 80%.

How are we doing? We also asked the WS-Testing users if they were aware of some of the free educational and support opportunities from iTKO, and deeper features of the full LISA edition, and the results are encouraging.

• Forum participation. More than 85% of active users have visited the iTKO Forums for support questions. Thanks for asking (and helping your fellow peers), as the community of all LISA users builds out a knowledge base of tips and tricks for services quality. Visit the forums here (registration required): http://www.itko.com/forums.
• How can iTKO help? A majority of users asked for more training and getting started resources from iTKO. Stay tuned for more beginner and advanced testing training materials and videos.
• World beyond SOAP testing? 75% of the WS-Testing edition participants were aware that the full version of LISA SOA Testing supports Web UIs, EJB, databases, ESB messaging, load testing and more.
• Useful enough? More than 90% of LISA WS-Testing users would recommend the tool to a friend or colleague. And more than 80% said you were satisfied with the tool. We guess the 10% who would still recommend it but aren't satisfied still think the price is right... :) Thanks to everyone for your input and comments - it goes directly to our product team!

And the winner is... Congrats to Larry from Southborough, and Deepti from Stamford, the respective winners in the prize drawing for the iPod Nano and the Amazon gift cards.

New SOA Testing Survey from Aberdeen Group

We've been looking for some research to benchmark the maturity of SOA Testing efforts in isolation of SOA Governance as a whole. So when I heard Perry Donham was doing this over at Aberdeen, we wanted to participate in underwriting the survey and asking our users to contribute. A free copy of the report is available here: "SOA and Web Services Testing: How Different Can It Be?" Of particular interest was less emphasis on component-level testing, and from 63% (average) to 81% (best in class) finding complete lifecycle testing to be a key to success - and these firms are overwhelmingly measuring quality by lifecycle metrics (time to release functionality, process quality) instead of counting the number of bugs per KLOC. One area of concern was in the level of SOA training or expertise within organizations -- averaging 35% right now. A shortage of good architectural talent is certainly evident, but fortunately a majority of the responding companies are planning to address that over the coming year. Anyway, it's fresh research, grab your own copy and join the discussion. Thanks to all of the LISA enterprise and WS-Testing users who took the time to participate in the sample group for Aberdeen. - Jason