Interop Workshop Guidelines (DRAFT)

The following are guidelines for conducting interop workshops for emergency management systems.

Why do we need interop workshops for crisis management systems?

The information systems space in now rich with systems that support the crisis response domain from tools that provide organization specific emergency management capabilities all the way to public crowd sourcing tools. Each tool is valuable and caters to a different use cases and associated target users. However due to the rich diversity of tools, there is an amplification of data isolation, unnecessary re-entry and redundancy. Effective information sharing mechanisms need to be found to reduce this data isolation that brings inefficiencies to disaster response. The first step is the agreement on a specific interop standard to share information and many such standards exist now for crisis response such as PFIF, EDXL family of standards, OGC GIS Standards, etc. However more often than not the standard specifications have room for ambiguity/creativity/growth, that can result in incompatibilities in how they are implemented and result in unexpected incompatibilities in response. This is where interop worhshops help. An interop worshop is an environment where software vendors and developers can bring in their systems to test for compliance to a agreed interop standard, such as PFIF, EDXL-HAVE, CAP. In addition interop workshops can serve as valuable opportunities for authors of new interop standards or version to get the feedback they need for refinement of their interop specification. The whole process also thus results in the needed buy in from the participants (most importantly the product vendors) that will help enable the new interop specification become a pervasive standard.

Such actions and results pre-disaster can greatly help not only assure systems can exchange information, but also build the needed partnerships to facilitate data exchange between solution providers in the domain.

How are interop workshops run?

The author or a member of the specification creating group presents the new or existing standard and the different scenarios/scope it is supposed to address. From this basis a series of test cases are derived that attempt to incrementally cover the specification progressively from the most basic test to more advanced scenarios. Usually the test cases are associated with a sequence diagram of expected communication between a client and server or two peers. One also needs to validate that the data exchange has happened correctly.

Overview

Goals

  • Apply standards software testing practices to web services specifications (using test cases)
  • Find and fix problems in new specifications before they are widely implemented
  • Gather feedback on specification and usability problems with specification
  • Prove Interoperability with multiple software products
  • Expert support for software product teams to implement new specification

Output

  • Validated implementations of specifications (implementation matrix of products vs spec test cases)
  • Community Feedback Report
  • Expanded Specification Awareness

Approach

Preparation

  • New Spec is sent out by author, highlighting new areas
  • Test cases are documented and sent out to participants (ideally on a WIKI)

Workshop

  • New Spec and Test Cases is presented by Author or representative
  • Any feedback is discussed with author on new specifications and alterations are made to test cases as needed
  • Interop matrix is placed on a WIKI or a Whiteboard (product vs Test cases).
  • As test cases are proved to work and validated they are updated on the matrix
  • First product to finish all test cases wins the prize! :-)

Acceptance of compatibility with specification

  • Coverage - Have all the main features of the specification been tested fully?
  • Data Exchange - Has data correctly been transferred to the target systems
  • QoS - Have acceptable minimum criteria on the Quality of Service been met

Standards to be tested

Communication Problem / Use Cases Protocol(s) Next Workshop
Missing Person Data Exchange Use Cases PFIF ISCRAM 2011
Hospital Data Exchange Use Cases EDXL-HAVE ISCRAM 2011
Alerting Data Exchange Use Cases CAP TBD

Example interop session: http://msdn.microsoft.com/en-us/library/ms996532.aspx

Tips

  • It is best to have the Emergency Managers or representatives from the response agencies define the scenarios they face getting systems to have to talk to each other for better efficiency. They will also help us prioritize the scenarios based on impact
  • The interop workshops for this domain also needs to include test cases that handle a great degree of fault tolerance as there is an expectation that internet bandwidth will either be slow and will get intermittently disconnected in the middle of a transaction.

From Jorgen Thelin (Microsoft)

  • It is sometimes useful to plan an optional “kick-off event” (conference call / webcast / etc) a few weeks before the workshop itself to introduce the scenarios to people so they can start thinking about them and preparing their test code before the event. It often depends on the complexity of the scenarios whether this is required, but I found it particularly useful for many of the security testing workshops I ran back then.
  • It can be really helpful if at least one product org can provide a public endpoint for some of the basic test cases before the event, as that can help participants work through some of the baseline Interop problems and usually means testing during the event is more productive.
  • If you can’t provide a pre-test endpoint, then some real sample XML message will be very useful for participants. You can only imagine how much fun some people had at some of the previous workshops just sorting our basic namespace/ XML parsing issues!

References

 
/home6/gyttjaor/public_html/humanitarianict/wiki/data/pages/interop_workshop_guidelines.txt · Last modified: 2011/05/03 13:49 by chamindra
 
Except where otherwise noted, content on this wiki is licensed under the following license:CC Attribution-Noncommercial-Share Alike 3.0 Unported
Recent changes RSS feed Donate Powered by PHP Valid XHTML 1.0 Valid CSS Driven by DokuWiki