Evaluating Tools for Developing with SOAP in Python

Evaluating Tools for Developing with SOAP in Python

Originally published in Python Magazine Volume 3 Issue 9 , September,
2009

Greg Jednaszewski was my co-author for this article.

In order to better meet the needs of partners, Racemi needed to
build a private web service to facilitate tighter integration
between our applications and theirs. After researching the state of
SOAP development in Python, we were able to find a set of tools
that met our needs quite well. In this article, we will describe
the criteria we used to evaluate the available tools and the
process we followed to decide which library was right for us.

Racemi’s product, DynaCenter, is a server provisioning and data center
management software suite focusing on large private installations
where automation is key for our end-users. Because we are a small
company, our business model is organized around partnering with larger
companies in the same industry and acting as an OEM. Those partners
typically provide their own user interface, and drive DynaCenter’s
capture and provision services through our API.

Many of our partners’ automation and workflow management systems are
designed to call scripts or external programs, so the first version of
our API was implemented as a series of command line programs.
However, we are increasingly seeing a desire for more seamless
integration through web service APIs. Since most of our partners are
Java shops, in their minds the term web service is synonymous with
SOAP (Simple Object Access Protocol), an HTTP and XML-based protocol
for communicating between applications. Since Python’s standard
library does not include support for SOAP, we knew we would need to
research third-party libraries to find one suitable for creating a web
service interface to DynaCenter. Our first step was to develop a set
of minimum requirements.

Basic Requirements

DynaCenter is designed with several discrete layers that communicate
with each other as needed. The command line programs that comprise the
existing OEM API communicate with internal services running in daemons
on a central control server or on the managed systems. This layered
approach separates the exposed interface from the implementation
details, allowing us to change the implementation but maintain a
consistent API for use by partners. All of the real work for capturing
and provisioning server images is implemented inside the DynaCenter
core engine, which is invoked by the existing command line programs.
The first requirement we established was that the new web service
layer had to be thin so we could reuse as much existing code as
possible, and avoid re-implementing any of the core engine
specifically for the web service.

This project was unique in that many of the features of full-stack web
frameworks would not be useful to meeting our short-term requirements.
We have our own ORM for accessing the DynaCenter’s database, so any
potential solution needed to be able to operate without a fully-
configured ORM component. In addition, we were not building a human
interface, so full-featured templating languages and integration with
Javascript toolkits were largely irrelevant to the project. On the
other hand, while we recognized that SOAP was a short-term requirement
from some of our partners, we did anticipate wanting to support other
protocols like JSON in the future without having to write a new
service completely from scratch.

We also knew that creating a polished product would require
comprehensive documentation.

We also knew that creating a polished product would require
comprehensive documentation. The WSDL (Web Service Definition
Language) file for the SOAP API, which is a formal machine-readable
declaration of what calls and data types an API supports, would be
helpful, but only as a reference. We planned to document the entire
API in a reference manual as well as with sample Java and Python code
bundled in a software development kit (SDK). We could write that
documentation manually, but integration with the documentation tools
was considered a bonus feature.

Finally, we needed support for complex data structures. Our data model
uses a fairly sophisticated representation of image meta-data,
including networking and storage requirements. DynaCenter also
maintains data about the peripherals in a server so that we can
reconfigure the contents of images as they are deployed to run under
new hardware configurations. This information is used as parameters
and return values throughout the API, so we needed to ensure that the
tool we chose would support data types beyond the simple built-ins
like strings and integers.

Meet the Candidates

Through our research, we were able to identify three viable candidate
solutions for building SOAP-based web services in Python.

The Zolera Soap Infrastucture (ZSI), is a part of the pywebsvcs
project. It provides complete server and client libraries for working
with SOAP. To use it, a developer writes the WSDL file (by hand or
using a WSDL editor), and then generates Python source for the client
and stubs for the server. The data structures defined in the WSDL file
are converted into Python classes that can be used in both client and
server code.

soaplib is a lightweight library from Optio Software. It also
supports only SOAP, but in contrast to ZSI it works by generating the
WSDL for your service based on your Python source code. soaplib is not
a full-stack solution, so it needs to be coupled with another
framework such as TurboGears or Pylons to create a service.

TGWebServices (TGWS) is a TurboGears-specific library written by
Kevin Dangoor and maintained by Christophe de Vienne. It provides a
special controller base class to act as the root of the service. It is
similar to soaplib in that it generates the WSDL for a service from
the source at runtime. In fact, we found a reference to the idea of
merging soaplib and TGWebServices, but that work seems to have stalled
out. One difference between the libraries is that TGWS also supports
JSON and “raw” XML messages for the same back-end code.

Now that we had the basic requirements identified and a few candidates
to test, we were able to create a list of evaluation criteria to help
us make our decision.

Installing

A primary concern was whether or not a tool could be installed and
made to work at all using any tutorial or guide from the
documentation. We used a clean virtualenv for each application and
used Python 2.6.2 for all tests. Initial evaluations were made under
Mac OS X 10.5 and eventually prototype servers were set up under
CentOS 4 so the rest of Racemi’s libraries could be used and the
service could work with real data.

The latest official release of ZSI (2.0-rc3) installed using
easy_install, including all dependencies and C extensions. A newer
alpha release (2.1-a1) also installed correctly from a source archive
we downloaded manually. The sample code provided with the source
archive had us up and running a test server in a short time.

We were less successful using easy_install with TGWS because we
did not start out with TurboGears installed and the dependencies were
not configured to bring it in automatically. After modifying the
dependencies in the package by hand, we were able to install it and
configure a test server following the documentation. Once we overcame
that problem, we found that the official distribution of TGWS is only
compatible with TurboGears 1.0. By asking on the support mailing list,
we found patches to make it compatible with TurboGears 1.1 and were
then able to bring up a test server. Since TurboGears 2.x has moved
away from CherryPy, and TGWS uses features of CherryPy, we did not try
to use TurboGears 2.x.

We never did get soaplib to install. It depends on lxml, and
installation on both of our our test platforms failed with compilation
and link errors. At this point, soaplib was moved off of the list of
primary candidates. We kept it open as an option in case the other
tools did not pan out, but not being able to install it hurt our
ability to evaluate it completely.

Feature Completeness

Since we anticipated other web-related work, we also considered the
completeness of the stack. Although ZSI provides a full SOAP server,
it does not easily support other protocols. Since our only hard
requirement for protocols in the first version of the service was
SOAP, this limitation did not rule ZSI out immediately.

Because TGWS sits on top of TurboGears, we knew that if we eventually
wanted to create a UI for the service we could use the same stack. It
also supports JSON out of the box, so third-party JavaScript
developers could create their own UI as well.

Interoperability

Another concern was whether the tool would be inter-operable with a
wide variety of clients. We were especially interested in the Java
applications we expected our partners to be writing. Since we are
primarily a Python shop, we also wanted to be able to test the SOAP
API using Python libraries. In order to verify that both sets of
clients would work without issue, we constructed prototype servers
using each tool and tested them using SOAP clients in Python and Java
(using the Axis libraries).

Both ZSI and TGWS passed the compatibility tests we ran using both
client libraries. The only interoperability issue we came across was
with the SOAP faults generated by TGWS, which did not pass through the
strict XML parser used by the Java Axis libraries. We were able to
overcome this with a few modifications to TGWS (which we have
published for possible inclusion in a future version of TGWS).

Freshness

Our investigations showed that there had not been much recent
development of SOAP libraries in Python, even from the top contenders
we were evaluating. It wasn’t clear whether this was because the
existing tools were stable and declared complete, or if the Python
community has largely moved on to other protocols like JSON. To get a
sense of the “freshness” of each project, we looked for the last
commit to the source repository and also examined mailing list
archives for recent activity. We were especially interested in
responses from developers to requests for support.

The recent activity on the ZSI forums on Sourceforge seemed mostly to
be requests for help. The alpha release we used for one of the tests
was posted to the project site in November of 2007. There had been
more recent activity in the source tree, but we did not want to use an
unreleased package if we could avoid it.

The situation with TGWS was confusing at first because we found
several old sites. By following the chain of links from the oldest to
the newest, we found the most recent code in a BitBucket repository
being maintained by Christophe de Vienne. As mentioned earlier, the
project mailing list was responsive to questions about making TGWS
work with TurboGears 1.1, and pointed us towards a separate set of
patches that were not yet incorporated in the official release.

Documentation

As new users, we wanted to find good documentation for any tool we
selected. Having the source is useful for understanding how you’re
doing something wrong, but learning what to do in the first place
calls for separate instructions. All of the candidates provided enough
documentation for us to create a simple prototype server without too
much trouble.

Just as we expect to have documentation for third-party tools we use,
we need to provide API references and tutorials for the users of our
web service. We use Sphinx for all customer-facing documentation at
Racemi, since it allows us to manage the documentation source along
with our application code, and to build HTML and PDF versions of all
of our manuals. TGWS includes a Sphinx extension that adds directives
for generating documentation for web service controllers, so we could
integrate it with our existing build process easily. ZSI has no native
documentation features. We did consider building something to parse
the WSDL file and generate API docs from that, but the existing Sphinx
integration TGWS provided was a big bonus in our eyes.

Deployment Complexity

We evaluated the options for deploying all of the tools, including how
much the deployment could be automated and how flexible they were. We
decided to run our service behind an Apache proxy so we could encrypt
the traffic with SSL. All of the tools support the standard options
for doing this (mod_proxy, mod_python, and in some cases mod_wsdl)
so there was no clear winner for this criteria.

In addition to simple production deployment, we also needed an option
for running a server in “development” mode without requiring root
access or modifications to a bunch of system services. We found that
both ZSI and TGWS have good development server configurations, and
could be run directly out of a project source tree (in fact, that is
how the prototype servers were tested).

Packaging Complexity

As a packaged OEM product, DynaCenter is a small piece of a larger
software suite being deployed on servers outside of our control. It
needs to play well with others and be easy to install in the field.
Most installations are performed by trained integrators, but they are
not Python programmers and we don’t necessarily want to make them deal
with a lot of our implementation details. We definitely do not want
them downloading dependencies from the Internet, so we package our own
copy of Python and the libraries we use so that installation is
simpler and avoids version conflicts.

ZSI’s only external dependency are PyXML and zope.interface. We
were already packaging PyXML for other reasons, and zope.interface
was easy to add. TGWS depends on TurboGears, which is a collection of
many separate packages. This made re-distribution less convenient,
since we had to grab the sources for each component separately.
Fortunately, the complete list is documented clearly in the
installation script for TurboGears and we were able to distill it down
to the few essential pieces we would actually be using. Those packages
were then integrated with our existing processes so they could be
included in the Python package we build.

Licensing

Although Racemi does contribute to open source tools when possible,
DynaCenter is not itself open source. We therefore had to eliminate
from consideration any tool that required the use of a GNU Public
License. ZSI uses a BSD-like license, which matched our requirements.
The zope.interface package is licensed under the Zope Public
License, which is also BSD-like. TGWS and most of the TurboGears
components are licensed under a BSD or MIT license. The only component
that even mentioned GNU was SQLObject, which uses the LGPL. That would
have been acceptable, but since we have our own ORM and do not need
SQLObject, we decided to skip including it in our package entirely to
avoid any question.

Elegance

SOAP toolkits tend to fall in one of two camps: Those that generate
source from a WSDL file and those that generate a WSDL document from
source. We didn’t particularly care which solution we ended up with,
as long as we didn’t have to write both the WSDL and the source code.
We also wanted to avoid writing vast amounts of boilerplate code, if
possible. As you will see from the examples below, the tools that
generated the WSDL from Python source turned out to be a much more
elegant in the long run.

We also considered the helpfulness of the error messages as part of
evaluating the elegance and usability of the tools. With TGWS, most of
what we were writing was Python. Many of the initial errors we saw
were from the interpreter, and so the error types and descriptions
were familiar. Once those were eliminated, the errors we saw generated
by TGWS code were usually direct and clear, although they did not
always point at the parts of our source code where the problem could
be fixed.

In contrast, we found ZSI’s errors to be very obscure. It seemed many
were caused by a failure of the library to trap problems in the
underlying code, such as indexing into a None value as a tuple.
Even the errors that were generated explicitly by the ZSI code left us
scratching our heads on occasion. We continued evaluating both tools,
but by this time we were leaning towards TGWS and growing more
frustrated with ZSI.

Testing

Automated testing is especially important for a complex product like
DynaCenter, so being able to write tests for the new web service and
integrate them with our existing test suite was an important feature.
ZSI does not preclude writing automated tests, but does not come with
any obvious framework or features for supporting them, so we would
need to roll our own. TGWS takes advantage of TurboGears’ integration
with WebTest to let the developer write unit and integration tests in
Python without even needing to start a test daemon.

Performance

Once we established the ease of creating and testing services with
TGWS, we had basically made our choice for that library. However,
there was one last criteria to check: performance. Using the prototype
servers we had set up for experimenting with the tools, we took some
basic timing measurements by writing a SOAP client in Python to invoke
a service that returned a large data set (500 copies of a complex type
with several properties of different types). We measured the time it
took for the client to ask for the data and then parse it into usable
objects.

The data structure definition was the same for both services, and we
found no significant difference in the performance of the two SOAP
implementations. Interestingly, as the amount of data increased, the
JSON performance reached a 10x improvement over SOAP. Our hypothesis
for the performance difference is that there was less data to parse,
the parser was more efficient, and the objects being created in the
client were simpler because JSON does not try to instantiate user-
defined classes.

Prototyping with ZSI

We were somewhat familiar with ZSI because we had used it in the past
for building a client for interacting with the VMware Virtual Center
web service, so we started with ZSI as our first prototype. For both
prototypes, we implemented a simple echo service that returns as
output whatever it gets as input from the client. Listing 1 contains
the hand-crafted WSDL inputs for the ZSI version of this service.

Listing 1

<?xml version="1.0" encoding="UTF-8"?>
<definitions
  xmlns="http://schemas.xmlsoap.org/wsdl/"
  xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
  xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/"
  xmlns:http="http://schemas.xmlsoap.org/wsdl/http/"
  xmlns:xsd="http://www.w3.org/2001/XMLSchema"
  xmlns:tns="urn:ZSI"
  targetNamespace="urn:ZSI" >

  <types>
    <xsd:schema elementFormDefault="qualified"
        targetNamespace="urn:ZSI">
      <xsd:element name="Echo">
        <xsd:complexType>
          <xsd:sequence>
            <xsd:element name="value" type="xsd:anyType"/>
          </xsd:sequence>
        </xsd:complexType>
      </xsd:element>
    </xsd:schema>
  </types>

  <message name="EchoRequest">
    <part name="parameters" element="tns:Echo" />
  </message>
  <message name="EchoResponse">
    <part name="parameters" element="tns:Echo"/>
  </message>

  <portType name="EchoServer">
    <operation name="Echo">
      <input message="tns:EchoRequest"/>
      <output message="tns:EchoResponse"/>
    </operation>
  </portType>

  <binding name="EchoServer" type="tns:EchoServer">
    <soap:binding style="document"
                  transport="http://schemas.xmlsoap.org/soap/http"/>
    <operation name="Echo">
      <soap:operation soapAction="Echo"/>
      <input>
        <soap:body use="literal"/>
      </input>
      <output>
        <soap:body use="literal"/>
      </output>
    </operation>
  </binding>

  <service name="EchoServer">
    <port name="EchoServer" binding="tns:EchoServer">
      <soap:address location="http://localhost:7000"/>
    </port>
  </service>

</definitions>

To generate the client and server code from the WSDL, feed it into the
wsdl2py program (included with ZSI). To add support for complex
types, add the -b option, but it isn’t required for this simple
example. wsdl2py will, in response, produce three files:

Listing 2

EchoServer_client.py is the code needed to build a client for the
SimpleEcho web service.

##################################################
# file: EchoServer_client.py
#
# client stubs generated by
# "ZSI.generate.wsdl2python.WriteServiceModule"
#
##################################################

from EchoServer_types import *
import urlparse, types
from ZSI.TCcompound import ComplexType, Struct
from ZSI import client
from ZSI.schema import GED, GTD
import ZSI
from ZSI.generate.pyclass import pyclass_type

# Locator
class EchoServerLocator:
    EchoServer_address = "http://localhost:7000"
    def getEchoServerAddress(self):
        return EchoServerLocator.EchoServer_address
    def getEchoServer(self, url=None, **kw):
        return EchoServerSOAP(
            url or EchoServerLocator.EchoServer_address,
            **kw)

# Methods
class EchoServerSOAP:
    def __init__(self, url, **kw):
        kw.setdefault("readerclass", None)
        kw.setdefault("writerclass", None)
        # no resource properties
        self.binding = client.Binding(url=url, **kw)
        # no ws-addressing

    # op: Echo
    def Echo(self, request, **kw):
        if isinstance(request, EchoRequest) is False:
            raise TypeError, "%s incorrect request type" % 
                (request.__class__)
        # no input wsaction
        self.binding.Send(None, None, request, soapaction="Echo", **kw)
        # no output wsaction
        response = self.binding.Receive(EchoResponse.typecode)
        return response

EchoRequest = GED("urn:ZSI", "Echo").pyclass

EchoResponse = GED("urn:ZSI", "Echo").pyclass

Listing 3

EchoServer_server.py contains code needed to build the
SimpleEcho web service server.

##################################################
# file: EchoServer_server.py
#
# skeleton generated by
#  "ZSI.generate.wsdl2dispatch.ServiceModuleWriter"
#
##################################################

from ZSI.schema import GED, GTD
from ZSI.TCcompound import ComplexType, Struct
from EchoServer_types import *
from ZSI.ServiceContainer import ServiceSOAPBinding

# Messages
EchoRequest = GED("urn:ZSI", "Echo").pyclass

EchoResponse = GED("urn:ZSI", "Echo").pyclass


# Service Skeletons
class EchoServer(ServiceSOAPBinding):
    soapAction = {}
    root = {}

    def __init__(self, post='', **kw):
        ServiceSOAPBinding.__init__(self, post)

    def soap_Echo(self, ps, **kw):
        request = ps.Parse(EchoRequest.typecode)
        return request,EchoResponse()

    soapAction['Echo'] = 'soap_Echo'
    root[(EchoRequest.typecode.nspname,EchoRequest.typecode.pname)] = 
        'soap_Echo'

Listing 4

EchoServer_types.py has type definitions used by both the client
and server code.

##################################################
# file: EchoServer_types.py
#
# schema types generated by
#  "ZSI.generate.wsdl2python.WriteServiceModule"
#
##################################################

import ZSI
import ZSI.TCcompound
from ZSI.schema import (LocalElementDeclaration, ElementDeclaration,
                        TypeDefinition, GTD, GED)
from ZSI.generate.pyclass import pyclass_type

##############################
# targetNamespace
# urn:ZSI
##############################

class ns0:
    targetNamespace = "urn:ZSI"

    class Echo_Dec(ZSI.TCcompound.ComplexType, ElementDeclaration):
        literal = "Echo"
        schema = "urn:ZSI"
        def __init__(self, **kw):
            ns = ns0.Echo_Dec.schema
            TClist = [ZSI.TC.AnyType(pname=(ns,"value"),
                      aname="_value", minOccurs=1, maxOccurs=1,
                      nillable=False, typed=False,
                      encoded=kw.get("encoded"))]
            kw["pname"] = ("urn:ZSI","Echo")
            kw["aname"] = "_Echo"
            self.attribute_typecode_dict = {}
            ZSI.TCcompound.ComplexType.__init__(self,None,TClist,
                                                inorder=0,**kw)
            class Holder:
                __metaclass__ = pyclass_type
                typecode = self
                def __init__(self):
                    # pyclass
                    self._value = None
                    return
            Holder.__name__ = "Echo_Holder"
            self.pyclass = Holder

# end class ns0 (tns: urn:ZSI)

Once generated, these files are not meant to be edited, because they
will be regenerated as part of a build process whenever the WSDL input
changes. The code in the files grows as more types and calls are added
to the service definition.

The implementation of the server goes in a separate file that imports
the generated code. In the example, the actual service is on lines
18–25 of Listing 5. The @soapmethod decorator defines the input
(an EchoRequest) and the output (an EchoResponse) for the call.
In the example, the implementation of soap_Echo() just fills in
the response value with the request value, and returns both the
request and the response. From there, ZSI takes care of building the
SOAP response and sending it back to the client.

Listing 5

import os
import sys
from EchoServer_client import *
from ZSI.twisted.wsgi import (SOAPApplication,
                              soapmethod,
                              SOAPHandlerChainFactory)

class EchoService(SOAPApplication):
    factory = SOAPHandlerChainFactory
    wsdl_content = dict(name='Echo',
                        targetNamespace='urn:echo',
                        imports=(),
                        portType='',
                        )

    def __call__(self, env, start_response):
        self.env = env
        return SOAPApplication.__call__(self, env, start_response)

    @soapmethod(EchoRequest.typecode,
                EchoResponse.typecode,
                operation='Echo',
                soapaction='Echo')
    def soap_Echo(self, request, response, **kw):
        # Just return what was sent
        response.Value = request.Value
        return request, response

def main():
    from wsgiref.simple_server import make_server
    from ZSI.twisted.wsgi import WSGIApplication

    application         = WSGIApplication()
    httpd               = make_server('', 7000, application)
    application['echo'] = EchoService()
    print "listening..."
    httpd.serve_forever()

if __name__ == '__main__':
    main()

Listing 6 includes a sample of how to use the ZSI client libraries to
access the servers from the client end. All that needs to be done is
to create a handle to the EchoServer web service, build an
EchoRequest, send it off to the web service, and read the
response.

Listing 6

from EchoServer_client import *
import sys, time

loc  = EchoServerLocator()
port = loc.getEchoServer(url='http://localhost:7000/echo')

print "Echo: ",
msg = EchoRequest()
msg.Value = "Is there an echo in here?"
rsp = port.Echo(msg)
print rsp.Value

Prototyping with TGWebServices

To get started with TGWebServices, first create a TurboGears project
by running tg-admin quickstart which will prompt you to name the
new project and Python package, and then produce a directory structure
full of skeleton code. The directory names are based on the project
and package names chosen when running tg-admin. The top-level
directory contains sample configuration files and a script for
starting the server, and a subdirectory containing all the Python code
for the web service.

tg-admin will generate several Python files, but the important
file for defining the web service is controllers.py. Listing 7
shows the controllers.py file for our prototype echo server. The
@wsexpose decorator on line 7 exposes the web service call and
defines the return type as a string. On line 8, @wsvalidate
defines the data types for each parameter. As with the ZSI example,
the actual implementation of the echo call just returns what is passed
in.

Listing 7

from turbogears import controllers, expose, flash
from tgwebservices.controllers import WebServicesRoot, wsexpose, wsvalidate

class EchoService(WebServicesRoot):
    """EchoService web service definition"""

    @wsexpose(str)
    @wsvalidate(value=str)
    def echo(self, value):
        "Echo the input back to the caller."
        return value

class Root(controllers.RootController):
    """The root controller of the application."""

    echo = EchoService('http://localhost:7000/echo/')

The auto-generated WSDL for the web service is accessible via
http://<server>/echo/soap/api.wsdl. Listing 8 shows an example of
the WSDL generated by TGWS for the prototype EchoService. It includes
definitions of all types used in the API (lines 3–20), the request and
response message wrappers for each call (lines 21–26), as well as the
ports (lines 27–45) and a service definition (lines 46–51) pointing to
the server generating the WSDL document. Each port includes the
docstring from the method implementing it (line 29).

Listing 8

<?xml version="1.0" encoding="UTF-8"?>
<wsdl:definitions xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/" xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" name="EchoService" xmlns:types="http://localhost:7000/echo/soap/types" xmlns:soapenc="http://www.w3.org/2001/09/soap-encoding" targetNamespace="http://localhost:7000/echo/soap/" xmlns:tns="http://localhost:7000/echo/soap/">
   <wsdl:types>
     <xsd:schema elementFormDefault="qualified" targetNamespace="http://localhost:7000/echo/soap/types">
          <xsd:element name="echo">
            <xsd:complexType>
              <xsd:sequence>
                <xsd:element name="value" type="xsd:string"/>
              </xsd:sequence>
            </xsd:complexType>
          </xsd:element>
          <xsd:element name="echoResponse">
            <xsd:complexType>
              <xsd:sequence>
                <xsd:element name="result" type="xsd:string"/>
              </xsd:sequence>
            </xsd:complexType>
          </xsd:element>
      </xsd:schema>
   </wsdl:types>
   <wsdl:message name="echoRequest" xmlns="http://localhost:7000/echo/soap/types">
       <wsdl:part name="parameters" element="types:echo"/>
   </wsdl:message>
   <wsdl:message name="echoResponse" xmlns="http://localhost:7000/echo/soap/types">
      <wsdl:part name="parameters" element="types:echoResponse"/>
   </wsdl:message>
   <wsdl:portType name="EchoService_PortType">
      <wsdl:operation name="echo">
        <wsdl:documentation>Echo the input back to the caller.</wsdl:documentation>
         <wsdl:input message="tns:echoRequest"/>
         <wsdl:output message="tns:echoResponse"/>
      </wsdl:operation>
   </wsdl:portType>
   <wsdl:binding name="EchoService_Binding" type="tns:EchoService_PortType">
      <soap:binding style="document" transport="http://schemas.xmlsoap.org/soap/http"/>
      <wsdl:operation name="echo">
         <soap:operation soapAction="echo"/>
         <wsdl:input>
            <soap:body use="literal"/>
         </wsdl:input>
         <wsdl:output>
            <soap:body use="literal"/>
         </wsdl:output>
      </wsdl:operation>
   </wsdl:binding>
   <wsdl:service name="EchoService">
      <wsdl:documentation>WSDL File for EchoService</wsdl:documentation>
      <wsdl:port binding="tns:EchoService_Binding" name="EchoService_PortType">
         <soap:address location="http://localhost:7000/echo/soap/"/>
      </wsdl:port>
   </wsdl:service>
</wsdl:definitions>

The tgwsdoc extension to Sphinx, distributed with TGWS, adds
several auto-documentation directives to make it easy to keep your
documentation in sync with your code. By using autotgwstype,
autotgwscontroller, and autotgwsfunction, you can insert
definitions of the complex types, controllers, or individual API calls
in with the rest of your hand-written documentation. This was
especially useful for us because we already had a lot of text
explaining our existing command line interface. We were able to reuse
a lot of the material and document all three interfaces (command line,
SOAP, and JSON) with a single tool.

Implementation Considerations

Once we had chosen TGWS as our framework, we set about working on the
first implementation of our real service. This helped us uncover a few
small problems with our original “pure” design, and some details we
had not considered while prototyping.

For example, we wanted to make sure that our web service was not only
interoperable with Java clients, but also that the API made sense to a
Java developer. One tool that they might be using, the Java Axis
client, is built by feeding the WSDL file into a code generator to
produce source code for client classes. After we tried working with
the generated Java code, we adjusted our web service API to make it
more usable. For instance, Java doesn’t allow you to specify defaults
for method arguments, which caused problems with a couple of web
service calls that had a handful of required arguments along with many
optional keyword arguments. On the Java side, the caller would have to
pass in all 23 parameters to the call, most of them null placeholders
for the optional parameters. To address that, we moved all the
optional parameters to a separate “options” object that could be
populated and passed in for advanced operations.

There were other minor annoyances, such as the way a camelCase
naming convention resulted in nicer-looking Java code than the
under_scored naming convention typically used by Python
programmers. We ended up going with camelCase names for attributes
and methods of classes used in the public side of the web service.
After making these tweaks, it is not difficult to design an API with
TGWS that makes sense to both Java and Python client developers.

Testing in Java was another challenge for us to work out. We have a
large suite of Python tests driven by nose, and we ultimately were
able to automate the client-side Java testing using junit. We then
integrated the two suites by writing a single Python test to run all
of the junit tests in a separate process and parse the results
from the output.

In addition to developer tests, Racemi has a dedicated group of test
engineers who perform QA and acceptance tests before each new version
of DynaCenter is released. The QA team needed a client library to use
for testing the new web service. None of them are Java programmers, so
the Dev team took on the task of basic Java integration testing. But
for full-on regression testing and automation, QA needed something
lightweight and easy to get up and running with quickly. Suds fit
this bill quite nicely. It is a client-only SOAP interface for Python
that reads the WSDL file at runtime and provides client bindings for
the web service API. Armed with our WSDL and the Suds documentation,
our QA team was able to start building a client test harness almost
immediately.

Conclusions

At the beginning of our evaluation process, we knew there were a lot
of ways to compare the available tools. At first, we weren’t sure if
the code-from-WSDL model used by ZSI or the WSDL-from-code model used
by TGWebServices and soaplib would be easier to use. After creating
the simple echo service prototype with both tools, we found that
writing Python and generating the WSDL worked much better for us.
Because WSDL is an XML format primarily concerned with types, we found
it excessively verbose compared to the Python needed to back it up.
It felt much more natural to express our API with Python code and then
generate the description of it. Starting with the code also lead to
fewer situations where translating to WSDL produced errors, unlike
when we tried to manage the WSDL by hand.

Because WSDL is an XML format primarily concerned with types, we found
it excessively verbose compared to the Python needed to back it up.

As mentioned earlier, we ended up needing to patch TGWebServices to
make it work correctly with TurboGears 1.1. Those patches were
available on the Internet as separate downloads, but we decided to
“fork” the original Mercurial repository and create a new version
that included them directly. We have also added a few other
enhancements, such as the option of specifying which formats (JSON
and/or XML) to use when documenting sample types, and better SOAP
error message handling. We are working with Christophe de Vienne to
move those changes upstream.

TGWebServices stood out as the clear winner for our needs.

Aside from the ease of use benefits and technical merits of
TGWebServices, there were several bonus features that made it
appealing. The integration with Sphinx for generating documentation
meant that not only would we not have to write the reference guide as
a completely separate task, but it would never grow stale as code
(especially data structures) changed during the evolution of the
API. Getting the JSON for “free” was another big win for us because it
made testing easier and did not lock us in to a SOAP solution for all
of our partners. Couple that with the benefit of having the TurboGears
framework already in place for a possible web UI down the road, and
TGWebServices stood out as the clear winner for our needs.