ODS Grizzly: Choosing a WSGI Framework for API Services

While building the first version of the API server for Ceilometer, I chose to use Flask instead of the WSGI framework in openstack-common. During this session, I will review the evaluation process that led me to that decision and present some notes on other WSGI frameworks and how they may, or may not, be suitable for use when building new API servers for OpenStack projects.

Session Goal

Start a discussion for changes that might happen during Grizzly & develop a list of requirements.

Background Reference Material

• http://plope.com/pyroptimization
• http://svn.repoze.org/whatsitdoing/
• http://packages.python.org/WSME/

Topics

• What’s wrong with what we have?

  • reproduces features of other frameworks and tools
  • XML & JSON serialization and deserialization
  • routing
  • server launching and management
  • verbose, with lots of boilerplate
  • seems like a lot of moving parts (TemplateBuilder, MetadataXMLDeserializer, CommonDeserializer, Controller, ViewBuilder)
  • tightly coupled to our notion of services and database management
  • relies on Paste, which has Python 3 issues
  • one more thing for us to maintain ourselves

• What do we do if we decide to change?

  • don’t re-write existing API versions
  • use for new API versions only
  • revisit API designs for cross-project consistency

• Other frameworks to consider

  • Django – http://www.djangoproject.com/
    • Pros
      • popular
      • excellent documentation
      • many plugins available
      • already being used in Horizon
    • Cons
      • not necessarily best suited for API work
    • TurboGears – http://www.turbogears.org/
      • Pros
        • I’ve had good success with using it for APIs in the past
        • object dispatch makes URL routing pretty obvious
      • Cons
        • long dependency list makes packaging complex
        • python 3 support questionable
        • project support questionable
    • Pyramid – http://www.pylonsproject.org/
      • Pros
        • robust community
        • REST support
        • documentation
      • Cons
        • long dependency list makes packaging complex
    • Flask – http://flask.pocoo.org
      • Pros
        • very simple routing declarations
        • simple functions for URL handlers
        • blueprints allow mounting multiple versions easily
        • nice documentation support for API endpoints using Sphinx
      • Cons
        • db transactions a little trickier to manage, but possible
        • JSON built-in but not XML
        • some issues, not updated in a while
    • Pecan – http://pecan.readthedocs.org
      • Pros
        • object dispatch, somewhat like the routing we have now
        • more features than Flask but simpler than Pyramid
        • REST support built in with special controller type
        • created in part by developers now at DreamHost
      • Cons
        • small community
    • Bottle – http://bottlepy.org/
      • Pros
        • small enough to make embedding easy
        • already integrated with WSME
      • Cons
        • Docs are less clear than flask
    • Twisted – http://twistedmatrix.com
      • Pros
        • async performance?
      • Cons
        • been there, done that
        • async development style
    • Tornado – http://www.tornadoweb.org/
      • Pros
        • async performance?
      • Cons
        • async development style
    • WSME (Web Services Made Easy) – http://packages.python.org/WSME/
      • Pros
        • separates model from view/serializer cleanly and transparently
        • declarative style for defining model classes used for input and return types
        • supports JSON and XML (and SOAP)
        • framework agnostic, more or less
        • integrates with Sphinx for documentation generation (endpoints, arguments, and return types)
        • Author (Christophe de Vienne) accepts patches willingly
      • Cons
        • mostly appears to like RPC-style calls, but adaptable
    • spyne – https://github.com/arskom/spyne
      • “Spyne aims to save the protocol implementers the hassle of implementing their own remote procedure call API and the application programmers the hassle of jumping through hoops just to expose their services using multiple protocols and transports. From another perspective, Spyne is a framework for building distributed solutions that strictly follow the MVC pattern, where Model = spyne.model, View = spyne.protocol and Controller = user code.”
    • ladon – https://launchpad.net/ladon
      • “Ladon is a python based framework to expose functions to multiple types of web services. For instance SOAP, JSON-WSP, HTTP POST arguments, etc.”

• OpenStack