Planting Acorns

This post is based on the closing keynote I gave for PyTennessee in February 2018, where I talked about how the governance of an open source project impacts  the health of the project, and some lessons we learned in building the OpenStack community that can be applied to other projects.

OpenStack is a cloud computing system written in Python. The project is 8 years old. During that time we have had 18 major releases, and to give you an idea of the size of the community, during 2017, we had 2,500 people contribute more than 65,000 patches. The age and size of the project means our community has been through several transitions and challenges that other smaller projects may not yet have encountered, and my hope is that by thinking about them early, you can prepare for them, and your communities will be able to grow in a healthier way.

I started working on OpenStack in 2012, 2 years after the project had been launched and right at the beginning of a period of increased interest and contributions. It was the leading edge of the up-slope for our hype curve. I started looking at the code with my team at Dreamhost, but my first significant exposure to the community was at a Design Summit, an in-person event for organizing the teams working on different parts of the software. It was an energizing experience.

I had been to conferences before, including open source conferences like PyCon and PyTennessee, but never anything quite like this. We had hundreds of contributors gathered together in one place for a week of meetings, discussions, arguments, decision making, and, just as importantly, socializing. The enthusiasm of the participants was amazing, and the event led me to dive deeper and deeper into the project over the following 6 years.

At that summit I was part of starting two new teams, one based on metering cloud use for billing and one based on building a set of reusable libraries for the different OpenStack services. My experience with trying to help launch those teams within the community sparked my ongoing interest in open source project governance — not just how to build open source software, but how to build the communities around it — in part because, at the time, OpenStack had no system in place for expanding the scope of the project, and the community struggled to work out how to do it. Today, I want to share 9 themes that may help you avoid pitfalls when considering governance in your own projects.

Continue reading “Planting Acorns”

git-nit 1.0.0

git-nit is a tool for grabbing existing reviews on gerrit and layering on a new patch to fix nits.

  • This is the first public release.

openstack-summit-counter 0.2.0

openstack-summit-counter is a plugin for python-openstackclient, the command line tool for interacting with OpenStack. This plugin helps you answer the summit registration question about how many summits you have attended in the past.

What’s new in 0.2.0?

  • Add support for counting PTGs (contributed by Colleen Murphy)

openstack-summit-counter 0.1.0

openstack-summit-counter is a plugin for python-openstackclient, the command line tool for interacting with OpenStack. This plugin helps you answer the summit registration question about how many summits you have attended in the past.

  • This is the first public release.

git-os-job 1.1.1

The OpenStack project stores the logs for all of the test jobs related to a commit on http://logs.openstack.org organized by the commit hash. To review the logs after a job runs, most developers start with the message jenkins leaves on gerrit, and click through to the log files. Not all jenkins jobs are triggered by or related to a gerrit review, though (e.g, release tags).

git-os-job makes it easy to find those logs by finding the hash of the commit and using it to build the right URL. It will then either print the URL or open a web browser directly.

What’s new in 1.1.1?

  • don’t decode bytes to unicode twice

git-os-job 1.1.0

The OpenStack project stores the logs for all of the test jobs related to a commit on http://logs.openstack.org organized by the commit hash. To review the logs after a job runs, most developers start with the message jenkins leaves on gerrit, and click through to the log files. Not all jenkins jobs are triggered by or related to a gerrit review, though (e.g, release tags).

git-os-job makes it easy to find those logs by finding the hash of the commit and using it to build the right URL. It will then either print the URL or open a web browser directly.

What’s new in 1.1.0?

  • add –reverse option to go from log URL to review URL
  • decode command output for python 3 support
  • add -u alias for –url command

Stop Working So Hard: Scaling Open Source Community Practices

Lately, I have been revising some of the OpenStack community’s processes to make them more sustainable. As we grew over the last 7 years to have more than 2,000 individual contributors to the current release, some practices that worked when they were implemented have begun causing trouble for us now that our community is changing in different ways. My goal in reviewing those practices is to find ways to eliminate the challenges.

OpenStack is developed by a collection of project teams, most of which focus on a feature-related area, such as block storage or networking. The areas where we have most needed to change intersect with all of those teams, such as release management and documentation. Although the teams responsible for those tasks have tended to be small, their members have been active and dedicated. At times that dedication has masked the near-heroic level of effort they were making to keep up with the work load.

When someone is overloaded in a corporate environment, where tasks are assigned and the performance and workload of team members are reviewed regularly, the employee can appeal to management for help. The solution may be to hire or assign new contributors, change the project schedule, or to make a short term trade-off that incurs technical debt. However, open source projects are largely driven by volunteers, so assigning people to work on a task isn’t an option. Even in a sponsor-driven community such as OpenStack, where many contributors are being paid to work on the project overall, sponsors typically give a relatively narrow mandate for the way their contributors can spend their time. Changing the project schedule is always an option, but if there are no volunteers for a task today, there is no guarantee volunteers will appear tomorrow, so it may not help.

We must use a different approach to eliminate the need for heroic effort.

Continue reading “Stop Working So Hard: Scaling Open Source Community Practices”

Lessons learned from working on large scale, cross-project initiatives in OpenStack

I have been involved with OpenStack development since just before the Folsom summit in 2012. Over the course of that time, I have participated in innumerable discussions about 3 big features tied to OpenStack logging: translating log messages, adding request IDs to log messages, and adding unique message IDs to log messages. We have had various degrees of success with the design, implementation, and ongoing maintenance of all three features, and the reasons for success or failure in each case provide helpful insight into how to approach changes with large community and product scope that should be considered before our next discussion at the summit/forum in Boston in 2017.
Continue reading “Lessons learned from working on large scale, cross-project initiatives in OpenStack”