OpenStack Requirements Handling, a.k.a. “Unbreak the World”

Last week the OpenStack community held our summit to discuss the work
we will be doing during the “Liberty” release cycle over the next six
months. On Friday, several of us met to discuss how we manage
dependencies. Unlike the other sessions where there are etherpads for
notes, we used a whiteboard and took pictures. This post includes
those pictures, along with transcriptions of the text.

Goals

We started by discussing the goals for requirements management, so we
could evaluate the proposal against those needs.

/blog/wp-content/uploads/2015/05/IMG_2202.png

  1. “Some” expression to downstream packagers/users
  2. Configure test jobs (unit, integration, & functional) (devstack &
    non-devstack)
  3. Encourage convergence/maintain co-installability
  4. Not be fiction
  5. pip install python-novaclient works
  6. Stop breaking all the time (esp. stable)
  7. Don’t make library release management painful
  • Current solution breaks 4 and 6 (and sometimes 5)
  • Robert’s solution breaks ?

Proposal

Next, Robert Collins presented his proposed solution for using
separate constraint lists for installing packages to be used in test
jobs, separate from the dependency specifications listed in each
project’s metadata.

/blog/wp-content/uploads/2015/05/IMG_2203.png

  • Global set(s) of complete consistent constraints
    • using == and exclusions
    • upper and lower
  • alternative testing
  • per project
    • excludes
    • minimums
    • semver upper
  • exclusions sync

Requirements specifications grow wider from left to right:

test sets (high/low) => coordinated => libraries/stackforge

! No guarantee that oslo releases won’t break project unit tests
(affirmative testing only runs integration tests)

TODO List

We prepared a list of actions we need to take this cycle to implement
the plan, and assigned each one to someone in the group.

/blog/wp-content/uploads/2015/05/IMG_2204.png

  • fix install requires in master+stable/kilo in each project (RC)
  • teach pip to honor constraints (RC)
  • change update.py to enforce wider range (until pip resolver exists)
    (RC)
  • initial global constraints for master + kilo (RC)
  • devstack patch to optionally use constraints (RC)
  • exp jobs to use constraints triggered by g-r (fungi)
  • Exp jobs triggered by local changes (fungi)
  • periodic job propose changes form pypi to master (RC)
  • unit test patch to use requirements
  • lower bounds test jobs

(I’m not sure what “exp jobs” are and may be mis-reading that)

Stable Branch Process

Then we reviewed the stable branch creation process to understand how
the change in requirements management will affect it.

pass 1

/blog/wp-content/uploads/2015/05/IMG_2205.png

Turn on master=>stable cross-check job
Release Oslo libraries
Freeze global requirements
Branch projects one at a time
Branch global requirements
Disable cross-check job
Unfreeze global-requirements

pass 2, with timeline

http://doughellmann.com/blog/wp-content/uploads/2015/05/IMG_2206.png

master/stable check
release oslo (FF-1)
converge constraints
l3/ff (soft requirements freeze) (FF)
hard freeze
branch projects (RC1)
branch g-r
disable cross-check
unfreeze

Conclusions

We finished with a little time to enjoy the beautiful scenery in
Vancouver.

/blog/wp-content/uploads/2015/05/IMG_2208.png