OpenStack Requirements Handling, a.k.a. “Unbreak the World”

Originally posted · 3 min read

Last week the OpenStack community held our summit to discuss the work we will be doing during the “Liberty” release cycle over the next six months. On Friday, several of us met to discuss how we manage dependencies. Unlike the other sessions where there are etherpads for notes, we used a whiteboard and took pictures. This post includes those pictures, along with transcriptions of the text.

Goals

We started by discussing the goals for requirements management, so we could evaluate the proposal against those needs.

goals

  • “Some” expression to downstream packagers/users
  • Configure test jobs (unit, integration, & functional) (devstack & non-devstack)
  • Encourage convergence/maintain co-installability
  • Not be fiction
  • pip install python-novaclient works
  • Stop breaking all the time (esp. stable)
  • Don’t make library release management painful
  • Current solution breaks 4 and 6 (and sometimes 5)
    • Robert’s solution breaks ?

Proposal

Next, Robert Collins presented his proposed solution for using separate constraint lists for installing packages to be used in test jobs, separate from the dependency specifications listed in each project’s metadata.

proposal

  • Global set(s) of complete consistent constraints
    • using == and exclusions
    • upper and lower
  • alternative testing
    • per project
    • excludes
    • minimums
  • semver upper
  • exclusions sync

Requirements specifications grow wider from left to right:

test sets (high/low) => coordinated => libraries/stackforge

! No guarantee that oslo releases won’t break project unit tests (affirmative testing only runs integration tests)

TODO List

We prepared a list of actions we need to take this cycle to implement the plan, and assigned each one to someone in the group.

todo list

  • fix install requires in master+stable/kilo in each project (RC)
  • teach pip to honor constraints (RC)
  • change update.py to enforce wider range (until pip resolver exists) (RC)
  • initial global constraints for master + kilo (RC)
  • devstack patch to optionally use constraints (RC)
  • exp jobs to use constraints triggered by g-r (fungi)
  • Exp jobs triggered by local changes (fungi)
  • periodic job propose changes form pypi to master (RC)
  • unit test patch to use requirements
  • lower bounds test jobs

(I’m not sure what “exp jobs” are and may be mis-reading that)

Stable Branch Process

Then we reviewed the stable branch creation process to understand how the change in requirements management will affect it.

pass 1

pass 1

Turn on master=>stable cross-check job Release Oslo libraries Freeze global requirements Branch projects one at a time Branch global requirements Disable cross-check job Unfreeze global-requirements

pass 2, with timeline

pass 2

  • master/stable check
  • release oslo
  • (FF-1)
  • converge constraints
  • l3/ff (soft requirements freeze)
  • (FF) hard freeze
  • branch projects
  • (RC1) branch g-r
  • disable cross-check unfreeze

Conclusions

We finished with a little time to enjoy the beautiful scenery in Vancouver.

view of Vancouver bay