While I'm not a particularly active python-developer, I have spent quite a bit of time using OpenStack's Zuul/gerrit system/workflow and all I can say is it: It is the right way to write software. Huge +1 from me on every project using it :-)
Alex On Sat, Jan 18, 2014 at 11:34 PM, Nick Coghlan <ncogh...@gmail.com> wrote: > One of the items I have on the language summit agenda is the core > reviewer workflow used by the OpenStack project, specifically Zuul, > the merge gating system that powers that workflow. I know some other > folks here are working on OpenStack these days so this will already be > familiar to them, but many aren't. I'm not either, but my day job > involves developing CI workflow automation tools for Red Hat, so I'm > paying very close attention to what the OpenStack infrastructure team > are up to. > > We don't necessarily have to wait until the language summit to discuss > it though, so I figured I'd pass along a link to James Blair's > presentation at linux.conf.au 2014: > > Stream: https://www.youtube.com/watch?v=sLD9LHc1QFM > Download: > http://mirror.linux.org.au/linux.conf.au/2014/Friday/106-How_OpenStack_Improves_Code_Quality_with_Project_Gating_and_Zuul_-_James_Blair.mp4 > > The Zuul docs are at: http://ci.openstack.org/zuul/ > The OpenStack Zuul dashboard is at: http://status.openstack.org/zuul/ > > The key difference, relative to our current workflow, is to take the > human out of the loop for the final pre-merge test run. Instead, once > a core reviewer gives a patch a +2, the process of taking that > reviewed patch, checking it still merges correctly, checking it passes > all the tests on the stable buildbots and then merging it into source > control would all be handled automatically by Zuul. > > This will require and/or enable eliminating several of the current > annoyances in the core development workflow: > > - the annoyingly manual "download patch, apply patch, run tests > locally, commit change, push change, watch the buildbots for problems" > approach goes away entirely (while what we have now is substantially > better than what we had a few years ago, times move on and projects > like OpenStack raise the bar when it comes to figuring out how to make > the most effective of scarce contributor time) > - spurious conflicts on NEWS will go away (as we'll have to finally > address this in order to avoid breaking Zuul's gating) > - push races will go away (since merging would now be handled by Zuul, > non conflicting changes would be rebased automatically, and only > conflicting changes would be bounced back to the submitter for > updates). Critically, conference sprints might create a backlog in > Zuul, but they wouldn't completely break a developer's workflow > through incessant push races relating to non-conflicting changes. > - POSIX developers breaking the Windows buildbots and vice-versa > (since Zuul would ensure at least the stable buildbots remain green by > blocking such changes before they hit the main branches). This means > even when we get such things wrong, we will no longer have the time > pressure of needing to unbreak other people's builds. > - approving and merging pure docs patches should become largely > trivial, as it should be possible to configure Zuul to only check that > the docs build cleanly for those, rather than running the full test > suite across all the stable buildbots. > - checking for a contributor licensing agreement in Roundup before > processing a patch could also be automated, rather than requiring core > developers to check for it manually. > > Zuul, as OpenStack use it, already has plugins for the Gerrit code > review/git repo management system (at least as customised by the > OpenStack infrastructure folks), as well as for Jenkins to run the > automated CI tests. > > Rather than suggesting wholesale changes to our own infrastructure, > what I am suggesting we consider is devoting time (and potentially PSF > funding) to getting Zuul to play nice with Roundup, Reitveld, BuildBot > and Mercurial. (Like the rest of our existing infrastructure, Zuul is > a web service implemented in Python). > > The main technical challenges I foresee are: > > * dealing with maintenance branches (especially for patches which > don't merge forward cleanly), since OpenStack currently appear to > "handle" that limitation by just not providing upstream maintenance > branches at all, leaving downstream vendors to fill the gap > * finally cleaning up the way we manage the NEWS file (see > http://bugs.python.org/issue18967 for discussion) > * replicating Gerrit's patch approval/voting mechanism in Reitveld > * replicating Gerrit's merge/rebase capabilities in Mercurial (I'm > sure Mercurial is functionally capable of this, I just don't know the > best way to model it). > * actually writing the new Zuul plugins to talk to the services used > by the CPython project, rather than those used by OpenStack > > There would also be a non-trivial update to the developer guide > involved in such a change, since it would heavily impact the core > developer workflows, as well as the way external contributors interact > with the core development team. > > If people find this idea interesting, I would like to invite some of > the Mercurial devs and OpenStack infrastructure folks (i.e. Zuul devs) > to the language summit (leaving it up to them if they stay for the > whole day, or just this part). I believe we already have enough > Reitveld, Roundup and Buildbot experience amongst the core development > team to assess the feasibility of the idea from that side of things. > > Regards, > Nick. > > -- > Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia > _______________________________________________ > python-committers mailing list > python-committers@python.org > https://mail.python.org/mailman/listinfo/python-committers > -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero GPG Key fingerprint: 125F 5C67 DFE9 4084
_______________________________________________ python-committers mailing list python-committers@python.org https://mail.python.org/mailman/listinfo/python-committers
