One open source project that uses such staging workflow is cmake. Consider the recent PR https://gitlab.kitware.com/cmake/cmake/-/merge_requests/4446 They use a robot that understands "Do: test", "Do: stage", "Do: unstage" and "Do: merge". Nightly tests are run on all staged PRs. A similar bot may also be helpful for MXNet.
For MXNet: Nightly tests will only fail if a commit introduced bugs. Having buggy commits on master affects development experience and makes bisecting issues harder. Staging PRs and running nightly tests on all staged PRs could address the problem. One requirement may be that the bot can identify which of the staged PRs broke the nighlty tests, if any. This may avoid the problems with the dual-branch model pointed out by Chris and it may not be too hard to understand for contributors. The main blocker could be finding ressources to implement it. I don't expect a ready-made solution to be available as plugin? On Wed, 2020-03-11 at 19:43 -0700, Chris Olivier wrote: > My $0.02 > > We had this model dual-branch when I was at GE and it was problematic. > Among other things, the two branches would tend to diverge to a large > degree and you ended up just cherry-picking in stuff here and there, which > caused even more problems, as well as the model allows the secondary branch > to get pretty buggy -- human nature being what it is -- to the point where > it's difficult to merge it into master without freezing them both and > stabilizing, merging into master, then stabilizing again (small things > almost certainly went into master in the meantime -- hotfixes, critical > features, etc, while everything was on hold stabilizing the secondary > branch). It just double the work in the end, is what I experienced. > > On Wed, Mar 11, 2020 at 5:47 PM Yuan Tang <terrytangy...@gmail.com> wrote: > > > Second to not introduce a dev branch. We should try to improve our release > > process instead and avoid another branch that may introduce confusion > > around the source of truth. > > > > On Wed, Mar 11, 2020 at 8:39 PM Tianqi Chen <tqc...@cs.washington.edu> > > wrote: > > > > > While the idea of staging seems to be reasonable. > > > Most OSS projects choose not to do so because having a complicated > > staging > > > will likely confuse the contributors, and increase the change of > > > divergence(between dev and master). > > > > > > Given that we have a release model, so in some sense the release itself > > > serves as a staging pt. > > > A good approach would simply setup the nightly if necessary strive to fix > > > regressions and make sure the formal release addresses the issues. > > > > > > TQ > > > > > > On Wed, Mar 11, 2020 at 5:32 PM Pedro Larroy < > > pedro.larroy.li...@gmail.com > > > wrote: > > > > > > > Hi > > > > > > > > I talk to some people about this and they thought it would be a good > > > idea, > > > > so sharing it here: > > > > > > > > I would propose to use a staging or "dev" branch into which nightly & > > > > performance tests are done periodically and then this branch is merged > > to > > > > master. The goal of this workflow would be to avoid having regressions > > > and > > > > nightly failures creeping into master. PRs would get merged into dev > > and > > > > dev promoted periodically / nightly into master. > > > > > > > > The names can be swapped as well, between dev and master, so PRS get > > > merged > > > > into master and it doesn't change the workflow, and staging is the > > branch > > > > where nightly changes are merged to. > > > > > > > > Have this been considered? > > > > > > > > Pedro. > > > > > > > > -- > > Yuan Tang > > https://terrytangyuan.github.io/about/ <http://twitter.com/TerryTangYuan> > > <https://terrytangyuan.github.io/about/> > >
