TL/DR; Next branch is to be considered evil, let's drop it and simplify
things, and make autotest related projects to use conventions closer to
what established projects such as linux and qemu adopt.
Hello folks:
During the last KVM Forum, it was brought to our attention about things
people felt uncomfortable with the current way we develop virt test and
autotest. The summary of the complaints, as I understand, go as:
1) It is unintuitive that the development branch is 'next', when pretty
much all the projects use 'master' for it. Supposedly people do not
expect master to be stable, and if they use 'master', they should have
the skill to fix problems that appear there. [1]
2) If people are looking for "production quality" code, they should look
for release tags rather than branches.
3) Non fast forward updates should never be done on any branch of a
public repo.
I understand 1) and 2), although IMO it is more a matter of convention
than anything else. The fact is that people expect things to be done as
other major projects that use git do. I suppose that is fine.
And given the problems we had before, I now agree with 3). It is
smoother and easier to just do classic git reverts, rather than non fast
forward updates.
Also, the way github works now sorts of enforces the same idea: Pull
requests are generated against 'master' by default (after all, that's a
pretty reasonable default, I won't argue against that), so it is easy to
miss and get things wrong when your main development branch is not 'master'.
Taking the above into consideration, my proposal to simplify the release
management process for virt-test is:
1) Drop 'next' from all autotest public repos (autotest, virt-test,
autotest-client-tests, autotest-server-tests, arc)
2) The development/integration branch becomes 'master'. All PR that look
good and pass some minimal pylint/PEP8/unittest checking will be pushed
to 'master'
3) Continuous integration should pick up the latest 'master' from all
repos and run the code in there. By looking at regression test results,
the flow is:
3a) Regression tests failed - depending on whether the offending PR's
author has a fix ready for it, it could go either:
3aa) If there's a fix ready, a new PR with the fix is made, reviewed,
and pushed to next, repeat 3).
3ab) If there's no fix ready, if things can be fixed reverting only
individual commits of the PR, that is done, rather than reverting the
entire PR. If not possible, the entire PR is reverted, repeat 3).
4) At the end of the week, if all is well, if 3) passes, tag a release
with ISO timestamp, as it is being done lately. This is because
virt-test is a far more fast paced project, and we are not yet to a
point where we can make a lot of API stability guarantees.
The release management for our larger project, autotest, that has a
slower pace and more API stability necessity, the process is similar,
except that:
1) We continue to follow semantic versioning (major.minor.patch
versions). No ISO timestamp tags are used here. Master is the dev
branch, goes trough CI, and if CI detects brokenness, it is fixed.
2) Once all issues opened as release goals are solved in master *and* CI
passes, a new release branch is created using master contents.
3) Some extra QA is made on the stable branch, and only bug fix PRs are
to be pushed into it, and of course, merged back to master as
appropriate. Due to the fact CI runs on master, the stable branch
shouldn't be very broken anyway (this of course assumes CI does a
reasonable job of testing new features, which is something hard to
guarantee and requires active effort made by the maintainers).
I am looking forward suggestions to improve on this idea, so that we
have a better and more complete specification, that can be properly
described in the project documentation, for new people contributing to
the projects.
Thanks,
Lucas
[1] This does mean that the autotest install script has to be modified
to use tags instead of branches.
_______________________________________________
Virt-test-devel mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/virt-test-devel
