Could we also add something about running new tests through the multiplexer?
On Fri, Dec 17, 2021 at 10:23 AM Joshua McKenzie <jmcken...@apache.org> wrote: > > So to clarify it all in one place, the proposed new CI process we should > test for consensus is: > > 1. Canonical CI for a release is ci-cassandra. We can optionally, and in > practice will, run circle as well but don't codify blocking on that. > 2. (NEW) We don't release unless we get a fully green run. > 3. Before any merge, you need either a non-regressing (i.e. no new > failures) run of circleci with a (specific suite of tests TBD) or of > ci-cassandra. > 3.a Non-regressing is defined here as "Doesn't introduce any new test > failures; any new failures in CI are clearly not attributable to this diff" > 3.b: (NEW) After merging tickets, ci-cassandra runs against the SHA > and the author gets an advisory update on the related JIRA for any new > errors on CI. The author of the ticket will take point on triaging this new > failure and either fixing (if clearly reproducible or related to their > work) or opening a JIRA for the intermittent failure and linking it in > butler (https://butler.cassandra.apache.org/#/) > 4. (NEW) The Build Lead role + Butler catches and documents all failures > and anything that slips through the procedural cracks in 3.b; resourcing > for fixing flakey tests TBD > > Our two TBD we can tackle separately from consensus on the above: > 1. Suite of tests on circle required to be considered ready for merge > 2. How we resource fixing flakey tests that are functionally impossible to > attribute without essentially fixing the flake > > On Fri, Dec 17, 2021 at 10:56 AM Ekaterina Dimitrova <e.dimitr...@gmail.com> > wrote: > > > +1 (nb) on my end too, I second Mick > > Thanks for putting this together Josh > > > > On Fri, 17 Dec 2021 at 10:48, Mick Semb Wever <m...@apache.org> wrote: > > > > > > > > > > > > > > 3.c: (NEW) After merging tickets, run ci-cassandra (already do this) > > and > > > > get an advisory update on the related JIRA for any new errors on the > > run > > > of > > > > the SHA > > > > > > > > I strongly prefer we amend our process with 3.c. > > > > > > > > > > > > +1 Yup, this is the most important missing piece for me. > > > > > > I also wouldn't mind we word the responsibility of the author at > > > post-commit fault to be involved/leading in the fix. This incentivises > > > people to do 2+3 properly, and not push it onto the build role. > > > > > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional commands, e-mail: dev-h...@cassandra.apache.org
