Re: [VOTE] Formalizing our CI process

72 hours has passed. With some tweaking and Joey's amendment, we have 12 binding +1 and 4 non-binding +1. I've updated the wiki to reflect that this process is now ratified: https://cwiki.apache.org/confluence/display/CASSANDRA/Cassandra+CI+Process Thanks everyone! ~Josh On Thu, Jan 13, 2022 at

Re: [VOTE] Formalizing our CI process

+1 with/without amendment > On 13 Jan 2022, at 05:26, Berenguer Blasi wrote: > > Oh I completely agree with you. I was just trying to explain that > getting to 2/3 failures is a realistic target. > > +1 also on the amendment. > > On 12/1/22 17:39, Joseph Lynch wrote: >> On Wed, Jan 12, 2022 at

Re: [VOTE] Formalizing our CI process

Oh I completely agree with you. I was just trying to explain that getting to 2/3 failures is a realistic target. +1 also on the amendment. On 12/1/22 17:39, Joseph Lynch wrote: > On Wed, Jan 12, 2022 at 3:25 AM Berenguer Blasi > wrote: >> jenkins CI was at 2/3 flakies consistently post 4.0 relea

Re: [VOTE] Formalizing our CI process

> "All releases by default are expected to have a green test run on > ci-cassandra Jenkins. In exceptional circumstances (security incidents, data > loss, etc requiring hotfix), members with binding votes on a release may > choose to approve a release with known failing tests." +1 with amendmen

Re: [VOTE] Formalizing our CI process

> > > "All releases by default are expected to have a green test run on > ci-cassandra Jenkins. In exceptional circumstances (security incidents, > data loss, etc requiring hotfix), members with binding votes on a release > may choose to approve a release with known failing tests." > +1, thanks J

Re: [VOTE] Formalizing our CI process

+1nb with the amendment On 2022/01/12 21:03:08 Andrés de la Peña wrote: > Still +1 with the amendment > > On Wed, 12 Jan 2022 at 19:57, C. Scott Andreas wrote: > > > +1nb, with and without the amendment. > > > > Reason for mentioning without: I see the ability to cut a release to > > address a

Re: [VOTE] Formalizing our CI process

Still +1 with the amendment On Wed, 12 Jan 2022 at 19:57, C. Scott Andreas wrote: > +1nb, with and without the amendment. > > Reason for mentioning without: I see the ability to cut a release to > address an urgent security or data loss issue as one of the strongest > arguments for maintaining g

Re: [VOTE] Formalizing our CI process

I agree with Scott’s sentiments. If we are keeping a green CI then an urgent release with green CI should not be hard. In the worst case you do the “urgent” release as a single commit fix on top of the previous release tag, not by cutting the current dev branch, in that case you should easily

Re: [VOTE] Formalizing our CI process

+1nb, with and without the amendment.Reason for mentioning without: I see the ability to cut a release to address an urgent security or data loss issue as one of the strongest arguments for maintaining green CI as a resting state so we are ready in the event of an emergency.Test results that we

Re: [VOTE] Formalizing our CI process

+1 > On Jan 12, 2022, at 8:39 AM, Joseph Lynch wrote: > > On Wed, Jan 12, 2022 at 3:25 AM Berenguer Blasi > wrote: >> >> jenkins CI was at 2/3 flakies consistently post 4.0 release. > > That is really impressive and I absolutely don't mean to downplay that > achievement. > >> Then things bro

Re: [VOTE] Formalizing our CI process

I remain +1 with the amendment.

Re: [VOTE] Formalizing our CI process

> > "All releases by default are expected to have a green test run on > ci-cassandra Jenkins. In exceptional circumstances (security incidents, > data loss, etc requiring hotfix), members with binding votes on a release > may choose to approve a release with known failing tests." +1 with the amen

Re: [VOTE] Formalizing our CI process

+1 w/ Joey's amendment On Wed, 12 Jan 2022 at 13:00, Michael Shuler wrote: > (still) +1 as amended > > Michael > > On 1/12/22 11:54, Caleb Rackliffe wrote: > > +1 w/ Joey's amendment > > > > On Wed, Jan 12, 2022 at 11:04 AM Joshua McKenzie > > wrote: > > > > I'

Re: [VOTE] Formalizing our CI process

(still) +1 as amended Michael On 1/12/22 11:54, Caleb Rackliffe wrote: +1 w/ Joey's amendment On Wed, Jan 12, 2022 at 11:04 AM Joshua McKenzie > wrote: I'd say an amendment with a directional poll would be fine. I don't think this is controversial. T

Re: [VOTE] Formalizing our CI process

+1 w/ Joey's amendment On Wed, Jan 12, 2022 at 11:04 AM Joshua McKenzie wrote: > I'd say an amendment with a directional poll would be fine. I don't think > this is controversial. > > That's me taking "the spirit of the law" rather than the letter though. > I'm good either way. > > ~Josh > > On

Re: [VOTE] Formalizing our CI process

I'd say an amendment with a directional poll would be fine. I don't think this is controversial. That's me taking "the spirit of the law" rather than the letter though. I'm good either way. ~Josh On Wed, Jan 12, 2022 at 11:51 AM Joseph Lynch wrote: > On Wed, Jan 12, 2022 at 11:43 AM Joshua McK

Re: [VOTE] Formalizing our CI process

On Wed, Jan 12, 2022 at 11:43 AM Joshua McKenzie wrote: > > I fully concede your point and concern Joey but I propose we phrase that > differently to emphasize the importance of clean tests. > > "All releases by default are expected to have a green test run on > ci-cassandra Jenkins. In exceptio

Re: [VOTE] Formalizing our CI process

I fully concede your point and concern Joey but I propose we phrase that differently to emphasize the importance of clean tests. "All releases by default are expected to have a green test run on ci-cassandra Jenkins. In exceptional circumstances (security incidents, data loss, etc requiring hotfix

Re: [VOTE] Formalizing our CI process

On Wed, Jan 12, 2022 at 3:25 AM Berenguer Blasi wrote: > > jenkins CI was at 2/3 flakies consistently post 4.0 release. That is really impressive and I absolutely don't mean to downplay that achievement. > Then things broke and we've been working hard to get back to the 2/3 flakies. > Most > cu

Re: [VOTE] Formalizing our CI process

I've witnessed PMCs -1 releases due to failing tests or bugs reported by users before, but prior to everyone's awesome work on CI I think a number of times folks might have been voting without knowing what the results of the full test runs were. One of the amazing contributions of this group (and o

Re: [VOTE] Formalizing our CI process

“I particularly like the suggestion PMCs can use failing tests as a reason to -1, but we do have critical patch releases now and again and common sense in getting such releases out quickly needs to be applied. ” For some reason I assumed this would always be the case in case of emergency, to consi

Re: [VOTE] Formalizing our CI process

On Wed, Jan 12, 2022 at 1:24 AM Mick Semb Wever wrote: > > Changing my vote to -1, aligned with Joey's concern here. (Thank you for > raising it.) > > I particularly like the suggestion PMCs can use failing tests as a reason to > -1, but we do have critical patch releases now and again and commo

Re: [VOTE] Formalizing our CI process

Hi Joseph jenkins CI was at 2/3 flakies consistently post 4.0 release. Then things broke and we've been working hard to get back to the 2/3 flakies. Most current failures imo are timeuuid C17133 or early termination of process C17140 related afaik. So getting back to the 2/3 'impossible' flakies s

Re: [VOTE] Formalizing our CI process

On Wed, 12 Jan 2022 at 07:23, Joseph Lynch wrote: > On Tue, Jan 11, 2022 at 4:48 PM Joshua McKenzie > wrote: > >> If this vote passes would that mean we cannot cut any release > > > > We would not cut a release with known failing tests, no. Which for > critical infrastructure software _seems_ li

Re: [VOTE] Formalizing our CI process

On Tue, Jan 11, 2022 at 4:48 PM Joshua McKenzie wrote: >> If this vote passes would that mean we cannot cut any release > > We would not cut a release with known failing tests, no. Which for critical > infrastructure software _seems_ like it should probably be table stakes, no? > While I very mu

Re: [VOTE] Formalizing our CI process

On Wed, Jan 12, 2022 at 12:47 AM Berenguer Blasi wrote: > > We shouldn't be at 15-20 failures but at 2 or 3. The problem is that those 2 > or 3 have already been hammered for over a year by 2 or 3 different > committers and they didn't crack. > Last I checked circleci was almost fully green on

Re: [VOTE] Formalizing our CI process

We shouldn't be at 15-20 failures but at 2 or 3. The problem is that those 2 or 3 have already been hammered for over a year by 2 or 3 different committers and they didn't crack. On 11/1/22 22:47, Joshua McKenzie wrote: > > this seems problematic given all four > release branches (2.2, 3.0

Re: [VOTE] Formalizing our CI process

> > this seems problematic given all four > release branches (2.2, 3.0, 3.11, 4.0) + trunk appear to have about > 15-20 failures on ci-cassandra.apache.org That's the intended outcome of making it a hard blocker for release: it incentivizes us fixing these tests. If this vote passes would that me

Re: [VOTE] Formalizing our CI process

> No release can be cut without a fully green CI run on ci-cassandra.apache.org I appreciate the goal but this seems problematic given all four release branches (2.2, 3.0, 3.11, 4.0) + trunk appear to have about 15-20 failures on ci-cassandra.apache.org at the time of this vote. If this vote passe

Re: [VOTE] Formalizing our CI process

+1 On Tue, Jan 11, 2022 at 7:02 AM Andrés de la Peña wrote: > +1 > > On Tue, 11 Jan 2022 at 13:45, Joshua McKenzie > wrote: > >> If my understanding is correct, we run the canonical set *before* merging, >>> and the runs triggered by the cassandra CI bot include the full set >>> *after* a commi

Re: [VOTE] Formalizing our CI process

+1 On Tue, 11 Jan 2022 at 13:45, Joshua McKenzie wrote: > If my understanding is correct, we run the canonical set *before* merging, >> and the runs triggered by the cassandra CI bot include the full set >> *after* a commit is merged. > > Good point. Clarified to indicate it's canonical *circlec

Re: [VOTE] Formalizing our CI process

> > If my understanding is correct, we run the canonical set *before* merging, > and the runs triggered by the cassandra CI bot include the full set > *after* a commit is merged. Good point. Clarified to indicate it's canonical *circleci tests* to run before merging, and the ci-cassandra jenkins i

Re: [VOTE] Formalizing our CI process

On Mon, 10 Jan 2022 at 20:00, Joshua McKenzie wrote: > Wiki draft article here: > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=199530280 > +1 small nits: - none of the other confluence pages use the "Apache" prefix - can it reference the CI Systems page please - post-vote:

Re: [VOTE] Formalizing our CI process

+1, same note as David On Tue, 11 Jan 2022 at 4:54, Benjamin Lerer wrote: > +1 > > Le mar. 11 janv. 2022 à 07:09, Berenguer Blasi > a écrit : > >> +1 >> >> On 10/1/22 20:00, Joshua McKenzie wrote: >> > Wiki draft article >> > here: >> https://cwiki.apache.org/confluence/pages/viewpage.action?pa

Re: [VOTE] Formalizing our CI process

+1 Le mar. 11 janv. 2022 à 07:09, Berenguer Blasi a écrit : > +1 > > On 10/1/22 20:00, Joshua McKenzie wrote: > > Wiki draft article > > here: > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=199530280 > > < > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=199

Re: [VOTE] Formalizing our CI process

+1 On 10/1/22 20:00, Joshua McKenzie wrote: > Wiki draft article > here:  > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=199530280 > > > > The vote will be open for 72 hours (it's short + early indicat

Re: [VOTE] Formalizing our CI process

Would you like to elaborate on when to run the "canonical set of tests" and when to run the others? If my understanding is correct, we run the canonical set *before* merging, and the runs triggered by the cassandra CI bot include the full set *after* a commit is merged. - Yifan On Mon, Jan 10, 2

Re: [VOTE] Formalizing our CI process

+1 nb > On Jan 10, 2022, at 1:00 PM, Joshua McKenzie wrote: > > Wiki draft article here: > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=199530280 > > > The vote will be open for 72 hours (it's sho

Re: [VOTE] Formalizing our CI process

> Finalize on the canonical set of tests and JDK env to run pre-commit (JIRA > Pending; to be linked) Would be nice to get that defined, but I am cool with +1 for the rest and deferring solving that till after > On Jan 10, 2022, at 1:18 PM, Joshua McKenzie wrote: > > +1 > > On Mon, Jan 10,

Re: [VOTE] Formalizing our CI process

+1 On Mon, Jan 10, 2022 at 3:32 PM Michael Shuler wrote: > +1 > > Michael > > On 1/10/22 13:00, Joshua McKenzie wrote: > > Wiki draft article here: > > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=199530280 > > < > https://cwiki.apache.org/confluence/pages/viewpage.action?p

Re: [VOTE] Formalizing our CI process

+1 Michael On 1/10/22 13:00, Joshua McKenzie wrote: Wiki draft article here: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=199530280 The vote will be open for 72 hours (it's short + early indicati

Re: [VOTE] Formalizing our CI process

+1 On Mon, Jan 10, 2022 at 1:00 PM Joshua McKenzie wrote: > > Wiki draft article here: > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=199530280 > > The vote will be open for 72 hours (it's short + early indication on > discussion was consensus). > Committer / pmc votes bind

[VOTE] Formalizing our CI process

Wiki draft article here: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=199530280 The vote will be open for 72 hours (it's short + early indication on discussion was consensus). Committer / pmc votes binding. Simple majority passes. References: Background: original ML thread he