Re: [DISCUSS] Releasable trunk and quality

[Berenguer Blasi]

+1. I would add a 'post-commit' step: check the jenkins CI run for your
merge and see if sthg broke regardless.

On 6/12/21 23:51, Ekaterina Dimitrova wrote:
> Hi Josh,
> All good questions, thank you for raising this topic.
> To the best of my knowledge, we don't have those documented but I will put
> notes on what tribal knowledge I know about and I personally follow :-)
>
>  Pre-commit test suites: * Which JDK's?  - both are officially supported so
> both.
>
> * When to include all python tests or do JVM only (if ever)? - if I test
> only a test fix probably
>
>  * When to run upgrade tests? - I haven't heard any definitive guideline.
> Preferably every time but if there is a tiny change I guess it can be
> decided for them to be skipped. I would advocate to do more than less.
>
> * What to do if a test is also failing on the reference root (i.e. trunk,
> cassandra-4.0, etc)? - check if a ticket exists already, if not - open one
> at least, even if I don't plan to work on it at least to acknowledge
> the issue and add any info I know about. If we know who broke it, ping the
> author to check it.
>
> * What to do if a test fails intermittently? - Open a ticket. During
> investigation - Use the CircleCI jobs for running tests in a loop to find
> when it fails or to verify the test was fixed. (This is already in my draft
> CircleCI document, not yet released as it was pending on the documents
> migration.)
>
> Hope that helps.
>
> ~Ekaterina
>
> On Mon, 6 Dec 2021 at 17:20, Joshua McKenzie  wrote:
>
>> As I work through the scripting on this, I don't know if we've documented
>> or clarified the following (don't see it here:
>> https://cassandra.apache.org/_/development/testing.html):
>>
>> Pre-commit test suites:
>> * Which JDK's?
>> * When to include all python tests or do JVM only (if ever)?
>> * When to run upgrade tests?
>> * What to do if a test is also failing on the reference root (i.e. trunk,
>> cassandra-4.0, etc)?
>> * What to do if a test fails intermittently?
>>
>> I'll also update the above linked documentation once we hammer this out and
>> try and bake it into the scripting flow as much as possible as well. Goal
>> is to make it easy to do the right thing and hard to do the wrong thing,
>> and to have these things written down rather than have it be tribal
>> knowledge that varies a lot across the project.
>>
>> ~Josh
>>
>> On Sat, Dec 4, 2021 at 9:04 AM Joshua McKenzie 
>> wrote:
>>
>>> After some offline collab, here's where this thread has landed on a
>>> proposal to change our processes to incrementally improve our processes
>> and
>>> hopefully stabilize the state of CI longer term:
>>>
>>> Link:
>>>
>> https://docs.google.com/document/d/1tJ-0K7d6PIStSbNFOfynXsD9RRDaMgqCu96U4O-RT84/edit#bookmark=id.16oxqq30bby4
>>> Hopefully the mail server doesn't butcher formatting; if it does, hit up
>>> the gdoc and leave comments there as should be open to all.
>>>
>>> Phase 1:
>>> Document merge criteria; update circle jobs to have a simple pre-merge
>> job
>>> (one for each JDK profile)
>>>  * Donate, document, and formalize usage of circleci-enable.py in ASF
>>> repo (need new commit scripts / dev tooling section?)
>>> * rewrites circle config jobs to simple clear flow
>>> * ability to toggle between "run on push" or "click to run"
>>> * Variety of other functionality; see below
>>> Document (site, help, README.md) and automate via scripting the
>>> relationship / dev / release process around:
>>> * In-jvm dtest
>>> * dtest
>>> * ccm
>>> Integrate and document usage of script to build CI repeat test runs
>>> * circleci-enable.py --repeat-unit org.apache.cassandra.SomeTest
>>> * Document “Do this if you add or change tests”
>>> Introduce “Build Lead” role
>>> * Weekly rotation; volunteer
>>> * 1: Make sure JIRAs exist for test failures
>>> * 2: Attempt to triage new test failures to root cause and assign out
>>> * 3: Coordinate and drive to green board on trunk
>>> Change and automate process for *trunk only* patches:
>>> * Block on green CI (from merge criteria in CI above; potentially
>>> stricter definition of "clean" for trunk CI)
>>> * Consider using github PR’s to merge (TODO: determine how to handle
>>> circle + CHANGES; see below)
>>> Automate process for *multi-branch* merges
>>> * Harden / contribute / document dcapwell script (has one which does
>>> the following):
>>> * rebases your branch to the latest (if on 3.0 then rebase
>> against
>>> cassandra-3.0)
>>> * check compiles
>>> * removes all changes to .circle (can opt-out for circleci
>> patches)
>>> * removes all changes to CHANGES.txt and leverages JIRA for the
>>> content
>>> * checks code still compiles
>>> * changes circle to run ci
>>> * push to a temp branch in git and run CI (circle + Jenkins)
>>> * when all branches are clean (waiting step is manual)
>>> * TODO: Define 

Re: [DISCUSS] Releasable trunk and quality

[Ekaterina Dimitrova]

Ok seems I was wrong and messed up the mails in my mailbox. Please ignore
my previous email

On Mon, 6 Dec 2021 at 18:01, Ekaterina Dimitrova 
wrote:

>
> I think the script discussion is on a different thread and attached
> document which I am also about to address soon :-)
>
> On Mon, 6 Dec 2021 at 17:59, bened...@apache.org 
> wrote:
>
>> Is there a reason we discounted modifying the merge strategy?
>>
>> I’m just a little wary of relying on scripts for consistency of behaviour
>> here. Environments differ, and it would be far preferable for consistency
>> of behaviour to rely on shared infrastructure if possible. I would probably
>> be against mandating these scripts, at least.
>>
>> From: Joshua McKenzie 
>> Date: Monday, 6 December 2021 at 22:20
>> To: dev@cassandra.apache.org 
>> Subject: Re: [DISCUSS] Releasable trunk and quality
>> As I work through the scripting on this, I don't know if we've documented
>> or clarified the following (don't see it here:
>> https://cassandra.apache.org/_/development/testing.html):
>>
>> Pre-commit test suites:
>> * Which JDK's?
>> * When to include all python tests or do JVM only (if ever)?
>> * When to run upgrade tests?
>> * What to do if a test is also failing on the reference root (i.e. trunk,
>> cassandra-4.0, etc)?
>> * What to do if a test fails intermittently?
>>
>> I'll also update the above linked documentation once we hammer this out
>> and
>> try and bake it into the scripting flow as much as possible as well. Goal
>> is to make it easy to do the right thing and hard to do the wrong thing,
>> and to have these things written down rather than have it be tribal
>> knowledge that varies a lot across the project.
>>
>> ~Josh
>>
>> On Sat, Dec 4, 2021 at 9:04 AM Joshua McKenzie 
>> wrote:
>>
>> > After some offline collab, here's where this thread has landed on a
>> > proposal to change our processes to incrementally improve our processes
>> and
>> > hopefully stabilize the state of CI longer term:
>> >
>> > Link:
>> >
>> https://docs.google.com/document/d/1tJ-0K7d6PIStSbNFOfynXsD9RRDaMgqCu96U4O-RT84/edit#bookmark=id.16oxqq30bby4
>> >
>> > Hopefully the mail server doesn't butcher formatting; if it does, hit up
>> > the gdoc and leave comments there as should be open to all.
>> >
>> > Phase 1:
>> > Document merge criteria; update circle jobs to have a simple pre-merge
>> job
>> > (one for each JDK profile)
>> >  * Donate, document, and formalize usage of circleci-enable.py in
>> ASF
>> > repo (need new commit scripts / dev tooling section?)
>> > * rewrites circle config jobs to simple clear flow
>> > * ability to toggle between "run on push" or "click to run"
>> > * Variety of other functionality; see below
>> > Document (site, help, README.md) and automate via scripting the
>> > relationship / dev / release process around:
>> > * In-jvm dtest
>> > * dtest
>> > * ccm
>> > Integrate and document usage of script to build CI repeat test runs
>> > * circleci-enable.py --repeat-unit org.apache.cassandra.SomeTest
>> > * Document “Do this if you add or change tests”
>> > Introduce “Build Lead” role
>> > * Weekly rotation; volunteer
>> > * 1: Make sure JIRAs exist for test failures
>> > * 2: Attempt to triage new test failures to root cause and assign
>> out
>> > * 3: Coordinate and drive to green board on trunk
>> > Change and automate process for *trunk only* patches:
>> > * Block on green CI (from merge criteria in CI above; potentially
>> > stricter definition of "clean" for trunk CI)
>> > * Consider using github PR’s to merge (TODO: determine how to handle
>> > circle + CHANGES; see below)
>> > Automate process for *multi-branch* merges
>> > * Harden / contribute / document dcapwell script (has one which does
>> > the following):
>> > * rebases your branch to the latest (if on 3.0 then rebase
>> against
>> > cassandra-3.0)
>> > * check compiles
>> > * removes all changes to .circle (can opt-out for circleci
>> patches)
>> > * removes all changes to CHANGES.txt and leverages JIRA for the
>> > content
>> > * checks code still compiles
>> > * changes circle to run ci
>> > * push to a temp branch in git and run CI (circle + Jenkins)
>> > * when all branches are clean (waiting step is manual)
>> > * TODO: Define “clean”
>> > * No new test failures compared to reference?
>> > * Or no test failures at all?
>> > * merge changes into the actual branches
>> > * merge up changes; rewriting diff
>> > * push --atomic
>> >
>> > Transition to phase 2 when:
>> > * All items from phase 1 are complete
>> > * Test boards for supported branches are green
>> >
>> > Phase 2:
>> > * Add Harry to recurring run against trunk
>> > * Add Harry to release pipeline
>> > * Suite of perf tests against trunk recurring
>> >
>> >
>> >
>> > On Wed, Nov 17, 2021 at 

Re: [DISCUSS] Nested YAML configs for new features

[Ekaterina Dimitrova]

 Please find my comments inline below

* Good with nested configs - the question is how they will be introduced
and maintained? I wouldn't advocate for maintaining more than yaml file but
probably as you once mentioned some time ago (if I remember correctly) -
having one format as default and just documenting the support for the other
one/ones. Now which is the default one is a different topic.
* Where/How we group is an open question, maybe we move this to a JIRA as
follow up work to CASSANDRA-15234? - not part of CASSANDRA-15234 as per all
the discussions, already in review (thank you for your first quick round
btw, appreciate it!)

Spoke with Ekaterina about this, and not solved in 15234; let's move this
to a follow up JIRA for 15234? - For the broader audience, currently what I
solve around naming in CASSANDRA-15234 is removing the unit suffix and
moving to the format noun_verb the config parameters names. After all
discussions and realizing the great interest and variety of opinions, I
tried really to split more tickets from CASSANDRA-15234 and to keep
primarily the new custom types and the new framework with backward
compatibility as the main body of work, good also for the reviewers. Last
year I came up with the idea of reorganizing the config file a bit which
led to discussions. So considering my previous point about splitting to a
more incremental approach considering the variety of opinions, I suggested
when submitting for review to open a new ticket for that new organization
of our config file. Probably we can add the abort/fail and any other
similar concerns/questions there post CASSANDRA-15234?


On Fri, 3 Dec 2021 at 13:34, David Capwell 
wrote:

> Thanks everyone for the feedback!  If I am reading this properly I am
> seeing the following
>
> * Good with nested configs
> * Good with YAML layer supporting flat structure (possible foo.bar.baz for
> the path foo: {bar: {baz: 42}}), how this relates with Settings table
> should be resolved, but there is a open ticket for this (enhance our YAML
> CASSANDRA-17166, and support updates to Settings vtable CASSANDRA-15254)
> * Where/How we group is an open question, maybe we move this to a JIRA as
> follow up work to CASSANDRA-15234?
>
> > We’re also mixing terminology already, with limits/thresholds and
> fail/abort.
>
> Spoke with Ekaterina about this, and not solved in 15234; lets move this
> to a follow up JIRA for 15234?
>
> > On Nov 30, 2021, at 6:08 AM, Ekaterina Dimitrova 
> wrote:
> >
> > Thank you for confirming as I misread your email at first :-)
> > I had a chat with David last week and I don’t think his plan is reworking
> > of 15234 but incremental improvements on top of it.
> > Regarding config, after spending time cleaning around and looking more
> into
> > detail my only appeal is:
> > - Centralized management and not 5 places to change things when you add
> new
> > config so we are less error-prone
> > - Documenting things for people who add new config or for our users (I
> > promised and I will do it for 15234 but it will be good to continue doing
> > it with any further changes down the road)
> > - be careful with breaking changes
> >
> > Thank you
> > Ekaterina
> >
> > On Tue, 30 Nov 2021 at 8:59, bened...@apache.org 
> > wrote:
> >
> >> I mean that it has been waiting for months, is ready to go, and I don’t
> >> want to hold you up any longer.
> >>
> >> From: Ekaterina Dimitrova 
> >> Date: Tuesday, 30 November 2021 at 13:44
> >> To: dev@cassandra.apache.org 
> >> Subject: Re: [DISCUSS] Nested YAML configs for new features
> >> “
> >> IMO 15234 has sailed – it’s been held up for a long time, and was
> brought
> >> to this list for discussion with no engagement. Ekaterina is long
> overdue
> >> being able to commit her work. “
> >>
> >>
> >> Sailed? I submitted the patch a week ago for review. Not sure how to
> >> understand this statement. Can elaborate, please?
> >>
> >> On Tue, 30 Nov 2021 at 8:09, bened...@apache.org 
> >> wrote:
> >>
> >>> The problem with scoping this to “features” is that we end up with at
> >> best
> >>> local coherence. The config file as a whole will end up just as
> >> incoherent
> >>> through its design evolution as it has historically.
> >>>
> >>> If you take a look at my proposed layout for the overall config, there
> is
> >>> a “limits” section that specifies thresholds for reporting warnings and
> >>> errors for various scenario. In this case, we probably don’t also want
> >>> per-feature limits? We’re also mixing terminology already, with
> >>> limits/thresholds and fail/abort.
> >>>
> >>> It’s a lot of work to come up with a coherent and intuitive config
> >> layout.
> >>> We probably want to at least create some documentation in-tree
> >> stipulating
> >>> terminology with respect to plurals, verbs/nouns, and specific terms
> >>> (period, abort, limit, datacenter vs dc, etc), but ideally we would
> have
> >> a
> >>> common end goal for the config file.
> >>>
>  leave non-features to 

Re: [DISCUSS] Releasable trunk and quality

[Ekaterina Dimitrova]

I think the script discussion is on a different thread and attached
document which I am also about to address soon :-)

On Mon, 6 Dec 2021 at 17:59, bened...@apache.org 
wrote:

> Is there a reason we discounted modifying the merge strategy?
>
> I’m just a little wary of relying on scripts for consistency of behaviour
> here. Environments differ, and it would be far preferable for consistency
> of behaviour to rely on shared infrastructure if possible. I would probably
> be against mandating these scripts, at least.
>
> From: Joshua McKenzie 
> Date: Monday, 6 December 2021 at 22:20
> To: dev@cassandra.apache.org 
> Subject: Re: [DISCUSS] Releasable trunk and quality
> As I work through the scripting on this, I don't know if we've documented
> or clarified the following (don't see it here:
> https://cassandra.apache.org/_/development/testing.html):
>
> Pre-commit test suites:
> * Which JDK's?
> * When to include all python tests or do JVM only (if ever)?
> * When to run upgrade tests?
> * What to do if a test is also failing on the reference root (i.e. trunk,
> cassandra-4.0, etc)?
> * What to do if a test fails intermittently?
>
> I'll also update the above linked documentation once we hammer this out and
> try and bake it into the scripting flow as much as possible as well. Goal
> is to make it easy to do the right thing and hard to do the wrong thing,
> and to have these things written down rather than have it be tribal
> knowledge that varies a lot across the project.
>
> ~Josh
>
> On Sat, Dec 4, 2021 at 9:04 AM Joshua McKenzie 
> wrote:
>
> > After some offline collab, here's where this thread has landed on a
> > proposal to change our processes to incrementally improve our processes
> and
> > hopefully stabilize the state of CI longer term:
> >
> > Link:
> >
> https://docs.google.com/document/d/1tJ-0K7d6PIStSbNFOfynXsD9RRDaMgqCu96U4O-RT84/edit#bookmark=id.16oxqq30bby4
> >
> > Hopefully the mail server doesn't butcher formatting; if it does, hit up
> > the gdoc and leave comments there as should be open to all.
> >
> > Phase 1:
> > Document merge criteria; update circle jobs to have a simple pre-merge
> job
> > (one for each JDK profile)
> >  * Donate, document, and formalize usage of circleci-enable.py in ASF
> > repo (need new commit scripts / dev tooling section?)
> > * rewrites circle config jobs to simple clear flow
> > * ability to toggle between "run on push" or "click to run"
> > * Variety of other functionality; see below
> > Document (site, help, README.md) and automate via scripting the
> > relationship / dev / release process around:
> > * In-jvm dtest
> > * dtest
> > * ccm
> > Integrate and document usage of script to build CI repeat test runs
> > * circleci-enable.py --repeat-unit org.apache.cassandra.SomeTest
> > * Document “Do this if you add or change tests”
> > Introduce “Build Lead” role
> > * Weekly rotation; volunteer
> > * 1: Make sure JIRAs exist for test failures
> > * 2: Attempt to triage new test failures to root cause and assign out
> > * 3: Coordinate and drive to green board on trunk
> > Change and automate process for *trunk only* patches:
> > * Block on green CI (from merge criteria in CI above; potentially
> > stricter definition of "clean" for trunk CI)
> > * Consider using github PR’s to merge (TODO: determine how to handle
> > circle + CHANGES; see below)
> > Automate process for *multi-branch* merges
> > * Harden / contribute / document dcapwell script (has one which does
> > the following):
> > * rebases your branch to the latest (if on 3.0 then rebase
> against
> > cassandra-3.0)
> > * check compiles
> > * removes all changes to .circle (can opt-out for circleci
> patches)
> > * removes all changes to CHANGES.txt and leverages JIRA for the
> > content
> > * checks code still compiles
> > * changes circle to run ci
> > * push to a temp branch in git and run CI (circle + Jenkins)
> > * when all branches are clean (waiting step is manual)
> > * TODO: Define “clean”
> > * No new test failures compared to reference?
> > * Or no test failures at all?
> > * merge changes into the actual branches
> > * merge up changes; rewriting diff
> > * push --atomic
> >
> > Transition to phase 2 when:
> > * All items from phase 1 are complete
> > * Test boards for supported branches are green
> >
> > Phase 2:
> > * Add Harry to recurring run against trunk
> > * Add Harry to release pipeline
> > * Suite of perf tests against trunk recurring
> >
> >
> >
> > On Wed, Nov 17, 2021 at 1:42 PM Joshua McKenzie 
> > wrote:
> >
> >> Sorry for not catching that Benedict, you're absolutely right. So long
> as
> >> we're using merge commits between branches I don't think auto-merging
> via
> >> train or blocking on green CI are options via the tooling, and
> 

Re: [DISCUSS] Releasable trunk and quality

[bened...@apache.org]

Is there a reason we discounted modifying the merge strategy?

I’m just a little wary of relying on scripts for consistency of behaviour here. 
Environments differ, and it would be far preferable for consistency of 
behaviour to rely on shared infrastructure if possible. I would probably be 
against mandating these scripts, at least.

From: Joshua McKenzie 
Date: Monday, 6 December 2021 at 22:20
To: dev@cassandra.apache.org 
Subject: Re: [DISCUSS] Releasable trunk and quality
As I work through the scripting on this, I don't know if we've documented
or clarified the following (don't see it here:
https://cassandra.apache.org/_/development/testing.html):

Pre-commit test suites:
* Which JDK's?
* When to include all python tests or do JVM only (if ever)?
* When to run upgrade tests?
* What to do if a test is also failing on the reference root (i.e. trunk,
cassandra-4.0, etc)?
* What to do if a test fails intermittently?

I'll also update the above linked documentation once we hammer this out and
try and bake it into the scripting flow as much as possible as well. Goal
is to make it easy to do the right thing and hard to do the wrong thing,
and to have these things written down rather than have it be tribal
knowledge that varies a lot across the project.

~Josh

On Sat, Dec 4, 2021 at 9:04 AM Joshua McKenzie  wrote:

> After some offline collab, here's where this thread has landed on a
> proposal to change our processes to incrementally improve our processes and
> hopefully stabilize the state of CI longer term:
>
> Link:
> https://docs.google.com/document/d/1tJ-0K7d6PIStSbNFOfynXsD9RRDaMgqCu96U4O-RT84/edit#bookmark=id.16oxqq30bby4
>
> Hopefully the mail server doesn't butcher formatting; if it does, hit up
> the gdoc and leave comments there as should be open to all.
>
> Phase 1:
> Document merge criteria; update circle jobs to have a simple pre-merge job
> (one for each JDK profile)
>  * Donate, document, and formalize usage of circleci-enable.py in ASF
> repo (need new commit scripts / dev tooling section?)
> * rewrites circle config jobs to simple clear flow
> * ability to toggle between "run on push" or "click to run"
> * Variety of other functionality; see below
> Document (site, help, README.md) and automate via scripting the
> relationship / dev / release process around:
> * In-jvm dtest
> * dtest
> * ccm
> Integrate and document usage of script to build CI repeat test runs
> * circleci-enable.py --repeat-unit org.apache.cassandra.SomeTest
> * Document “Do this if you add or change tests”
> Introduce “Build Lead” role
> * Weekly rotation; volunteer
> * 1: Make sure JIRAs exist for test failures
> * 2: Attempt to triage new test failures to root cause and assign out
> * 3: Coordinate and drive to green board on trunk
> Change and automate process for *trunk only* patches:
> * Block on green CI (from merge criteria in CI above; potentially
> stricter definition of "clean" for trunk CI)
> * Consider using github PR’s to merge (TODO: determine how to handle
> circle + CHANGES; see below)
> Automate process for *multi-branch* merges
> * Harden / contribute / document dcapwell script (has one which does
> the following):
> * rebases your branch to the latest (if on 3.0 then rebase against
> cassandra-3.0)
> * check compiles
> * removes all changes to .circle (can opt-out for circleci patches)
> * removes all changes to CHANGES.txt and leverages JIRA for the
> content
> * checks code still compiles
> * changes circle to run ci
> * push to a temp branch in git and run CI (circle + Jenkins)
> * when all branches are clean (waiting step is manual)
> * TODO: Define “clean”
> * No new test failures compared to reference?
> * Or no test failures at all?
> * merge changes into the actual branches
> * merge up changes; rewriting diff
> * push --atomic
>
> Transition to phase 2 when:
> * All items from phase 1 are complete
> * Test boards for supported branches are green
>
> Phase 2:
> * Add Harry to recurring run against trunk
> * Add Harry to release pipeline
> * Suite of perf tests against trunk recurring
>
>
>
> On Wed, Nov 17, 2021 at 1:42 PM Joshua McKenzie 
> wrote:
>
>> Sorry for not catching that Benedict, you're absolutely right. So long as
>> we're using merge commits between branches I don't think auto-merging via
>> train or blocking on green CI are options via the tooling, and multi-branch
>> reverts will be something we should document very clearly should we even
>> choose to go that route (a lot of room to make mistakes there).
>>
>> It may not be a huge issue as we can expect the more disruptive changes
>> (i.e. potentially destabilizing) to be happening on trunk only, so perhaps
>> we can get away with slightly different workflows or policies based on
>> whether you're 

Re: [DISCUSS] Releasable trunk and quality

[Joshua McKenzie]

As I work through the scripting on this, I don't know if we've documented
or clarified the following (don't see it here:
https://cassandra.apache.org/_/development/testing.html):

Pre-commit test suites:
* Which JDK's?
* When to include all python tests or do JVM only (if ever)?
* When to run upgrade tests?
* What to do if a test is also failing on the reference root (i.e. trunk,
cassandra-4.0, etc)?
* What to do if a test fails intermittently?

I'll also update the above linked documentation once we hammer this out and
try and bake it into the scripting flow as much as possible as well. Goal
is to make it easy to do the right thing and hard to do the wrong thing,
and to have these things written down rather than have it be tribal
knowledge that varies a lot across the project.

~Josh

On Sat, Dec 4, 2021 at 9:04 AM Joshua McKenzie  wrote:

> After some offline collab, here's where this thread has landed on a
> proposal to change our processes to incrementally improve our processes and
> hopefully stabilize the state of CI longer term:
>
> Link:
> https://docs.google.com/document/d/1tJ-0K7d6PIStSbNFOfynXsD9RRDaMgqCu96U4O-RT84/edit#bookmark=id.16oxqq30bby4
>
> Hopefully the mail server doesn't butcher formatting; if it does, hit up
> the gdoc and leave comments there as should be open to all.
>
> Phase 1:
> Document merge criteria; update circle jobs to have a simple pre-merge job
> (one for each JDK profile)
>  * Donate, document, and formalize usage of circleci-enable.py in ASF
> repo (need new commit scripts / dev tooling section?)
> * rewrites circle config jobs to simple clear flow
> * ability to toggle between "run on push" or "click to run"
> * Variety of other functionality; see below
> Document (site, help, README.md) and automate via scripting the
> relationship / dev / release process around:
> * In-jvm dtest
> * dtest
> * ccm
> Integrate and document usage of script to build CI repeat test runs
> * circleci-enable.py --repeat-unit org.apache.cassandra.SomeTest
> * Document “Do this if you add or change tests”
> Introduce “Build Lead” role
> * Weekly rotation; volunteer
> * 1: Make sure JIRAs exist for test failures
> * 2: Attempt to triage new test failures to root cause and assign out
> * 3: Coordinate and drive to green board on trunk
> Change and automate process for *trunk only* patches:
> * Block on green CI (from merge criteria in CI above; potentially
> stricter definition of "clean" for trunk CI)
> * Consider using github PR’s to merge (TODO: determine how to handle
> circle + CHANGES; see below)
> Automate process for *multi-branch* merges
> * Harden / contribute / document dcapwell script (has one which does
> the following):
> * rebases your branch to the latest (if on 3.0 then rebase against
> cassandra-3.0)
> * check compiles
> * removes all changes to .circle (can opt-out for circleci patches)
> * removes all changes to CHANGES.txt and leverages JIRA for the
> content
> * checks code still compiles
> * changes circle to run ci
> * push to a temp branch in git and run CI (circle + Jenkins)
> * when all branches are clean (waiting step is manual)
> * TODO: Define “clean”
> * No new test failures compared to reference?
> * Or no test failures at all?
> * merge changes into the actual branches
> * merge up changes; rewriting diff
> * push --atomic
>
> Transition to phase 2 when:
> * All items from phase 1 are complete
> * Test boards for supported branches are green
>
> Phase 2:
> * Add Harry to recurring run against trunk
> * Add Harry to release pipeline
> * Suite of perf tests against trunk recurring
>
>
>
> On Wed, Nov 17, 2021 at 1:42 PM Joshua McKenzie 
> wrote:
>
>> Sorry for not catching that Benedict, you're absolutely right. So long as
>> we're using merge commits between branches I don't think auto-merging via
>> train or blocking on green CI are options via the tooling, and multi-branch
>> reverts will be something we should document very clearly should we even
>> choose to go that route (a lot of room to make mistakes there).
>>
>> It may not be a huge issue as we can expect the more disruptive changes
>> (i.e. potentially destabilizing) to be happening on trunk only, so perhaps
>> we can get away with slightly different workflows or policies based on
>> whether you're doing a multi-branch bugfix or a feature on trunk. Bears
>> thinking more deeply about.
>>
>> I'd also be game for revisiting our merge strategy. I don't see much
>> difference in labor between merging between branches vs. preparing separate
>> patches for an individual developer, however I'm sure there's maintenance
>> and integration implications there I'm not thinking of right now.
>>
>> On Wed, Nov 17, 2021 at 12:03 PM bened...@apache.org 
>> wrote:
>>
>>> I raised this 

Re: [DISCUSS] Disabling MIME-part filtering on this mailing list

[Brandon Williams]

+1

On Mon, Dec 6, 2021 at 5:00 AM Aleksey Yeschenko  wrote:
>
> Agreed as well.
>
> > On 4 Dec 2021, at 23:28, bened...@apache.org wrote:
> >
> > I think you were correct to start a DISCUSS thread, Bowen. You should not 
> > start a vote until you have first established if there is consensus.
> >
> > Also, I agree with the proposal.
> >
> > From: Bowen Song 
> > Date: Saturday, 4 December 2021 at 23:12
> > To: dev@cassandra.apache.org 
> > Subject: Re: [DISCUSS] Disabling MIME-part filtering on this mailing list
> > Hmm.. It's too late to change that. I opened it as "DISCUSS" because I
> > was not sure if the information in it is enough for people to vote on
> > it. There's clearly a lot more can be asked or discussed. For example
> > the change will also stop the "To unsubscribe, ..." footer being
> > appended to some emails (but not this one), and that may cause some
> > difficulties for some people to find the unsubscribe link/address.
> >
> > Hint #1: you can find the unsubscribe button where you found
> >  the subscribe button.
> >
> > Hint #2: you can also find the unsubscribe address in the email header
> > "List-Unsubscribe"
> >
> > Hint #3: failing all the above, you can also ask in the list. I'm sure
> > other people will help.
> >
> > On 04/12/2021 22:31, Mick Semb Wever wrote:
> >>> What's your opinion on this? Do you support or oppose disabling the
> >>> MIME-part filtering on the Cassandra-dev mailing list?
> >>
> >> +1
> >>
> >> (nit: am thinking the thread should have had a [VOTE] prefix, since you are
> >> calling out for consensus)
> >>
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: [DISCUSS] Disabling MIME-part filtering on this mailing list

[Ekaterina Dimitrova]

+1, thank you for looking into this

On Mon, 6 Dec 2021 at 6:15, Benjamin Lerer  wrote:

> +1
>
> Le lun. 6 déc. 2021 à 12:00, Aleksey Yeschenko  a
> écrit :
>
> > Agreed as well.
> >
> > > On 4 Dec 2021, at 23:28, bened...@apache.org wrote:
> > >
> > > I think you were correct to start a DISCUSS thread, Bowen. You should
> > not start a vote until you have first established if there is consensus.
> > >
> > > Also, I agree with the proposal.
> > >
> > > From: Bowen Song 
> > > Date: Saturday, 4 December 2021 at 23:12
> > > To: dev@cassandra.apache.org 
> > > Subject: Re: [DISCUSS] Disabling MIME-part filtering on this mailing
> list
> > > Hmm.. It's too late to change that. I opened it as "DISCUSS" because I
> > > was not sure if the information in it is enough for people to vote on
> > > it. There's clearly a lot more can be asked or discussed. For example
> > > the change will also stop the "To unsubscribe, ..." footer being
> > > appended to some emails (but not this one), and that may cause some
> > > difficulties for some people to find the unsubscribe link/address.
> > >
> > > Hint #1: you can find the unsubscribe button where you found
> > >  the subscribe button.
> > >
> > > Hint #2: you can also find the unsubscribe address in the email header
> > > "List-Unsubscribe"
> > >
> > > Hint #3: failing all the above, you can also ask in the list. I'm sure
> > > other people will help.
> > >
> > > On 04/12/2021 22:31, Mick Semb Wever wrote:
> > >>> What's your opinion on this? Do you support or oppose disabling the
> > >>> MIME-part filtering on the Cassandra-dev mailing list?
> > >>
> > >> +1
> > >>
> > >> (nit: am thinking the thread should have had a [VOTE] prefix, since
> you
> > are
> > >> calling out for consensus)
> > >>
> >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >
>

Re: [DISCUSS] Disabling MIME-part filtering on this mailing list

[Aleksey Yeschenko]

Agreed as well.

> On 4 Dec 2021, at 23:28, bened...@apache.org wrote:
> 
> I think you were correct to start a DISCUSS thread, Bowen. You should not 
> start a vote until you have first established if there is consensus.
> 
> Also, I agree with the proposal.
> 
> From: Bowen Song 
> Date: Saturday, 4 December 2021 at 23:12
> To: dev@cassandra.apache.org 
> Subject: Re: [DISCUSS] Disabling MIME-part filtering on this mailing list
> Hmm.. It's too late to change that. I opened it as "DISCUSS" because I
> was not sure if the information in it is enough for people to vote on
> it. There's clearly a lot more can be asked or discussed. For example
> the change will also stop the "To unsubscribe, ..." footer being
> appended to some emails (but not this one), and that may cause some
> difficulties for some people to find the unsubscribe link/address.
> 
> Hint #1: you can find the unsubscribe button where you found
>  the subscribe button.
> 
> Hint #2: you can also find the unsubscribe address in the email header
> "List-Unsubscribe"
> 
> Hint #3: failing all the above, you can also ask in the list. I'm sure
> other people will help.
> 
> On 04/12/2021 22:31, Mick Semb Wever wrote:
>>> What's your opinion on this? Do you support or oppose disabling the
>>> MIME-part filtering on the Cassandra-dev mailing list?
>> 
>> +1
>> 
>> (nit: am thinking the thread should have had a [VOTE] prefix, since you are
>> calling out for consensus)
>> 


-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org