I'm new to this project and here are my two cents. If there are tests that are constantly failing or flaky and you have had releases despite their failures, then they're not useful and can be disabled. They can always be reenabled if they are in fact valuable. Having 100% blue dashboard is not idealistic IMHO. Hardware failures are harder but they can be addressed too. I could pitch in to fix the noisy tests or just help in other ways to get the dashboard to blue. Dinesh On Thursday, February 15, 2018, 1:14:33 PM PST, Josh McKenzie <jmcken...@apache.org> wrote: > > We’ve said in the past that we don’t release without green tests. The PMC > gets to vote and enforce it. If you don’t vote yes without seeing the test > results, that enforces it.
I think this is noble and ideal in theory. In practice, the tests take long enough, hardware infra has proven flaky enough, and the tests *themselves* flaky enough, that there's been a consistent low-level of test failure noise that makes separating signal from noise in this context very time consuming. Reference 3.11-test-all for example re:noise: https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-3.11-test-all/test/?width=1024&height=768 Having spearheaded burning test failures to 0 multiple times and have them regress over time, my gut intuition is we should have one person as our Source of Truth with a less-flaky source for release-vetting CI (dedicated hardware, circle account, etc) we can use as a reference to vote on release SHA's. We’ve declared this a requirement multiple times Declaring things != changed behavior, and thus != changed culture. The culture on this project is one of having a constant low level of test failure noise in our CI as a product of our working processes. Unless we change those (actually block release w/out green board, actually aggressively block merge w/any failing tests, aggressively retroactively track down test failures on a daily basis and RCA), the situation won't improve. Given that this is a volunteer organization / project, that kind of daily time investment is a big ask. On Thu, Feb 15, 2018 at 1:10 PM, Jeff Jirsa <jji...@gmail.com> wrote: > Moving this to it’s own thread: > > We’ve declared this a requirement multiple times and then we occasionally > get a critical issue and have to decide whether it’s worth the delay. I > assume Jason’s earlier -1 on attempt 1 was an enforcement of that earlier > stated goal. > > It’s up to the PMC. We’ve said in the past that we don’t release without > green tests. The PMC gets to vote and enforce it. If you don’t vote yes > without seeing the test results, that enforces it. > > -- > Jeff Jirsa > > > > On Feb 15, 2018, at 9:49 AM, Josh McKenzie <jmcken...@apache.org> wrote: > > > > What would it take for us to get green utest/dtests as a blocking part of > > the release process? i.e. "for any given SHA, here's a link to the tests > > that passed" in the release vote email? > > > > That being said, +1. > > > >> On Wed, Feb 14, 2018 at 4:33 PM, Nate McCall <zznat...@gmail.com> > wrote: > >> > >> +1 > >> > >> On Thu, Feb 15, 2018 at 9:40 AM, Michael Shuler <mich...@pbandjelly.org > > > >> wrote: > >>> I propose the following artifacts for release as 3.0.16. > >>> > >>> sha1: 890f319142ddd3cf2692ff45ff28e71001365e96 > >>> Git: > >>> http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a= > >> shortlog;h=refs/tags/3.0.16-tentative > >>> Artifacts: > >>> https://repository.apache.org/content/repositories/ > >> orgapachecassandra-1157/org/apache/cassandra/apache-cassandra/3.0.16/ > >>> Staging repository: > >>> https://repository.apache.org/content/repositories/ > >> orgapachecassandra-1157/ > >>> > >>> Debian and RPM packages are available here: > >>> http://people.apache.org/~mshuler > >>> > >>> *** This release addresses an important fix for CASSANDRA-14092 *** > >>> "Max ttl of 20 years will overflow localDeletionTime" > >>> https://issues.apache.org/jira/browse/CASSANDRA-14092 > >>> > >>> The vote will be open for 72 hours (longer if needed). > >>> > >>> [1]: (CHANGES.txt) https://goo.gl/rLj59Z > >>> [2]: (NEWS.txt) https://goo.gl/EkrT4G > >>> > >>> --------------------------------------------------------------------- > >>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > >>> For additional commands, e-mail: dev-h...@cassandra.apache.org > >>> > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > >> For additional commands, e-mail: dev-h...@cassandra.apache.org > >> > >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org > >
