Hi,
I created https://issues.apache.org/jira/browse/CASSANDRA-14241 for this issue.
You are right there is a solid chunk of failing tests on Apache infrastructure
that don't fail on CircleCI. I'll find someone to get it done.
I think that fix before commit is only going to happen if we go all t
It stands out to me that Google a) does pre-submit test runs, b) has a
dedicated team to identify and work with flaky tests, c) has some kind of
quarantining, and d) doesn't report on a test being flaky unless it fails
3x in a row if it's marked as flaky. While we obviously can't pursue b
since we'
Hi,
I'm ecstatic others are now running the tests and, more importantly, that
we're having the conversation.
I've become convinced we cannot always have 100% green tests. I am reminded
of this [1] blog post from Google when thinking about flaky tests.
The TL;DR is "flakiness happens", to the tune
I'm new to this project and here are my two cents.
If there are tests that are constantly failing or flaky and you have had
releases despite their failures, then they're not useful and can be disabled.
They can always be reenabled if they are in fact valuable. Having 100% blue
dashboard is not i
cassandra.apache.org
Subject: Re: Release votes
It seems there has been a bit of a slip in testing as of recently, mostly due
to the fact that there's no canonical testing environment that isn't flaky. We
probably need to come up with some ideas and a plan on how we're going to do
t
It seems there has been a bit of a slip in testing as of recently, mostly
due to the fact that there's no canonical testing environment that isn't
flaky. We probably need to come up with some ideas and a plan on how we're
going to do testing in the future, and how we're going to make testing
access
>
> We’ve said in the past that we don’t release without green tests. The PMC
> gets to vote and enforce it. If you don’t vote yes without seeing the test
> results, that enforces it.
I think this is noble and ideal in theory. In practice, the tests take long
enough, hardware infra has proven flak
Moving this to it’s own thread:
We’ve declared this a requirement multiple times and then we occasionally get a
critical issue and have to decide whether it’s worth the delay. I assume
Jason’s earlier -1 on attempt 1 was an enforcement of that earlier stated goal.
It’s up to the PMC. We’ve sai
