Thanks for opening an epic @Jacek.
It seems the dtest_offheap job is replaced by dtest_latest which means we
will have the same amount of jobs after the current ticket and I am not
worried about Jenkins.
Though in CircleCI we did not have the dtest_offheap job mandatory run
pre-commit but as far
I’m +1 once the tests are passing ands +0 while they are failingSent from my iPhoneOn Feb 16, 2024, at 6:08 AM, Paulo Motta wrote:Thanks for clarifying Branimir! I'm +1 on proceeding as proposed and I think this change will make it easier to gain confidence to update configurations.Interesting
Thanks for clarifying Branimir! I'm +1 on proceeding as proposed and I
think this change will make it easier to gain confidence to update
configurations.
Interesting discussion and suggestions on this thread - I think we can
follow-up on improving test/CI workflow in a different thread/proposal
We should conclude this discussion by answering Branimir's original
question.* I vote for merging that and exposing issues to the CI.*
For pre-commit optimization I've opened
https://issues.apache.org/jira/browse/CASSANDRA-19406 epic and we should
add exact tasks there to make this valuable
I love the idea of David to make this dual config stuff directly the part
of the tests, I just leave this here where I quickly put some super
primitive runner together
https://github.com/smiklosovic/cassandra/commit/693803772218b52c424491b826c704811d606a31
We could just run by default with one
>
> Perhaps this was needed during j17 stabilization but is no longer required?
No, I only switched from tests running J8+J11 to tests running J11+J17.
What we tested was something decided in the 4.0 era when JDK11 was added,
and I was not even part of the community yet :-)
> Any known
This thread got large quick, yay!
>> is there a reason all guardrails and reliability (aka repair retries)
>> configs are off by default? They are off by default in the normal config
>> for backwards compatibility reasons, but if we are defining a config saying
>> what we recommend, we should
This goes back to what I was talking about previously, we might even run
tests for both j11 and j17 e.g. in Circle _but only on a selected set of
tests_ where there is some kind of a "tension" between the code and Java
version, whatever it means. Like Chronicle queues or BTrees etc, I merely
For the sake of argument, if we picked one, would you (anyone, not just
Stefan) be OK with the JVM that's selected being the one you don't use?
I'm willing to bet most folks will have a preference for the JVM they run
in production. I think both should be run on some frequent basis but I can
I don't think there is as simple a way to identify those since there
are many ways you can specify a single token.
Kind Regards,
Brandon
On Thu, Feb 15, 2024 at 11:45 AM Jacek Lewandowski
wrote:
>
> Brandon, that should be doable with the current filters I think - that is,
> select only
Only requiring building on supported JDKs and running all tests only on a
pre-defined version is definitely an improvement in terms of build time.
Building it is cheap, one worker and 5 minutes.
As I already said, just want to reiterate that, instead of _running with
all Java's_ we might run with
Brandon, that should be doable with the current filters I think - that is,
select only those tests which do not support vnodes. Do you know about such
in-jvm dtests as well?
- - -- --- - -
Jacek Lewandowski
czw., 15 lut 2024 o 18:21 Brandon Williams napisał(a):
> On
On Thu, Feb 15, 2024 at 1:10 AM Jacek Lewandowski
wrote:
> For dtests we have vnodes/no-vnodes, offheap/onheap, and nothing about other
> stuff. To me running no-vnodes makes no sense because no-vnodes is just a
> special case of vnodes=1. On the other hand offheap/onheap buffers could be
>
Great summary Josh,
>
>- JDK-based test suites on highest supported jdk using other config
>
> Do you mean a smoke test suite by that ^ ?
- - -- --- - -
Jacek Lewandowski
czw., 15 lut 2024 o 18:12 Josh McKenzie napisał(a):
> Would it make sense to only block
> Would it make sense to only block commits on the test strategy you've listed,
> and shift the entire massive test suite to post-commit?
> Lots and lots of other emails
;)
There's an interesting broad question of: What config do we consider
"recommended" going forward, the "conservative"
Would it make sense to only block commits on the test strategy you've
listed, and shift the entire massive test suite to post-commit? If there
really is only a small % of times the entire suite is useful this seems
like it could unblock the dev cycle but still have the benefit of the full
test
Paulo:
> 1) Will cassandra.yaml remain the default test config? Is the plan moving
> forward to require green CI for both configurations on pre-commit, or
> pre-release?
The plan is to ensure both configurations are green pre-commit. This should
not increase the CI cost as this replaces extra
> It's also been questioned about why we don't just enable settings we
recommend. These are settings we recommend for new clusters. *Our
existing cassandra.yaml needs to be tailored for existing clusters being
upgraded, where we are very conservative about changing defaults.*
I think this
On reducing circle ci usage during dev while iterating, not with the
intention to replace the pre-commit CI (yet), we could do away with
testing only dtests, jvm-dtests, units and cqlsh for a _single_
configuration imo. That would greatly reduce usage. I hacked it quickly
here for illustration
On the merging failing tests discussion I _do_ spend the time looking if
my patch did cause them or not and I certainly enforce that in the
reviews I do. The current failures are a manageable number to check
against Butler/Jenkins/Circle/Jira so I was under the impression
everybody else was
> Mick and Ekaterina (and everyone really) - any thoughts on what test
> coverage, if any, we should commit to for this new configuration?
> Acknowledging that we already have *a lot* of CI that we run.
>
Branimir in this patch has already done some basic cleanup of test
variations, so this is
I fully understand you. Although I have that luxury to use more containers,
I simply feel that rerunning the same code with different configurations
which do not impact that code is just a waste of resources and money.
- - -- --- - -
Jacek Lewandowski
czw., 15 lut 2024
By the way, I am not sure if it is all completely transparent and
understood by everybody but let me guide you through a typical patch which
is meant to be applied from 4.0 to trunk (4 branches) to see how it looks
like.
I do not have the luxury of running CircleCI on 100 containers, I have just
>
> Excellent point, I was saying for some time that IMHO we can reduce
> to running in CI at least pre-commit:
> 1) Build J11 2) build J17
> 3) run tests with build 11 + runtime 11
> 4) run tests with build 11 and runtime 17.
Ekaterina, I was thinking more about:
1) build J11
2) build J17
3)
Something along what Paulo is proposing makes sense to me. To sum it up,
knowing what workflows we have now:
java17_pre-commit_tests
java11_pre-commit_tests
java17_separate_tests
java11_separate_tests
We would have couple more, together like:
java17_pre-commit_tests
> Perhaps it is also a good opportunity to distinguish subsets of tests
which make sense to run with a configuration matrix.
Agree. I think we should define a “standard/golden” configuration for each
branch and minimally require precommit tests for that configuration.
Assignees and reviewers can
> If there’s an “old compatible default” and “latest recommended settings”,
when does the value in “old compatible default” get updated? Never?
How about replacing cassandra.yaml with cassandra_latest.yaml on trunk when
cutting cassandra-6.0 branch? Any new default changes on trunk go to
I share Jacek’s and Stefan’s sentiment about the low value of requiring
precommit j11+j17 tests for all changes.
Perhaps this was needed during j17 stabilization but is no longer required?
Please correct if I’m missing some context.
To have a practical proposal to address this, how about:
1)
Jon,
I was mostly referring to Circle CI where we have two pre-commit workflows.
(just click on anything here
https://app.circleci.com/pipelines/github/instaclustr/cassandra)
java17_pre-commit_tests
This workflow is compiling & testing everything with Java 17
java11_pre-commit_tests
This
>
> I'm ok with breaking trunk CI temporarily as long as failures are tracked
> and triaged/addressed before the next release.
>From the ticket, I understand it is meant for 5.0-rc
I share this sentiment for the release we decide to ship with:
> The failures should block release or we should
Stefan, can you elaborate on what you are proposing? It's not clear (at
least to me) what level of testing you're advocating for. Dropping testing
both on dev branches, every commit, just on release? In addition, can you
elaborate on what is a hassle about it? It's been a long time since I
I agree with Jacek, I don't quite understand why we are running the
pipeline for j17 and j11 every time. I think this should be opt-in.
Majority of the time, we are just refactoring and coding stuff for
Cassandra where testing it for both jvms is just pointless and we _know_
that it will be fine
śr., 14 lut 2024 o 17:30 Josh McKenzie napisał(a):
> When we have failing tests people do not spend the time to figure out if
> their logic caused a regression and merge, making things more unstable… so
> when we merge failing tests that leads to people merging even more failing
> tests...
>
>
1) If there’s an “old compatible default” and “latest recommended settings”,
when does the value in “old compatible default” get updated? Never?
2) If there are test failures with the new values, it seems REALLY IMPORTANT to
make sure those test failures are discovered + fixed IN THE FUTURE
Cool stuff! This will make it easier to advance configuration defaults
without affecting stable configuration.
Wording looks good to me. +1 to include a NEWS.txt note. I'm ok with
breaking trunk CI temporarily as long as failures are tracked and
triaged/addressed before the next release.
I
> When we have failing tests people do not spend the time to figure out if
> their logic caused a regression and merge, making things more unstable… so
> when we merge failing tests that leads to people merging even more failing
> tests...
What's the counter position to this Jacek / Berenguer?
+1 to not doing, imo, the ostrich lol
On 14/2/24 10:58, Jacek Lewandowski wrote:
We should not block merging configuration changes given it is a valid
configuration - which I understand as it is correct, passes all config
validations, it matches documented rules, etc. And this provided
latest
We should not block merging configuration changes given it is a valid
configuration - which I understand as it is correct, passes all config
validations, it matches documented rules, etc. And this provided latest
config matches those requirements I assume.
The failures should block release or we
Wording looks good to me. I would also put that into NEWS.txt but I am not
sure what section. New features, Upgrading nor Deprecation does not seem to
be a good category.
On Tue, Feb 13, 2024 at 5:42 PM Branimir Lambov wrote:
> Hi All,
>
> CASSANDRA-18753 introduces a second set of defaults (in
is there a reason all guardrails and reliability (aka repair retries)
configs are off by default? They are off by default in the normal config
for backwards compatibility reasons, but if we are defining a config saying
what we recommend, we should enable these things by default IMO.
This is one
> so can cause repairs to deadlock forever
Small correction, I finished fixing the tests in CASSANDRA-19042 and we don’t
deadlock, we timeout and fail repair if any of those messages are dropped.
> On Feb 13, 2024, at 11:04 AM, David Capwell wrote:
>
>> and to point potential users that are
> and to point potential users that are evaluating the technology to an
> optimized set of defaults
Left this comment in the GH… is there a reason all guardrails and reliability
(aka repair retries) configs are off by default? They are off by default in
the normal config for backwards
Hi All,
CASSANDRA-18753 introduces a second set of defaults (in a separate
"cassandra_latest.yaml") that enable new features of Cassandra. The
objective is two-fold: to be able to test the database in this
configuration, and to point potential users that are evaluating the
technology to an
43 matches
Mail list logo
