[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[ https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17687488#comment-17687488 ] Ekaterina Dimitrova commented on CASSANDRA-18001: - To https://github.com/apache/cassandra.git [fa18ea1852..476d31bce2|https://github.com/apache/cassandra/commit/476d31bce2597cf71cdb7dd446d5cac28ee69a82] cassandra-3.0 -> cassandra-3.0 [74b0b2086f..788b4eca2d|https://github.com/apache/cassandra/commit/788b4eca2d0b5f3d508daf9d1b08426f075b8fca] cassandra-3.11 -> cassandra-3.11 [32c56df067..cb4f90542d|https://github.com/apache/cassandra/commit/cb4f90542d005c0cacc7cbc464e5edb2fd748b60] cassandra-4.0 -> cassandra-4.0 [3e0d38dab2..f345ee8a4e|https://github.com/apache/cassandra/commit/f345ee8a4e7c8c2dc495b6282805d15acf35b737] cassandra-4.1 -> cassandra-4.1 [a8465376f7..ff6ad4e48a|https://github.com/apache/cassandra/commit/ff6ad4e48a04e5f78215da287fcf037ecb556016] trunk -> trunk Thank you [~adelapena] for all your help! Also, for completeness - further to getting rid of the patch warnings, when we run the generate.sh script, the patch also removed the repeated jobs from MIDRES and HIGHRES files as they were re-added by mistake in CASSANDRA-18094. > Add missing tests suites to CircleCI > > > Key: CASSANDRA-18001 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18001 > Project: Cassandra > Issue Type: Task > Components: CI >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Urgent > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1-rc1, 4.1, 4.1.x, 4.2, 4.x > > > Large Python DTests (with/without vnodes), cqlshlib not tested in all > branches and with all jdks; Java distributed tests not running with J8/J11 > 4.0+ > A few bugs identified down the road have been fixed. More in the comments -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[
https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17687270#comment-17687270
]
Ekaterina Dimitrova commented on CASSANDRA-18001:
-
{quote}In any case, I think we are done with this one and can follow on
CASSANDRA-18051, +1.
{quote}
Thanks, I will commit it over the weekend.
{quote}That's interesting! But the two infamous tests fail with L containers in
all branches, right?
{quote}
Yes, I will push more tests for those two on the repeatable runner and move the
discussion on the other ticket and edit it to be for a fix...
I see also some errors which might be timing
> Add missing tests suites to CircleCI
>
>
> Key: CASSANDRA-18001
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18001
> Project: Cassandra
> Issue Type: Task
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Urgent
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1-rc1, 4.1, 4.1.x, 4.2, 4.x
>
>
> Burn tests to all branches, large Python DTests (with/without vnodes),
> cqlshlib not tested in all branches and with all jdks; Java distributed tests
> not running with J8/J11 4.0+
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[ https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17686376#comment-17686376 ] Andres de la Peña commented on CASSANDRA-18001: --- bq. 3.0 and 3.11 with HIGH Resources XLarge containers and parallelism 1 fail the two tests we discussed. From what I see I am not sure it is a resource issue though. That's interesting! But the two infamous tests fail with L containers in all branches, right? So we have a branch-specific bug on top a general resource limitations addressed here with the XL containers. We'll have to investigate why those fail and maybe also consider the simplification to less DCs, if those two things are not related. In any case, I think we are done with this one and can follow on CASSANDRA-180151, +1. > Add missing tests suites to CircleCI > > > Key: CASSANDRA-18001 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18001 > Project: Cassandra > Issue Type: Task > Components: CI >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Urgent > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1-rc1, 4.1, 4.1.x, 4.2, 4.x > > > Burn tests to all branches, large Python DTests (with/without vnodes), > cqlshlib not tested in all branches and with all jdks; Java distributed tests > not running with J8/J11 4.0+ -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[
https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17686040#comment-17686040
]
Ekaterina Dimitrova commented on CASSANDRA-18001:
-
{quote}Yes, I was hitting the same problem with CircleCI yesterday. It seems
that something is broken on their side.
{quote}
Thanks for confirming!
{quote}Otherwise the changes look good to me, let's see if we can get a good
run from Circle.
{quote}
I think I got that:
||PR||LOWRES||MIDRES||HIGHRES||
|[3.0|https://github.com/apache/cassandra/pull/2136]| |[CircleCI
#2265|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-3.0-final-mid]|[CircleCI
#2264|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-3.0-final-high]|
|[3.11|https://github.com/apache/cassandra/pull/2137]| |[CircleCI
#2268|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-3.11-final-mid]|[CircleCI
#2267|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-3.11-final-high]|
|[4.0|https://github.com/apache/cassandra/pull/2138]| |[CircleCI
#2274|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-4.0-final-mid]|[CircleCI
#2275|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-4.0-final-high]|
|[4.1|https://github.com/apache/cassandra/pull/2139]| |[CircleCI
#2256|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-4.1-final-mid]|[CircleCI
#2258|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-4.1-final-high]|
|[trunk|https://github.com/apache/cassandra/pull/2140]| |[CircleCI
#2263|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-trunk-final2-mid]|[CircleCI
#2262|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-trunk-final2-high]|
3.0 and 3.11 with HIGH Resources XLarge containers and parallelism 1 fail the
two tests we discussed. From what I see I am not sure it is a resource issue
though.
At this point I think we have to change CASSANDRA-18051 to optimize and fix
probably?
> Add missing tests suites to CircleCI
>
>
> Key: CASSANDRA-18001
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18001
> Project: Cassandra
> Issue Type: Task
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Urgent
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1-rc1, 4.1, 4.1.x, 4.2, 4.x
>
>
> Burn tests to all branches, large Python DTests (with/without vnodes),
> cqlshlib not tested in all branches and with all jdks; Java distributed tests
> not running with J8/J11 4.0+
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[ https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17685822#comment-17685822 ] Andres de la Peña commented on CASSANDRA-18001: --- Yes, I was hitting the same problem with CircleCI yesterday. It seems that something is broken on their side. Otherwise the changes look good to me, let's see if we can get a good run from Circle. > Add missing tests suites to CircleCI > > > Key: CASSANDRA-18001 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18001 > Project: Cassandra > Issue Type: Task > Components: CI >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Urgent > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1-rc1, 4.1, 4.1.x, 4.2, 4.x > > > Burn tests to all branches, large Python DTests (with/without vnodes), > cqlshlib not tested in all branches and with all jdks; Java distributed tests > not running with J8/J11 4.0+ -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[ https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17685603#comment-17685603 ] Ekaterina Dimitrova commented on CASSANDRA-18001: - Please ignore the current runs. Rerunning some of the tests as some containers get canceled for unknown to me reason. There is on top a message saying "CircleCI failed to run this build, check your config. Try re-running the build and if this issue persists, open a Support ticket." I saw it also on other tickets so I do not think it is related to what we do here. Also, it was weird as the cancelled containers were marked green and I thought for a moment it was rerun but then the number of tests run was smaller so I opened the steps and found message about a cancelled build... I also saw a bug in some of the runs that I will have to fix. > Add missing tests suites to CircleCI > > > Key: CASSANDRA-18001 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18001 > Project: Cassandra > Issue Type: Task > Components: CI >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Urgent > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1-rc1, 4.1, 4.1.x, 4.2, 4.x > > > Burn tests to all branches, large Python DTests (with/without vnodes), > cqlshlib not tested in all branches and with all jdks; Java distributed tests > not running with J8/J11 4.0+ -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[
https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17685421#comment-17685421
]
Ekaterina Dimitrova commented on CASSANDRA-18001:
-
{quote}Also, one of the tests that won't work with the multiplexer,
test_network_topology_strategy, is used as an example value of
REPEATED_LARGE_DTESTS in config-2_1.yml, generate.sh and readme.md. If we are
going to special case them to save us the executor we should probably choose a
test that actually works as an example of a valid value for
REPEATED_LARGE_DTESTS.
{quote}
Good point, thanks. I fixed it in this
[commit|https://github.com/ekaterinadimitrova2/cassandra/commit/d7cc41b3b5eaefdda103887b8121d0b888b5a270]
which I cherry-picked also to the rest of the branches
Running CI below, I will check the results later:
||PR||LOWRES||MIDRES||HIGHRES||
|[3.0|https://github.com/apache/cassandra/pull/2136]| |[CircleCI
#2247|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-3.0-final-mid]|[CircleCI
#2248|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-3.0-final-high]|
|[3.11|https://github.com/apache/cassandra/pull/2137]| |[CircleCI
#2250|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-3.11-final-mid]|[CircleCI
#2251|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-3.11-final-high]|
|[4.0|https://github.com/apache/cassandra/pull/2138]| |[CircleCI
#2253|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-4.0-final-mid]|[CircleCI
#2254|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-4.0-final-high]|
|[4.1|https://github.com/apache/cassandra/pull/2139]| |[CircleCI
#2256|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-4.1-final-mid]|[CircleCI
#2258|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-4.1-final-high]|
|[trunk|https://github.com/apache/cassandra/pull/2140]| |[CircleCI
#2263|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-trunk-final2-mid]|[CircleCI
#2262|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-trunk-final2-high]|
> Add missing tests suites to CircleCI
>
>
> Key: CASSANDRA-18001
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18001
> Project: Cassandra
> Issue Type: Task
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Urgent
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1-rc1, 4.1, 4.1.x, 4.2, 4.x
>
>
> Burn tests to all branches, large Python DTests (with/without vnodes),
> cqlshlib not tested in all branches and with all jdks; Java distributed tests
> not running with J8/J11 4.0+
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[
https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17684780#comment-17684780
]
Andres de la Peña commented on CASSANDRA-18001:
---
{quote}PS I opted out of raising the repeated jobs containers to XLarge, only
when we run the whole suite. I think we can leave a comment in the docs that
for those two people will need more resources.
{quote}
So the regular jobs use xlarge, and the repeated job still uses large. Works
for me. There could be a brief comment on [the {{REPEATED_LARGE_DTESTS}} env
var|https://github.com/apache/cassandra/blob/73f6098d012169a0ce18000ef73454291cd036f6/.circleci/config-2_1.yml#L91-L96]
mentioning that those two tests won't work on the multiplexer.
Also, one of the tests that won't work with the multiplexer,
{{{}test_network_topology_strategy{}}}, is used as an example value of
{{REPEATED_LARGE_DTESTS}} in
[{{config-2_1.yml}}|https://github.com/apache/cassandra/blob/73f6098d012169a0ce18000ef73454291cd036f6/.circleci/config-2_1.yml#L94],
[{{generate.sh}}|https://github.com/apache/cassandra/blob/73f6098d012169a0ce18000ef73454291cd036f6/.circleci/generate.sh#L54]
and {{{}readme.md{}}}. If we are going to special case them to save us the
executor we should probably choose a test that actually works as an example of
a valid value for {{{}REPEATED_LARGE_DTESTS{}}}.
Other than that the changes look good to me.
> Add missing tests suites to CircleCI
>
>
> Key: CASSANDRA-18001
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18001
> Project: Cassandra
> Issue Type: Task
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Urgent
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1-rc1, 4.1, 4.1.x, 4.2, 4.x
>
>
> Burn tests to all branches, large Python DTests (with/without vnodes),
> cqlshlib not tested in all branches and with all jdks; Java distributed tests
> not running with J8/J11 4.0+
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[ https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17684038#comment-17684038 ] Ekaterina Dimitrova commented on CASSANDRA-18001: - PS I opted out of raising the repeated jobs containers to XLarge, only when we run the whole suite. I think we can leave a comment in the docs that for those two people will need more resources. I don't expect many people to run them in a loop, they are also not to be discovered on a change as unit tests, to be run in a loop > Add missing tests suites to CircleCI > > > Key: CASSANDRA-18001 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18001 > Project: Cassandra > Issue Type: Task > Components: CI >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Urgent > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1-rc1, 4.1, 4.1.x, 4.2, 4.x > > > Burn tests to all branches, large Python DTests (with/without vnodes), > cqlshlib not tested in all branches and with all jdks; Java distributed tests > not running with J8/J11 4.0+ -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[ https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17684034#comment-17684034 ] Ekaterina Dimitrova commented on CASSANDRA-18001: - 3.0: * Rebased * Fixed the two typos * In the pre-commit workflow the jobs for large dtests now share a single approval button * Large Python DTests now use XLarge containers as per the agreement on CASSANDRA-18051 3.11 is the same patch, except clean was already removed. 4.0 adds only the missing Python Large DTests with XLarge containers and a single button in the pre-commit workflow; plus it fixes the split bug 4.1 - move to one button in the pre-commit workflow for Python Large DTests trunk - same as 4.1 plus regenerated the patch files as there were the annoying warnings forgotten from previous update ||PR||LOWRES||MIDRES||HIGHRES|| |[3.0|https://github.com/apache/cassandra/pull/2136]|[CircleCI #2219|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-3.0-final]|[CircleCI #2237|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-3.0-final-mid]|[CircleCI #2238|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-3.0-final-high]| |[3.11|https://github.com/apache/cassandra/pull/2137]|[CircleCI #2220|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-3.11-final]|[CircleCI #2235|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-3.11-final-mid]|[CircleCI #2236|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-3.11-final-high]| |[4.0|https://github.com/apache/cassandra/pull/2138]|[CircleCI #2231|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-4.0-final]|[CircleCI #2232|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-4.0-final-mid]|[CircleCI #2233|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-4.0-final-high]| |[4.1|https://github.com/apache/cassandra/pull/2139]|[CircleCI #2226|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-4.1-final]|[CircleCI #2227|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-4.1-final-mid]|[CircleCI #2228|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-4.1-final-high]| |[trunk|https://github.com/apache/cassandra/pull/2140|]|[CircleCI #2221|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-trunk-final2]|[CircleCI #2224|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-trunk-final2-mid]|[CircleCI #2225|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-trunk-final2-high]| In case everything looks fine on your end, I will push the CI runs PS Sorry [~stefan.miklosovic], I forgot to update the PR messages :( I will close the PRs when we are done here > Add missing tests suites to CircleCI > > > Key: CASSANDRA-18001 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18001 > Project: Cassandra > Issue Type: Task > Components: CI >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Urgent > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1-rc1, 4.1, 4.1.x, 4.2, 4.x > > > Burn tests to all branches, large Python DTests (with/without vnodes), > cqlshlib not tested in all branches and with all jdks; Java distributed tests > not running with J8/J11 4.0+ -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[
https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17681124#comment-17681124
]
Ekaterina Dimitrova commented on CASSANDRA-18001:
-
{quote}I have also commented a couple of typos on [the
commit|https://github.com/ekaterinadimitrova2/cassandra/commit/514e07dfee300b1b972e860c2f61674fae0793c6].
{quote}
Thank you for spotting those, I will fix them
{quote}Just one nit, I think that in the pre-commit workflow the jobs for large
dtests should share a single approval button. That button would approve regular
and repeated large dtests, possibly with and without vnodes. That would
simplify the workflow of the pre-commit workflow, while we still have the
separate workflow for fine tuning.
{quote}
We can do that. It was done that way in 4.1 and trunk so we will have to change
it there too. The only thing maybe is that those are resource intensive and I
think that's why we kept them separate
You actually also reminded me I should add some test change to trigger the
testing also for the repeated runs jobs
> Add missing tests suites to CircleCI
>
>
> Key: CASSANDRA-18001
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18001
> Project: Cassandra
> Issue Type: Task
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Urgent
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1-rc1, 4.1, 4.1.x, 4.2, 4.x
>
>
> Burn tests to all branches, large Python DTests (with/without vnodes),
> cqlshlib not tested in all branches and with all jdks; Java distributed tests
> not running with J8/J11 4.0+
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[ https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17680950#comment-17680950 ] Andres de la Peña commented on CASSANDRA-18001: --- Looks good. Just one nit, I think that in the pre-commit workflow the jobs for large dtests should share a single approval button. That button would approve regular and repeated large dtests, possibly with and without vnodes. That would simplify the workflow of the pre-commit workflow, while we still have the separate workflow for fine tuning. I have also commented a couple of typos on [the commit|https://github.com/ekaterinadimitrova2/cassandra/commit/514e07dfee300b1b972e860c2f61674fae0793c6]. > Add missing tests suites to CircleCI > > > Key: CASSANDRA-18001 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18001 > Project: Cassandra > Issue Type: Task > Components: CI >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Urgent > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1-rc1, 4.1, 4.1.x, 4.2, 4.x > > > Burn tests to all branches, large Python DTests (with/without vnodes), > cqlshlib not tested in all branches and with all jdks; Java distributed tests > not running with J8/J11 4.0+ -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[ https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17680802#comment-17680802 ] Ekaterina Dimitrova commented on CASSANDRA-18001: - Rebased and finished 3.0: ||Patch||LOWRES||MIDRES||HIGHRES|| |[3.0|https://github.com/ekaterinadimitrova2/cassandra/tree/18001-3.0-final]|[CircleCI #2196|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-3.0-final]|[CircleCI #2197|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-3.0-final-mid]|[CircleCI #2198|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-3.0-final-high]| As part of the patch: - Large DTests and cqlshlib tests were added - [the fix for split|https://github.com/ekaterinadimitrova2/cassandra/commit/514e07dfee300b1b972e860c2f61674fae0793c6#diff-bc7cfdde47dcf913bec381c90c03b51e430f1e4b34ff2ac1c419d87436c85137L914] of tests was back ported - [clean was removed|https://github.com/ekaterinadimitrova2/cassandra/commit/514e07dfee300b1b972e860c2f61674fae0793c6#diff-bc7cfdde47dcf913bec381c90c03b51e430f1e4b34ff2ac1c419d87436c85137L776] - [fixed|https://github.com/ekaterinadimitrova2/cassandra/commit/514e07dfee300b1b972e860c2f61674fae0793c6#diff-bc7cfdde47dcf913bec381c90c03b51e430f1e4b34ff2ac1c419d87436c85137L1278] number of tokens at one place as the default number is 32 not 16 (which was a change done in 4.0) *Note:* we still have Large DTests running with Large and not XLarge container, I can change that if we all agree about that on CASSANDRA-18051 If this is ok I will propagate the patch to 3.11 and 4.0. 4.0 will need everything but cqlshlib tests which were already added in the past. Then I will also run CI on all branches [~adelapena] , can you, please, take a look when you have time? > Add missing tests suites to CircleCI > > > Key: CASSANDRA-18001 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18001 > Project: Cassandra > Issue Type: Task > Components: CI >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Urgent > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1-rc1, 4.1, 4.1.x, 4.2, 4.x > > > Burn tests to all branches, large Python DTests (with/without vnodes), > cqlshlib not tested in all branches and with all jdks; Java distributed tests > not running with J8/J11 4.0+ -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[ https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17651417#comment-17651417 ] Ekaterina Dimitrova commented on CASSANDRA-18001: - Now when 4.1 is out getting this one back to WIP to finish the older branches. > Add missing tests suites to CircleCI > > > Key: CASSANDRA-18001 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18001 > Project: Cassandra > Issue Type: Task > Components: CI >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Urgent > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1-rc1, 4.1, 4.1.x, 4.2, 4.x > > > Burn tests to all branches, large Python DTests (with/without vnodes), > cqlshlib not tested in all branches and with all jdks; Java distributed tests > not running with J8/J11 4.0+ -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[ https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17634569#comment-17634569 ] Ekaterina Dimitrova commented on CASSANDRA-18001: - EDIT: CASSANDRA-18051 was just opened for the two mentioned tests > Add missing tests suites to CircleCI > > > Key: CASSANDRA-18001 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18001 > Project: Cassandra > Issue Type: Task > Components: CI >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Urgent > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1-rc, 4.1.x, 4.x > > > Burn tests to all branches, large Python DTests (with/without vnodes), > cqlshlib not tested in all branches and with all jdks; Java distributed tests > not running with J8/J11 4.0+ -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[
https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17634563#comment-17634563
]
Ekaterina Dimitrova commented on CASSANDRA-18001:
-
Committed to
To [https://github.com/apache/cassandra.git]
[ca00caffab..444da423d5 cassandra-4.1 ->
cassandra-4.1|https://github.com/apache/cassandra/commit/444da423d5a0e846852a51c80bb7a128b7a50d24]
[f317eb4f2f..d5fd0dceee trunk ->
trunk|https://github.com/apache/cassandra/commit/d5fd0dceee8f9ac5ed454e781a8c319a7b1b6677]
Before commit I noticed num_tokens=16 provided to the pytest command for
no-vnode tests. That was corrected.
Commits squashed and rebased on the patch from CASSANDRA-18039, so I ran final
pre-commit runs with midres and highres with those changes and verified the
right jvm was pulled and tests completed successfully.
I will open a follow up ticket to deal with the two large DTests failing with
midres with large containers, probably what [~adelapena] suggested.
{quote}As for the two tests starting nine nodes each,
{{test_network_topology_strategy}} and
{{{}test_network_topology_strategy_each_quorum{}}}, I think they were added by
CASSANDRA-10584 to test multi-DC consistency levels. I wonder if they actually
need to start 3 data centers with 3 nodes each. Would two data centers with 3
nodes be enough to test multi-DC consistency levels with multiple data centers?
6 nodes should be doable on Circle with the current resources.
{quote}
Below are the latest runs for completenes approved in Slack:
||MIDRES||HIGHRES||
|[4.1
CI|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-4.1-mid2]|[4.1
CI|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-4.1-high2]|
|[trunk
CI|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=trunk-final-mid]|[trunk
CI|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=trunk-final-high]
(cqlshlib are on the canceled workflow with low resources)|
Leaving the ticket open to finish the rest of the branches in peace - 3.0,
3.11, 4.0.
//CC [~mck] - this ticket is not an rc blocker anymore. Thank you!
> Add missing tests suites to CircleCI
>
>
> Key: CASSANDRA-18001
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18001
> Project: Cassandra
> Issue Type: Task
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Urgent
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1-rc, 4.1.x, 4.x
>
>
> Burn tests to all branches, large Python DTests (with/without vnodes),
> cqlshlib not tested in all branches and with all jdks; Java distributed tests
> not running with J8/J11 4.0+
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[ https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17634412#comment-17634412 ] Ekaterina Dimitrova commented on CASSANDRA-18001: - Thanks [~adelapena] , I also reviewed the runs and came to similar conclusions. I will squash, verify again nothing moved and commit 4.1 and trunk. Thanks! testMessageDeliveryOnReconnect is also flaky in 4.0 according to this ticket - CASSANDRA-16677 > Add missing tests suites to CircleCI > > > Key: CASSANDRA-18001 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18001 > Project: Cassandra > Issue Type: Task > Components: CI >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Urgent > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1-rc, 4.1.x, 4.x > > > Burn tests to all branches, large Python DTests (with/without vnodes), > cqlshlib not tested in all branches and with all jdks; Java distributed tests > not running with J8/J11 4.0+ -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[
https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17634333#comment-17634333
]
Andres de la Peña commented on CASSANDRA-18001:
---
Looks good. As for the failing tests on the CI runs:
*
[{{paxosRepairHistoryIsntUpdatedInForcedRepair}}|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/2134/workflows/ef90d431-ff8a-4b5e-a065-27797f1c9948/jobs/16779/tests]
is getting [a waiver on the mail
list|https://lists.apache.org/thread/0sb8hqctb54phn195zmp4xlc7cqsg2h3].
*
[{{testServiceTopPartitionsSingleTable}}|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/2134/workflows/b32ba097-10b2-4b40-a82e-0178efc65311/jobs/16821/tests]
is [reported on
Butler|https://butler.cassandra.apache.org/#/ci/upstream/workflow/Cassandra-4.1/failure/org.apache.cassandra.tools/TopPartitionsTest/testServiceTopPartitionsSingleTable]
*
[{{test_multi_dc_replace_with_rf1}}|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/2134/workflows/b32ba097-10b2-4b40-a82e-0178efc65311/jobs/16811/tests]
is a dtest timeout
*
[{{testMessageDeliveryOnReconnect}}|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/2138/workflows/33a80bbe-09ae-492a-84f3-bbcc1132f132/jobs/16862/tests]
fails on trunk, but Butler reports it as [failing on
4.1|https://butler.cassandra.apache.org/#/ci/upstream/compare/Cassandra-4.1/cassandra-4.1].
So I'd say we're ready to commit, if I'm not missing anything.
> Add missing tests suites to CircleCI
>
>
> Key: CASSANDRA-18001
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18001
> Project: Cassandra
> Issue Type: Task
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Urgent
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1-rc, 4.1.x, 4.x
>
>
> Burn tests to all branches, large Python DTests (with/without vnodes),
> cqlshlib not tested in all branches and with all jdks; Java distributed tests
> not running with J8/J11 4.0+
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[
https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17634140#comment-17634140
]
Ekaterina Dimitrova commented on CASSANDRA-18001:
-
Almost forgotJVM tests j8/j11...
Added on top of the current patches for 4.1 and trunk and started CI runs.
I set the following parameters and the jobs successfully detected that I had
missing in my local 4.1 and trunk branches a newly committed today
{_}org.apache.cassandra.distributed.test.PaxosRepair2Test{_}:
{code:java}
-e REPEATED_JVM_DTESTS=org.apache.cassandra.distributed.test.PagingTest \
-e REPEATED_JVM_DTESTS_COUNT=5
{code}
||Patch||LOWRES||MIDRES||HIGHRES||
|[4.1|https://github.com/ekaterinadimitrova2/cassandra/commit/5b8b86c8feb026b539d32cf7747b4f7e62746915]|[CI|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-final]|[CI|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-final-mid]|[CI|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-final-high]|
|[trunk|https://github.com/ekaterinadimitrova2/cassandra/commit/9e4d3b60af846af86037d63b6cdad9518b0a1b3e]|[CI|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-trunk-final]|[CI|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-trunk-final-mid]|[CI|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-trunk-final-high]|
> Add missing tests suites to CircleCI
>
>
> Key: CASSANDRA-18001
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18001
> Project: Cassandra
> Issue Type: Task
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Urgent
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1-rc, 4.1.x, 4.x
>
>
> Burn tests to all branches, large Python DTests (with/without vnodes),
> cqlshlib not tested in all branches and with all jdks; Java distributed tests
> not running with J8/J11 4.0+
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[ https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17634054#comment-17634054 ] Ekaterina Dimitrova commented on CASSANDRA-18001: - Interesting so with XLarge containers we did not manage to run those two tests on 3.0 - [https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/2076/workflows/828b3105-c0f8-443a-8b28-88c238afb71a/jobs/16441/tests] But now on 4.1 and trunk with high resources those are not failing. See the latest runs. The tests completed successfully I still do not think we should abuse the XLarge containers just for those two tests I suggest we continue with the plan we mentioned, to open a follow up ticket and potentially improve the test and not break the bank. I think we are ready to commit this one? There are a few other random failures. Some have already tickets, others don't. I will open now tickets, they are on trunk. CASSANDRA-18045 - org.apache.cassandra.service.LeaveAndBootstrapTest.testSimultaneousMove CASSANDRA-18046 - test_resume_failed_replace and test_add_dc_after_mv_network_replication CASSANDRA-16861 - test_compression_cql_options > Add missing tests suites to CircleCI > > > Key: CASSANDRA-18001 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18001 > Project: Cassandra > Issue Type: Task > Components: CI >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Urgent > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1-rc, 4.1.x, 4.x > > > Burn tests to all branches, large Python DTests (with/without vnodes), > cqlshlib not tested in all branches and with all jdks; Java distributed tests > not running with J8/J11 4.0+ -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[ https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17633974#comment-17633974 ] Ekaterina Dimitrova commented on CASSANDRA-18001: - Made one more check of the patches and started the rest of the tests - the other workflows + highres Posting the table again for completeness: ||Patch||LOWRES||MIDRES||HIGHRES|| |[4.1|https://github.com/ekaterinadimitrova2/cassandra/tree/18001-4.1-3]|[CI|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-4.1-3]|[CI|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-4.1-3-mid]|[CI|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-4.1-3-high2]| |[trunk|https://github.com/ekaterinadimitrova2/cassandra/tree/18001-trunk-3]|[CI|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-trunk-3]|[CI|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/2130/workflows/9df2d073-c26a-4750-82e4-8b6645f37f9c]|[CI|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-trunk-3-high2]| I will check back later the CI results and commit if there are no surprises. > Add missing tests suites to CircleCI > > > Key: CASSANDRA-18001 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18001 > Project: Cassandra > Issue Type: Task > Components: CI >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Urgent > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1-rc, 4.1.x, 4.x > > > Burn tests to all branches, large Python DTests (with/without vnodes), > cqlshlib not tested in all branches and with all jdks; Java distributed tests > not running with J8/J11 4.0+ -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[ https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17633896#comment-17633896 ] Andres de la Peña commented on CASSANDRA-18001: --- bq. Shall I start the rest of the runs and if everything completes fine commit the 4.1 and trunk patches? Sure, let's start those runs and commit if they look good. > Add missing tests suites to CircleCI > > > Key: CASSANDRA-18001 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18001 > Project: Cassandra > Issue Type: Task > Components: CI >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Urgent > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1-rc, 4.1.x, 4.x > > > Burn tests to all branches, large Python DTests (with/without vnodes), > cqlshlib not tested in all branches and with all jdks; Java distributed tests > not running with J8/J11 4.0+ -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[
https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17633885#comment-17633885
]
Ekaterina Dimitrova commented on CASSANDRA-18001:
-
{quote}So the plan is not adding burn tests to CircleCI, isn't it?
{quote}
Yes, seems those are problematic for 4.0+ in the CircleCI containers. It was
also mentioned in CASSANDRA-14806.
They also run successfully in Jenkins, butler doesn't show any flakiness so we
can. use that for the release.
{quote}Ah, those also fail on Jenkins for trunk:
{quote}
Thanks a lot for checking! I will open follow-up tickets and ping the authors
on the tickets where those broke.
{quote}Would two data centers with 3 nodes be enough to test multi-DC
consistency levels with multiple data centers? 6 nodes should be doable on
Circle with the current resources.
{quote}
I think you are right, I just didn't have the chance to look into those. I will
open a follow up ticket for those and link it in butler.
Shall I start the rest of the runs and if everything completes fine commit the
4.1 and trunk patches?
> Add missing tests suites to CircleCI
>
>
> Key: CASSANDRA-18001
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18001
> Project: Cassandra
> Issue Type: Task
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Urgent
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1-rc, 4.1.x, 4.x
>
>
> Burn tests to all branches, large Python DTests (with/without vnodes),
> cqlshlib not tested in all branches and with all jdks; Java distributed tests
> not running with J8/J11 4.0+
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[
https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17633813#comment-17633813
]
Andres de la Peña commented on CASSANDRA-18001:
---
As for the two tests starting nine nodes each,
{{test_network_topology_strategy}} and
{{test_network_topology_strategy_each_quorum}}, I think they were added by
CASSANDRA-10584 to test multi-DC consistency levels. I wonder if they actually
need to start 3 data centers with 3 nodes each. Would two data centers with 3
nodes be enough to test multi-DC consistency levels with multiple data centers?
6 nodes should be doable on Circle with the current resources.
> Add missing tests suites to CircleCI
>
>
> Key: CASSANDRA-18001
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18001
> Project: Cassandra
> Issue Type: Task
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Urgent
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1-rc, 4.1.x, 4.x
>
>
> Burn tests to all branches, large Python DTests (with/without vnodes),
> cqlshlib not tested in all branches and with all jdks; Java distributed tests
> not running with J8/J11 4.0+
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[
https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17633798#comment-17633798
]
Andres de la Peña commented on CASSANDRA-18001:
---
{quote}I see that the MIDRES tests for trunk have 8 large dtests failing, 6
more that the two starting 9 nodes. Looking into them.
{quote}
Ah, those also fail on Jenkins for trunk:
*
[https://butler.cassandra.apache.org/#/ci/upstream/workflow/Cassandra-trunk/failure/materialized_views_test/TestMaterializedViews/test_add_dc_after_mv_network_replication]
*
[https://butler.cassandra.apache.org/#/ci/upstream/workflow/Cassandra-trunk/failure/replace_address_test/TestReplaceAddress/test_resume_failed_replace]
> Add missing tests suites to CircleCI
>
>
> Key: CASSANDRA-18001
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18001
> Project: Cassandra
> Issue Type: Task
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Urgent
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1-rc, 4.1.x, 4.x
>
>
> Burn tests to all branches, large Python DTests (with/without vnodes),
> cqlshlib not tested in all branches and with all jdks; Java distributed tests
> not running with J8/J11 4.0+
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[ https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17633760#comment-17633760 ] Andres de la Peña commented on CASSANDRA-18001: --- So the plan is not adding burn tests to CircleCI, isn't it? I see that the MIDRES tests for trunk have 8 large dtests failing, 6 more that the two with starting 9 nodes. Looking into them. > Add missing tests suites to CircleCI > > > Key: CASSANDRA-18001 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18001 > Project: Cassandra > Issue Type: Task > Components: CI >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Urgent > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1-rc, 4.1.x, 4.x > > > Burn tests to all branches, large Python DTests (with/without vnodes), > cqlshlib not tested in all branches and with all jdks; Java distributed tests > not running with J8/J11 4.0+ -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[
https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17633533#comment-17633533
]
Ekaterina Dimitrova commented on CASSANDRA-18001:
-
A few updates on my end... trying to be a good citizen it seems my try to do
good to the HIGHRES and reduce the parallelism so that people do not spin empty
containers for the new jobs didn't work. Manual verification of the file shows
things shifted wrong. Also, I hit the issue with CircleCI where commands are
squashed again and this makes it super hard for debugging.
Last response from CircleCI team on this issue:
{color:#de350b}After syncing with our team, I want to share that:{color}
{color:#de350b}[YAML is essentially a
specification|https://urldefense.com/v3/__https://yaml.org/__;!!PbtH5S7Ebw!b1SNWAq5oUmhmKoW44AWaB-eBX5wSa2T0dD6mpaTuIO6gYotfZnnuccv4Zc0vbn4bQ7ulVg9G_6Sz_k0lEhSMIY7oL_azCRsbw$],
and its implementation will depend on the libraries any tooling uses.{color}
{color:#de350b}[As stated in the
specification|https://urldefense.com/v3/__https://yaml.org/spec/1.2.2/*chapter-6-structural-productions__;Iw!!PbtH5S7Ebw!b1SNWAq5oUmhmKoW44AWaB-eBX5wSa2T0dD6mpaTuIO6gYotfZnnuccv4Zc0vbn4bQ7ulVg9G_6Sz_k0lEhSMIY7oL_w7U4BbA$],
"The amount of indentation is a presentation detail and must not be used to
convey content information."{color}
{color:#de350b}This means that as long as a YAML expression is still data-wise
equivalent, the formatted output of a YAML expression can depend on the library
itself.{color}
{color:#de350b}As such, certain libraries can choose to "compress" the
expressions, so long as the data is still the same data-wise.{color}
{color:#de350b}Such compressions can be beneficial, since the library may see
reducing the overall filesize being an important thing.{color}
{color:#de350b}In other words, there is really no hard-guarantees to preserving
a YAML expression's format.{color}
{color:#de350b}We think, *if* you really want to ensure the {{commands}} format
are always preserved / expressed in blocks, you would thus want to
post-process.{color}
{color:#de350b}I have set up an example here using the js-yaml Node
package:{color}
{color:#de350b}[https://replit.com/@kelvintaywl/YAMLprettifier?v=1|https://urldefense.com/v3/__https://replit.com/@kelvintaywl/YAMLprettifier?v=1__;!!PbtH5S7Ebw!b1SNWAq5oUmhmKoW44AWaB-eBX5wSa2T0dD6mpaTuIO6gYotfZnnuccv4Zc0vbn4bQ7ulVg9G_6Sz_k0lEhSMIY7oL-fJ6Nr3Q$]{color}
{color:#de350b}(If you run the repl code, you should see the post-processing
expanding the "compressed" command into a block){color}
*I do not think we can/should fix the whole world in one ticket and also that
trying that hides risks.*
{color:#172b4d}On the bright side MIDRES tests and files seem good to me.{color}
{color:#172b4d}*STEPS TAKEN TO UNBLOCK 4.1 OVER THE WEEKEND:* {color}
- 4.1 and trunk finished as there are various things that differ in branches
and few different bugs we caught... After that, when we are not pressed by the
4.1 release, I will fix 3.11, 4.0 and 3.0 patches over the next week.
* I fixed now the 4.1+ patches for HIGHRES
* I finished the cqlshlib
* My testing and an old ticket for JDK 11 improvements proved that the burn
tests in 4.1 and trunk (some of them) crash even with X-Large containers. I
verified in Jenkins and Butler that all the burn tests are run and they finish
successfully. SO I think those are not a blocker, we can use Jenkins
||Patch||LOWRES||MIDRES||HIGHRES||
|[4.1|https://github.com/ekaterinadimitrova2/cassandra/tree/18001-4.1-3]|[CI|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-4.1-3]|[CI|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-4.1-3-mid]|[CI|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-4.1-3-high]|
|[trunk|https://github.com/ekaterinadimitrova2/cassandra/tree/18001-trunk-3]|[CI|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-trunk-3]|[CI|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/2130/workflows/9df2d073-c26a-4750-82e4-8b6645f37f9c]|[CI|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=18001-trunk-3-high2]|
_*trunk still running - I will check it tomorrow morning_
Reminder about dtests large - _There are two tests that fail because there are
not enough resources to bring up 9 nodes. Spinning a 2X Large container for all
large tests because of 2 is not justified._
I can push the HIGHRES and pre-commit runs tomorrow if people are ok with the
MIDRES and the current flow (trying to save some resources)
> Add missing tests suites to CircleCI
>
>
> Key: CASSANDRA-18001
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18001
> Project: Cassandra
> Issue
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[ https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17632599#comment-17632599 ] Ekaterina Dimitrova commented on CASSANDRA-18001: - Ok, _I think_ all 5 branches are ready for large dtests and burn tests. At least in UI they look fine to me. Changing the highres for those large tests gave me a bit of hard time. Having all tests everywhere now makes merge commits hard. :( Hopefully people add from now on all new suites to trunk on time. I will review the code in the morning and push for testing and finish the cqlshlib. > Add missing tests suites to CircleCI > > > Key: CASSANDRA-18001 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18001 > Project: Cassandra > Issue Type: Task > Components: CI >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Urgent > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1-rc, 4.1.x, 4.x > > > Burn tests to all branches, large Python DTests (with/without vnodes), > cqlshlib not tested in all branches and with all jdks; Java distributed tests > not running with J8/J11 4.0+ -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[ https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17632571#comment-17632571 ] Ekaterina Dimitrova commented on CASSANDRA-18001: - So burn tests are solved by removing clean as suggested by Andres. Regarding the large tests, [~adelapena] spotted a bug in the split command where if we provide more containers than we have test classes, CircleCI will run all DTests in the containers that receive an empty split. I fixed this to fail those containers with a message for the user - Tune your parallelism, there are more containers then test classes. Nothing to do in this container We would need 35 containers for the large DTests in theory at max, so I got 36 containers with low resources and started the tests - [this is how a spare container would fail now |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/2083/workflows/20a7da50-8b96-4829-84db-7b80fd94034d/jobs/16455/parallel-runs/35?filterBy=ALL]. (the rest are failures as I didn't want to waste higher resources and run all tests). I also tuned the large DTests to run on 4 large containers. There are two tests that fail because there are not enough resources to bring up 9 nodes. Spinning a 2X Large container for all large tests because of 2 is not justified. Also noticed that DTests for 3 and 11 run with 32 and not 16 tokens. I will open a follow up ticket where we fix that and add a one stop parameter for the number of tokens for all DTests per branch. I suggest we commit those two tests failing but open a follow up ticket where we can pull them in a bigger container and additional job or just remove them. That will require probably some annotation to be added for them. I want us now to unblock 4.1 here I will propagate now the patches to all branches and add the missing Cython CQLSHLIB tests in 4.0, 4.1 and trunk now when CASSANDRA-18028 and CASSANDRA-18036 are closed. I will open a follow up ticket for adding the CQLSHLIB tests for previous branches. > Add missing tests suites to CircleCI > > > Key: CASSANDRA-18001 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18001 > Project: Cassandra > Issue Type: Task > Components: CI >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Urgent > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1-rc, 4.1.x, 4.x > > > Burn tests to all branches, large Python DTests (with/without vnodes), > cqlshlib not tested in all branches and with all jdks; Java distributed tests > not running with J8/J11 4.0+ -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[ https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17632410#comment-17632410 ] Ekaterina Dimitrova commented on CASSANDRA-18001: - Yes, I came to the same conclusion. Also there are some discussions about burn tests and flakiness. Considering they run fine on our turtle Jenkins things might have significantly changed since those discussions were happening years ago. I will make the changes and test. Thanks for your help > Add missing tests suites to CircleCI > > > Key: CASSANDRA-18001 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18001 > Project: Cassandra > Issue Type: Task > Components: CI >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Urgent > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1-rc, 4.1.x, 4.x > > > Burn tests to all branches, large Python DTests (with/without vnodes), > cqlshlib not tested in all branches and with all jdks; Java distributed tests > not running with J8/J11 4.0+ -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[
https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17632409#comment-17632409
]
Andres de la Peña commented on CASSANDRA-18001:
---
I think the {{clean}} was removed in the other branches by CASSANDRA-15108,
when j11 builds were added. There is even [a comment about why it was
removed|https://issues.apache.org/jira/browse/CASSANDRA-15108?focusedCommentId=16836725=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16836725].
Definitively I don't think we need that {{clean}} on {{run_junit_tests}} for
3.0.
> Add missing tests suites to CircleCI
>
>
> Key: CASSANDRA-18001
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18001
> Project: Cassandra
> Issue Type: Task
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Urgent
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1-rc, 4.1.x, 4.x
>
>
> Burn tests to all branches, large Python DTests (with/without vnodes),
> cqlshlib not tested in all branches and with all jdks; Java distributed tests
> not running with J8/J11 4.0+
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[
https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17632402#comment-17632402
]
Andres de la Peña commented on CASSANDRA-18001:
---
Note that in the very same 3.0
[{{run_parallel_junit_tests}}|https://github.com/apache/cassandra/blob/cassandra-3.0/.circleci/config-2_1.yml#L815]
doesn't use {{{}clean{}}}. That's why the regular unit tests pass with
{{{}no-build-test{}}}, being both things mutually exclusive.
> Add missing tests suites to CircleCI
>
>
> Key: CASSANDRA-18001
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18001
> Project: Cassandra
> Issue Type: Task
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Urgent
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1-rc, 4.1.x, 4.x
>
>
> Burn tests to all branches, large Python DTests (with/without vnodes),
> cqlshlib not tested in all branches and with all jdks; Java distributed tests
> not running with J8/J11 4.0+
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[ https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17632376#comment-17632376 ] Ekaterina Dimitrova commented on CASSANDRA-18001: - Thanks for investigating it [~adelapena] . I was focused on 3.0 and didn't see the other branches. I guess someone meant to say jdk 11 on jdk 8 as we definitely do not support jdk8 on jdk11. Or that we clean every time to be sure there is no different jdk build from the previous jobs but I guess considering we have our jobs on specific per-idk builds this is not really possible. Let me dig into the code too. Thanks again > Add missing tests suites to CircleCI > > > Key: CASSANDRA-18001 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18001 > Project: Cassandra > Issue Type: Task > Components: CI >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Urgent > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1-rc, 4.1.x, 4.x > > > Burn tests to all branches, large Python DTests (with/without vnodes), > cqlshlib not tested in all branches and with all jdks; Java distributed tests > not running with J8/J11 4.0+ -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[
https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17632349#comment-17632349
]
Andres de la Peña commented on CASSANDRA-18001:
---
The reason for the {{clean}} before running tests is:
{code}
Please note that we run `clean` and therefore rebuild the project, as we can't
run tests on Java 8 in case based on Java 11 builds.
{code}
But if I'm right we never run tests with Java 8 based on Java 11, do we? For
sure all branches but 3.0 don't do the {{{}clean{}}}, and in 3.0 we don't even
have Java 11. So I guess we should just remove the {{clean}} in 3.11 and
preserve the {{no-build-test}} flag. wdyt?
> Add missing tests suites to CircleCI
>
>
> Key: CASSANDRA-18001
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18001
> Project: Cassandra
> Issue Type: Task
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Urgent
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1-rc, 4.1.x, 4.x
>
>
> Burn tests to all branches, large Python DTests (with/without vnodes),
> cqlshlib not tested in all branches and with all jdks; Java distributed tests
> not running with J8/J11 4.0+
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[
https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17632342#comment-17632342
]
Andres de la Peña commented on CASSANDRA-18001:
---
I think the removal of the call to {{clean}} on {{run_junit_tests}} was done by
CASSANDRA-15508 in [this
commit|https://github.com/apache/cassandra/commit/3bcabdc5d2131e43f37fab456e1846bc245b2ac8#],
so it was preserved in 3.0 and removed in 3.11.
> Add missing tests suites to CircleCI
>
>
> Key: CASSANDRA-18001
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18001
> Project: Cassandra
> Issue Type: Task
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Urgent
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1-rc, 4.1.x, 4.x
>
>
> Burn tests to all branches, large Python DTests (with/without vnodes),
> cqlshlib not tested in all branches and with all jdks; Java distributed tests
> not running with J8/J11 4.0+
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[
https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17632336#comment-17632336
]
Andres de la Peña commented on CASSANDRA-18001:
---
{quote}Comments addressed and patch finished for [3.0
|https://github.com/apache/cassandra/compare/trunk...ekaterinadimitrova2:cassandra:18001-3.0-2-mid]
(or at least I hoped I was done)
{quote}
Looks great, I have left a couple of minor nits in the commit.
{quote}Unfortunately, I discovered that CASSANDRA-18000 broke the long unit
tests and it also breaks now the burn tests.
{quote}
I think the problem is that {{ant clean}} is used in combination with
{{-Dno-build-test=true}}
[here|https://github.com/apache/cassandra/blob/cassandra-3.0/.circleci/config-2_1.yml#L776].
The reason for that {{clean}} is mentioned
[here|https://github.com/apache/cassandra/blob/cassandra-3.0/.circleci/config-2_1.yml#L767-L768],
although that doesn't seem to make sense for 3.0 since Java 11 is not used.
Also, it seems that the {{clean}} is not used in 3.11, 4.0, 4.1 nor trunk,
although the comment about it is present in all branches. I wonder if we should
be running {{clean}} in those branches, or the comment is wrong.
{quote}I tried locally and in CI without my patch. I guess it wasn't noticed up
to now as those are not mandatory pre-commit.
{quote}
Indeed I didn't run long unit tests on all branches during CASSANDRA-18000, and
I should have. But I [ran them for
trunk|https://app.circleci.com/pipelines/github/adelapena/cassandra/2370/workflows/29e394c4-d76f-43df-a011-57dd5727e890]
and it passed, probably because of the absent {{{}clean{}}}.
> Add missing tests suites to CircleCI
>
>
> Key: CASSANDRA-18001
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18001
> Project: Cassandra
> Issue Type: Task
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Urgent
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1-rc, 4.1.x, 4.x
>
>
> Burn tests to all branches, large Python DTests (with/without vnodes),
> cqlshlib not tested in all branches and with all jdks; Java distributed tests
> not running with J8/J11 4.0+
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[
https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17632045#comment-17632045
]
Ekaterina Dimitrova commented on CASSANDRA-18001:
-
{quote}I have left a few comments on the commit, around the generate.sh script
and repeated tests.
{quote}
Comments addressed and patch finished for 3.0
Unfortunately, I discovered that CASSANDRA-18000 broke the long unit tests and
it also breaks now the burn tests.
I tried locally and in [CI without my
patch|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/2075/workflows/e0915bfe-90f1-4e41-9fd3-ba2ed3996ee6/jobs/16437].
I guess it wasn't noticed up to now as those are not mandatory pre-commit.
I suggest we take this new target as optional only to the tests where it is
feasible. In case there is no other solution I am missing, of course. I can
work on this tomorrow.
I also started the large dtests with large container and p50 as they are around
70 so I hoped that should be something reasonable. But looking
[here|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/2074/workflows/c660799f-5d12-4e7c-af83-d2d2bf89c412/jobs/16433/parallel-runs/6?filterBy=ALL]
(some containers are still running though) it seems two tests already failed
as they miss enough resources. We might need X-Large for those. I will check
back tomorrow how this finished and test more.
> Add missing tests suites to CircleCI
>
>
> Key: CASSANDRA-18001
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18001
> Project: Cassandra
> Issue Type: Task
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Urgent
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1-rc, 4.1.x, 4.x
>
>
> Burn tests to all branches, large Python DTests (with/without vnodes),
> cqlshlib not tested in all branches and with all jdks; Java distributed tests
> not running with J8/J11 4.0+
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[ https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17631826#comment-17631826 ] Andres de la Peña commented on CASSANDRA-18001: --- I have left a few comments on the commit, around the `generate.sh` script and repeated tests. bq. I will be working in parallel on cqlshlib in a separate commit. bq. We can split the work into phases and separate commits Or separate tickets, as you prefer. > Add missing tests suites to CircleCI > > > Key: CASSANDRA-18001 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18001 > Project: Cassandra > Issue Type: Task > Components: CI >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Urgent > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1-rc, 4.1.x, 4.x > > > Burn tests to all branches, large Python DTests (with/without vnodes), > cqlshlib not tested in all branches and with all jdks; Java distributed tests > not running with J8/J11 4.0+ -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[ https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17631786#comment-17631786 ] Ekaterina Dimitrova commented on CASSANDRA-18001: - And that is how CASSANDRA-18028 was born and super quick fix was already delivered for Jenkins by [~mck] and [~Bereng] . Thank you both! I am working now on similar fix for 4.0+ and also noticed I have to add also cqlshlib for jdk8/11 > Add missing tests suites to CircleCI > > > Key: CASSANDRA-18001 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18001 > Project: Cassandra > Issue Type: Task > Components: CI >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Urgent > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1-rc, 4.1.x, 4.x > > > Burn tests to all branches, large Python DTests (with/without vnodes), > cqlshlib not tested in all branches and with all jdks; Java distributed tests > not running with J8/J11 4.0+ -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[ https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17631449#comment-17631449 ] Ekaterina Dimitrova commented on CASSANDRA-18001: - Rebased the 3.0 patch on top of CASSANDRA-17950 [here|https://github.com/ekaterinadimitrova2/cassandra/commit/7aaa1486bc455f1a3543d3371b7d237a26fcb39a] > Add missing tests suites to CircleCI > > > Key: CASSANDRA-18001 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18001 > Project: Cassandra > Issue Type: Task > Components: CI >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Urgent > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1-rc, 4.1.x, 4.x > > > Burn tests to all branches, large Python DTests (with/without vnodes), > cqlshlib not tested in all branches and with all jdks; Java distributed tests > not running with J8/J11 4.0+ -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[ https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17627372#comment-17627372 ] Ekaterina Dimitrova commented on CASSANDRA-18001: - Website quick fix submitted in CASSANDRA-18005 > Add missing tests suites to CircleCI > > > Key: CASSANDRA-18001 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18001 > Project: Cassandra > Issue Type: Task > Components: CI >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Urgent > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 4.x > > > Burn tests to all branches, large Python DTests (with/without vnodes), > cqlshlib not tested in all branches and with all jdks; Java distributed tests > not running with J8/J11 4.0+ -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[ https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17627360#comment-17627360 ] Ekaterina Dimitrova commented on CASSANDRA-18001: - For the record I noticed we have _microbench_ in the Jenkins scripts but [~mck] confirmed those are not running now and I understand they are not a requirement. Also, they are probably exploratory and not necessarily meant to pass at the moment. > Add missing tests suites to CircleCI > > > Key: CASSANDRA-18001 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18001 > Project: Cassandra > Issue Type: Task > Components: CI >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Urgent > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 4.x > > > Burn tests to all branches, large Python DTests (with/without vnodes), > cqlshlib not tested in all branches and with all jdks; Java distributed tests > not running with J8/J11 4.0+ -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[ https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17627333#comment-17627333 ] Ekaterina Dimitrova commented on CASSANDRA-18001: - I also noticed a few updates needed in our testing docs. I will push a quick patch soon. We do not need jars to run the burn tests. The jars are for remote execution as per build.xml and they also do not exist in older branches. I also found a typo in one command. Fix coming soon. > Add missing tests suites to CircleCI > > > Key: CASSANDRA-18001 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18001 > Project: Cassandra > Issue Type: Task > Components: CI >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Urgent > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 4.x > > > Burn tests to all branches, large Python DTests (with/without vnodes), > cqlshlib not tested in all branches and with all jdks; Java distributed tests > not running with J8/J11 4.0+ -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18001) Add missing tests suites to CircleCI
[ https://issues.apache.org/jira/browse/CASSANDRA-18001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17627328#comment-17627328 ] Ekaterina Dimitrova commented on CASSANDRA-18001: - Blocking this one on other two (CASSANDRA-17912 and CASSANDRA-17987) to be committed first as they are almost ready. Also, I will create the patches when those are ready and committed so I do not have to recreate them a few times. For now I added the two suites which were completely missing in all branches. [~adelapena], do you mind to take a look when you are around and have time, please? If those are fine I will forward-port to the other branches and JDKs. The other jobs (cqlshlib etc) are already there, just need to be multiplied at a few places so I will do it quickly when the mentioned tickets get merged. 3.0 - [patch|https://github.com/apache/cassandra/compare/trunk...ekaterinadimitrova2:cassandra:17930-3.0], [separate jobs workflow|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/2019/workflows/e1e1fc43-a870-4b68-8161-c30a2bf9ae10] and [pre-commit jobs workflow|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/2019/workflows/c305e0a3-6c0a-4eb0-8b2c-c3fc8ea1851b] > Add missing tests suites to CircleCI > > > Key: CASSANDRA-18001 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18001 > Project: Cassandra > Issue Type: Task > Components: CI >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Urgent > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 4.x > > > Burn tests to all branches, large Python DTests (with/without vnodes), > cqlshlib not tested in all branches and with all jdks; Java distributed tests > not running with J8/J11 4.0+ -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
