[
https://issues.apache.org/jira/browse/CASSANDRA-19813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17870866#comment-17870866
]
Michael Semb Wever commented on CASSANDRA-19813:
------------------------------------------------
In `dtest-upgrade-large` (and variants) there are only 57 tests (actually 53,
as four are template tests that need to be "generated").
When splits are "empty" there are supposed to only run the first test found in
the test_list.txt file, so that some test results exist (and don't fail the
build):
https://github.com/apache/cassandra/blob/cassandra-5.0/.build/run-tests.sh#L135-L137
versus
https://github.com/apache/cassandra/blob/cassandra-5.0/.build/run-python-dtests.sh#L148
Something changed, so the "empty" splits are now running many more tests.
Tests themselves are not taking longer.
Reducing the split size from 64 to 32 validated this:
- https://ci-cassandra.apache.org/job/Cassandra-devbranch-5/40/
- [^Cassandra-devbranch-5_40_ci_summary.html]
[^Cassandra-devbranch-5_40_results_details.tar.xz]
> timeout on
> upgrade_tests/upgrade_through_versions_test.py::TestProtoV3Upgrade_AllVersions_RandomPartitioner_EndsAt_Trunk_HEAD::test_parallel_upgrade
> -----------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-19813
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19813
> Project: Cassandra
> Issue Type: Bug
> Components: CI
> Reporter: Michael Semb Wever
> Priority: Normal
> Fix For: 5.0-rc
>
> Attachments: Cassandra-devbranch-5_40_ci_summary.html,
> Cassandra-devbranch-5_40_results_details.tar.xz,
> test_rolling_upgrade.122.log, test_rolling_upgrade.123-1.log,
> test_rolling_upgrade.123-2.log, test_rolling_upgrade.134.log,
> test_rolling_upgrade.257.good.log, test_rolling_upgrade.261-1.log,
> test_rolling_upgrade.261-2.log
>
>
> The
> upgrade_tests/upgrade_through_versions_test.py::TestProtoV3Upgrade_AllVersions_RandomPartitioner_EndsAt_Trunk_HEAD::test_parallel_upgrade
> test is taking longer than usual.
> CI in 5.0 is timing out the dtest-upgrade-large 59/64 split always now.
> Our 5.0-rc testing results are tainted because these time outs always abort
> the 5.0 pipeline runs.
> It looks like that split is taking 16x times as long now…
> This appears to be caused from either/both
> -
> https://github.com/apache/cassandra/commit/08e1fecf36507397cf3122d77f84aa23150da588
> -
> https://github.com/apache/cassandra-dtest/commit/2b17c1293056068bb3e94c332d6fb99df6a0b0fa
> Example of good run.
> [^test_rolling_upgrade.257.good.log] (ci-cassandra.a.o)
> Examples of bad runs.
> [^test_rolling_upgrade.122.log], [^test_rolling_upgrade.123-2.log],
> [^test_rolling_upgrade.123-1.log], [^test_rolling_upgrade.134.log]
> (internal ci)
> [^test_rolling_upgrade.261-1.log] , [^test_rolling_upgrade.261-2.log]
> (ci-cassandra.a.o)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
