[
https://issues.apache.org/jira/browse/CASSANDRA-17939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17617784#comment-17617784
]
Andres de la Peña edited comment on CASSANDRA-17939 at 10/14/22 3:02 PM:
-------------------------------------------------------------------------
It seems that 100 is an acceptable number for long unit tests.
As for Python upgrade dtests, those can be super slow if one tries to run an
entire test file, as in {{{}cql_tests.py{}}}. Those files contains many tests
that are applied to different upgrade paths, so they can take ages. Also, the
syntax for running particular tests isn't super friendly, as in
{{{}REPEATED_UPGRADE_DTESTS:
cql_tests.py::TestCQLNodes2RF1_Upgrade_current_4_0_x_To_indev_4_0_x{}}}. So I'm
setting the default to only 25 iterations, which should be something between 1h
and 2h for a large test file. After all, there isn't automatic detection of
Python dtests so the users will probably manually provide their own count.
Given that we have different counts for each test family, I'm removing the
default generic {{{}REPEATED_TESTS_COUNT{}}}. That can only lead to
misunderstandings if one gets used to it and then get surprised when some
especially long tests ignore it. Also, the way it worked was quite hacky and
I'm afraid it could fail in future versions of CircleCI CLI.
I'm also leaving a single {{REPEATED_TESTS_STOP_ON_FAILURE}} env var instead of
one per family of tests. This is so partly to avoid the error-prone approach
for default env vars. Also I can't see a case where we could have different
setting per family of tests, and the less vars we have the better.
The {{readme.md}} file is updated to describe the multiplexer env vars.
||Patch||CI without test changes||CI with test changes||
|[trunk|https://github.com/adelapena/cassandra/tree/17939-trunk-generate]|[j8_pre-commit|https://app.circleci.com/pipelines/github/adelapena/cassandra/2248/workflows/c53a390b-d08a-4a2a-b63d-3e6d35f9984b]
[j11_pre-commit|https://app.circleci.com/pipelines/github/adelapena/cassandra/2248/workflows/d6f51790-a17d-44cd-810f-fc679b1f5379]
[j8_separate|https://app.circleci.com/pipelines/github/adelapena/cassandra/2248/workflows/3f4a47dd-1bfa-4a5c-a384-85d708fbd41c]
[j11_separate|https://app.circleci.com/pipelines/github/adelapena/cassandra/2248/workflows/66462337-401d-4f87-9b5b-b1c016c50613]|[j8_pre-commit|https://app.circleci.com/pipelines/github/adelapena/cassandra/2247/workflows/a88e2d87-828c-4fe5-84de-d190d294c2a4]
[j11_pre-commit|https://app.circleci.com/pipelines/github/adelapena/cassandra/2247/workflows/9ff67d2f-cae6-4fc8-9537-376a6cd267e1]
[j8_separate|https://app.circleci.com/pipelines/github/adelapena/cassandra/2247/workflows/7cb93de7-b54f-46e9-aac1-ab45b736caaa]
[j11_separate|https://app.circleci.com/pipelines/github/adelapena/cassandra/2247/workflows/9f083479-f7c9-4a94-9bcb-b2f47851a5e3]|
[~jmckenzie] if you are ok with the changes I can start preparing the patches
for all the other branches (since 3.0).
was (Author: adelapena):
It seems that 100 is an acceptable number for long unit tests.
As for Python upgrade dtests, those can be super slow if one tries to run an
entire test file, as in {{{}cql_tests.py{}}}. Those files contains many tests
that are applied to different upgrade paths, so they can take ages. Also, the
syntax for running particular tests isn't super friendly, as in
{{{}REPEATED_UPGRADE_DTESTS:
cql_tests.py::TestCQLNodes2RF1_Upgrade_current_4_0_x_To_indev_4_0_x{}}}. So I'm
setting the default to only 25 iterations, which should be something between 1h
and 2h for a large test file. After all, there isn't automatic detection of
Python dtests so the users will probably manually provide their own count.
Given that we have different counts for each test family, I'm removing the
default generic {{{}REPEATED_TESTS_COUNT{}}}. That can only lead to
misunderstandings if one gets used to it and then get surprised when some
especially long tests ignore it. Also, the way it worked was quite hacky and
I'm afraid in future versions of CircleCI CLI.
I'm also leaving a single {{REPEATED_TESTS_STOP_ON_FAILURE}} env var instead of
one per family of tests. This is so partly to avoid the error-prone approach
for default env vars. Also I can't see a case where we could have different
setting per family of tests, and the less vars we have the better.
The {{readme.md}} file is updated to describe the multiplexer env vars.
||Patch||CI without test changes||CI with test changes||
|[trunk|https://github.com/adelapena/cassandra/tree/17939-trunk-generate]|[j8_pre-commit|https://app.circleci.com/pipelines/github/adelapena/cassandra/2248/workflows/c53a390b-d08a-4a2a-b63d-3e6d35f9984b]
[j11_pre-commit|https://app.circleci.com/pipelines/github/adelapena/cassandra/2248/workflows/d6f51790-a17d-44cd-810f-fc679b1f5379]
[j8_separate|https://app.circleci.com/pipelines/github/adelapena/cassandra/2248/workflows/3f4a47dd-1bfa-4a5c-a384-85d708fbd41c]
[j11_separate|https://app.circleci.com/pipelines/github/adelapena/cassandra/2248/workflows/66462337-401d-4f87-9b5b-b1c016c50613]|[j8_pre-commit|https://app.circleci.com/pipelines/github/adelapena/cassandra/2247/workflows/a88e2d87-828c-4fe5-84de-d190d294c2a4]
[j11_pre-commit|https://app.circleci.com/pipelines/github/adelapena/cassandra/2247/workflows/9ff67d2f-cae6-4fc8-9537-376a6cd267e1]
[j8_separate|https://app.circleci.com/pipelines/github/adelapena/cassandra/2247/workflows/7cb93de7-b54f-46e9-aac1-ab45b736caaa]
[j11_separate|https://app.circleci.com/pipelines/github/adelapena/cassandra/2247/workflows/9f083479-f7c9-4a94-9bcb-b2f47851a5e3]|
[~jmckenzie] if you are ok with the changes I can start preparing the patches
for all the other branches (since 3.0).
> CircleCI: Automatically detect and repeat new or modified JUnit tests
> ---------------------------------------------------------------------
>
> Key: CASSANDRA-17939
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17939
> Project: Cassandra
> Issue Type: Task
> Components: CI
> Reporter: Andres de la Peña
> Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.x
>
>
> The purpose of this ticket is adding a new CircleCI job that automatically
> detects new or modified test classes and runs them repeatedly. That way we
> wouldn't need to manually specify those tests with {{.circleci/generate.sh}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
