[
https://issues.apache.org/jira/browse/CASSANDRA-17939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17613712#comment-17613712
]
Andres de la Peña edited comment on CASSANDRA-17939 at 10/6/22 5:02 PM:
------------------------------------------------------------------------
I'm glad you both like it :)
On a further refinement, I have added support for specifying particular method
names in the lists of test classes that are manually passed to the companion
jobs. For example:
{code:java}
.circleci/generate.sh -m \
-e REPEATED_TESTS_COUNT=5 \
-e
REPEATED_UTESTS=org.apache.cassandra.cql3.DurationTest#testAddTo,org.apache.cassandra.cql3.validation.entities.DateTypeTest
\
-e REPEATED_UTESTS_COUNT=10 \
-e
REPEATED_JVM_DTESTS=org.apache.cassandra.distributed.test.AlterTest#getAndSetCompressionParametersTest
\
-e REPEATED_JVM_DTESTS_COUNT=10 \
-e
REPEATED_UTESTS_LONG=org.apache.cassandra.io.sstable.CQLSSTableWriterLongTest#testWideRow
\
-e REPEATED_UTESTS_LONG_COUNT=2
{code}
||PR||CI||
|[trunk|https://github.com/apache/cassandra/pull/1899]|[j8_pre-commit|https://app.circleci.com/pipelines/github/adelapena/cassandra/2157/workflows/d470fdb7-c321-4f18-82b5-6eac6d7c0992]
[j11_pre-commit|https://app.circleci.com/pipelines/github/adelapena/cassandra/2157/workflows/2bd34d9e-a32e-460d-a075-8ffdf7db4f09]
[j8_separate|https://app.circleci.com/pipelines/github/adelapena/cassandra/2157/workflows/3018332a-0c93-4e27-918f-48e4166de932]
[j11_separate|https://app.circleci.com/pipelines/github/adelapena/cassandra/2157/workflows/56c80e7d-d3f4-4e46-ba9a-021e80cf9314]|
We can see on [the artifacts
tabs|https://app.circleci.com/pipelines/github/adelapena/cassandra/2157/workflows/d470fdb7-c321-4f18-82b5-6eac6d7c0992/jobs/21753/artifacts]
that the jobs are running the manually specified test classes and methods,
together with the automatically detected tests that come from [this test
commit|https://github.com/adelapena/cassandra/commit/6639cc5d1c39080da271ef8a5c8d8366a5b713af].
With this changes the new companion jobs are mostly equivalent in functionality
to the classic utest multiplexer. The only relevant thing that is missing in
the new companion jobs is that the classic utest multiplexer is able to run any
Ant target, and we don't have regular not-multiplexer jobs for some Ant
targets, like {{test-cdc}} or {{{}msg-ser-test{}}}.
I guess that someday we'll manage to get CircleCI jobs for every possible type
of test, all of them with its companion multiplexer job. That day we could
probably get rid of the classic utest multiplexer.
In the meantime, I wonder if we should remove the classic utest multiplexer
from the pre-commit workflows and leave it only on the separate workflows. Or
perhaps move it to a separate workflow dedicated to that kind of debugging.
After all, the sole purpose of the classic utest multiplexer will be
reproducing existing flakies.
was (Author: adelapena):
I'm glad you both like it :)
On a further refinement, I have added support for specifying particular method
names in the lists of test classes that are manually passed to the companion
jobs. For example:
{code:java}
.circleci/generate.sh -m \
-e REPEATED_TESTS_COUNT=5 \
-e
REPEATED_UTESTS=org.apache.cassandra.cql3.DurationTest#testAddTo,org.apache.cassandra.cql3.validation.entities.DateTypeTest
\
-e REPEATED_UTESTS_COUNT=10 \
-e
REPEATED_JVM_DTESTS=org.apache.cassandra.distributed.test.AlterTest#getAndSetCompressionParametersTest
\
-e REPEATED_JVM_DTESTS_COUNT=10 \
-e
REPEATED_UTESTS_LONG=org.apache.cassandra.io.sstable.CQLSSTableWriterLongTest#testWideRow
\
-e REPEATED_UTESTS_LONG_COUNT=2
{code}
||PR||CI||
|[trunk|https://github.com/apache/cassandra/pull/1899]|[j8_pre-commit|https://app.circleci.com/pipelines/github/adelapena/cassandra/2157/workflows/d470fdb7-c321-4f18-82b5-6eac6d7c0992]
[j11_pre-commit|https://app.circleci.com/pipelines/github/adelapena/cassandra/2157/workflows/2bd34d9e-a32e-460d-a075-8ffdf7db4f09]
[j8_separate|https://app.circleci.com/pipelines/github/adelapena/cassandra/2157/workflows/3018332a-0c93-4e27-918f-48e4166de932]
[j11_separate|https://app.circleci.com/pipelines/github/adelapena/cassandra/2157/workflows/56c80e7d-d3f4-4e46-ba9a-021e80cf9314]|
We can see on [the artifacts
tabs|https://app.circleci.com/pipelines/github/adelapena/cassandra/2157/workflows/d470fdb7-c321-4f18-82b5-6eac6d7c0992/jobs/21753/artifacts]
that the jobs are running the manually specified test classes and methods,
together with the automatically detected tests that come from [this test
commit|https://github.com/adelapena/cassandra/commit/6639cc5d1c39080da271ef8a5c8d8366a5b713af].
With this changes the new companion jobs are mostly equivalent in functionality
to the classic utest multiplexer. The only relevant thing that is missing in
the new companion jobs is that the classic utest multiplexer is able to run any
Ant target, and we don't have regular not-multiplexer jobs for some Ant
targets, like {{test-cdc}} or {{{}msg-ser-test{}}}.
I guess that someday we'll manage to get CircleCI jobs for every possible type
of test, all of them with its companion multiplexer job. That day we could
probably get rid of the classic utest multiplexer. In the meantime, I wonder if
we should remove the classic utest multiplexer from the pre-commit workflows
and leave it only on the separate workflows. After all, its sole purpose will
be reproducing existing flakies, and we usually do that with the separate
workflows.
> CircleCI: Automatically detect and repeat new or modified JUnit tests
> ---------------------------------------------------------------------
>
> Key: CASSANDRA-17939
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17939
> Project: Cassandra
> Issue Type: Task
> Components: CI
> Reporter: Andres de la Peña
> Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.x
>
>
> The purpose of this ticket is adding a new CircleCI job that automatically
> detects new or modified test classes and runs them repeatedly. That way we
> wouldn't need to manually specify those tests with {{.circleci/generate.sh}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
