[
https://issues.apache.org/jira/browse/CASSANDRA-17939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17612943#comment-17612943
]
Andres de la Peña edited comment on CASSANDRA-17939 at 10/5/22 10:58 AM:
-------------------------------------------------------------------------
[The proposed PR|https://github.com/apache/cassandra/pull/1899] adds new jobs
for automatically repeating new or modified JUnit tests. Each regular JUnit job
has a companion job to run the tests on that suite that are new or modified,
according to the results of {{{}git diff --diff-filter=AMR{}}}. Each pair of
all/repeated jobs is subjected to the same approval step, so the new workflow
has more jobs but an identical number of approval steps.
These CI results use a synthetic patch doing some arbitrary changes on every
test family, so we can verify how the new jobs work:
||PR||Test patch||CI||
|[trunk|https://github.com/apache/cassandra/pull/1899]|[trunk|https://github.com/adelapena/cassandra/tree/17939-trunk-dev-test]|[j8_pre-commit|https://app.circleci.com/pipelines/github/adelapena/cassandra/2135/workflows/e54ff736-015e-4002-b170-6ea826b591d1]
[j11_pre-commit|https://app.circleci.com/pipelines/github/adelapena/cassandra/2135/workflows/c2811225-9a38-4216-9b7e-b15c9bb6d53c]
[j8_separate|https://app.circleci.com/pipelines/github/adelapena/cassandra/2135/workflows/13593914-4cdf-4f88-b5be-3201908e3e01]
[j11_separate|https://app.circleci.com/pipelines/github/adelapena/cassandra/2135/workflows/7c7d779d-0d22-4977-abca-0ddf01adb327]|
I'll add PRs for the remaining branches once we settle on a design.
This ticket is focused only on JUnit tests. Similar jobs for Python tests can
be added in a separate ticket if we see value on it. However, that would be
quite more challenging because we would need to find a way to get the class and
method name of the new or modified test files identified by {{{}git diff{}}}.
We probably shouldn't run entire {{.py}} files in the same way that we run
{{.java}} files because Python tests are more resource intensive and their test
files use to be quite crowded. To further complicate things, the names of some
Python tests are automatically generated and don't appear on the compared
files. Also, adding Python tests is not as common as adding JUnit tests these
days, so there is probably less value on automatically multiplexing them.
was (Author: adelapena):
[The proposed PR|https://github.com/apache/cassandra/pull/1899] adds new jobs
for automatically repeating new or modified JUnit tests. Each regular JUnit job
has a companion job to run the tests on that suite that are new or modified,
according to the results of {{{}git diff --diff-filter=AMR{}}}. Each pair of
all/repeated jobs is subjected to the same approval step, so the new workflow
has more jobs but an identical number of approval steps.
These CI results use a synthetic patch doing some arbitrary changes on every
test family, so we can verify how the new jobs work:
||PR||Test patch||CI||
|[trunk|https://github.com/apache/cassandra/pull/1899]|[trunk|https://github.com/adelapena/cassandra/tree/17939-trunk-dev-test]|[j8_pre-commit|https://app.circleci.com/pipelines/github/adelapena/cassandra/2135/workflows/e54ff736-015e-4002-b170-6ea826b591d1]
[j11_pre-commit|https://app.circleci.com/pipelines/github/adelapena/cassandra/2135/workflows/c2811225-9a38-4216-9b7e-b15c9bb6d53c]
[j8_separate|https://app.circleci.com/pipelines/github/adelapena/cassandra/2135/workflows/13593914-4cdf-4f88-b5be-3201908e3e01]
[j11_separate|https://app.circleci.com/pipelines/github/adelapena/cassandra/2135/workflows/7c7d779d-0d22-4977-abca-0ddf01adb327]|
This ticket is focused only on JUnit tests. Similar jobs for Python tests can
be added in a separate ticket if we see value on it. However, that would be
quite more challenging because we would need to find a way to get the class and
method name of the new or modified test files identified by {{{}git diff{}}}.
We probably shouldn't run entire {{.py}} files in the same way that we run
{{.java}} files because Python tests are more resource intensive and their test
files use to be quite crowded. To further complicate things, the names of some
Python tests are automatically generated and don't appear on the compared
files. Also, adding Python tests is not as common as adding JUnit tests these
days, so there is probably less value on automatically multiplexing them.
> CircleCI: Automatically detect and repeat new or modified JUnit tests
> ---------------------------------------------------------------------
>
> Key: CASSANDRA-17939
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17939
> Project: Cassandra
> Issue Type: Task
> Components: CI
> Reporter: Andres de la Peña
> Priority: Normal
>
> The purpose of this ticket is adding a new CircleCI job that automatically
> detects new or modified test classes and runs them repeatedly. That way we
> wouldn't need to manually specify those tests with {{.circleci/generate.sh}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
