andygrove opened a new pull request, #7484:
URL: https://github.com/apache/datafusion-comet/pull/7484
## Which issue does this PR close?
None yet.
## Rationale for this change
The `linux-test` and macOS test matrices name all 180 Comet suites in two
YAML lists that have to
be kept in sync by hand. Only GitHub Actions can read them, so running a CI
group locally or on
another CI system means copying the lists and keeping the copy current.
This is the first of two PRs that move group membership onto the suites. It
adds a group tag to
every suite and a script that runs a group by tag, and leaves the workflows
as they are. #7483 then
drops the lists and has CI run the script.
## What changes are included in this PR?
- Four tag annotations in `org.apache.comet.tags`: `ScansGroup`,
`ShuffleGroup`, `ExecGroup` and
`ExpressionsGroup`. Each concrete suite carries the tag of the group it is
listed in. The 180
suites map to the same groups as the lists, checked mechanically.
- `dev/ci/run-comet-tests.sh <group> [maven args]` runs
`mvnw test -DtagsToInclude=org.apache.comet.tags.<Group>`. `--goal
install` also runs the
`spark-integration` checks.
- `dev/ci/check-suites.py` now requires exactly one group tag on every
concrete suite outside
`contrib`, including each per-Spark-version copy, and none on the suites
it ignores. It also
requires both workflow files to list each suite under its tag's group and
to list nothing else,
so a group's tag and its CI job hold the same suites until the lists go.
- The development guide says a new suite needs its tag as well as its
listing, and how to run a
group locally.
## How are these changes tested?
- `check-suites.py` and `check-ci-config.py` pass. Breaking a file on
purpose makes the check fail
for a missing or second tag, a tag on an abstract class or on an ignored
suite, a tag that
disagrees with the listing, a suite missing from a list, and a listed name
with no suite.
- `test-compile` passes with the Spark 3.4, 3.5, 4.0, 4.1 and 4.2 profiles.
- Locally, `-DtagsToInclude=org.apache.comet.tags.ExpressionsGroup` with a
wildcard naming one
`expressions` suite and one `exec` suite runs the first suite's 22 tests
and none of the second's.
- CI itself is unchanged; the workflows are not touched.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
