goutamadwant opened a new pull request, #12470:
URL: https://github.com/apache/seatunnel/pull/12470
### Purpose of this pull request
When a unit or integration test job in `backend.yml` fails, the Surefire and
Failsafe reports are lost with the runner. Maven's message says "Please refer
to .../target/failsafe-reports", but those files no longer exist. The only way
to find the failing test is the job log, which for connector and engine IT jobs
is 150k–430k lines, since the test and container output goes straight to the
console. `backend.yml` has no `upload-artifact` step and writes nothing to the
job summary.
**Change**
- New composite action `.github/actions/upload-test-reports/action.yml` with
two steps:
1. **List failed tests in the job summary.** A short Python script
(`shell: python`, so it also runs on the Windows unit-test leg) reads
`**/target/{surefire,failsafe}-reports/TEST-*.xml`. For each failed or errored
test case it writes a row to `$GITHUB_STEP_SUMMARY` with `class#method`,
`failure`/`error`, and the first line of the message. The list is capped at 100
rows. If there is no failed test case, for example after a compile error, a
dependency download failure or a timeout, it says so.
2. **Upload the report directories** with `actions/upload-artifact@v4`,
the same action and tag the repo already uses. Settings: `retention-days: 7`,
`if-no-files-found: ignore`. The artifact is named
`test-reports-<job>-<java>-<os>-attempt-<n>`, so matrix legs and reruns never
collide.
- The 46 test jobs in `backend.yml` (unit-test, benchmark-test, the
`updated-modules` shards, and all engine, transform and connector IT jobs) get
the same final step:
```yaml
- name: Upload test reports
if: failure()
uses: ./.github/actions/upload-test-reports
with:
id: ${{ matrix.java }}-${{ matrix.os }}
```
**Before vs after**
- Before: a red `kafka-connector-it (8, ubuntu-latest)` means downloading or
scrolling a ~400k-line log to find the test.
- After: the job page shows, for example:
| Test | Result | Message |
| --- | --- | --- |
| `org.apache.seatunnel.e2e...KafkaIT#testX` | failure | expected:
<1> but was: <2> |
The XML reports, including stack traces and captured output, can be
downloaded from the run's artifacts for 7 days.
Jobs that pass are unchanged: the step is skipped. The step only runs after
a failure, so it adds nothing to green runs.
### Does this PR introduce _any_ user-facing change?
No. CI only.
### How was this patch tested?
- `actionlint` 1.7.7 on `backend.yml`, which also checks the inputs of the
local action: no findings. YAML parsed with PyYAML. Every one of the 46 jobs
ends with the new step, and no other job changed.
- The summary script was run locally (Python 3.12) against real reports. I
built a small module with `maven-surefire-plugin` and `maven-failsafe-plugin`
2.22.2 and JUnit 5.9.0 (the versions in the root pom), on JDK 8 with Maven
3.8.4, with one failing unit test and one IT that throws. The summary listed
`demo.DemoTest#failsAssertion` (failure, message escaped for `|` and `<>`) and
`demo.DemoIT#throwsError` (error, first line only). In a directory with no
reports it printed the "no failed test cases found" note.
- On a fork push this change does not start any test job, because `.github/`
changes are lightweight in `ci_scope.py`, so the fork run of this branch does
not exercise the new step. To exercise it, I ran a separate throwaway fork
branch that adds a demo workflow: one job on `ubuntu-latest` and
`windows-latest` builds a module with failing tests and then calls this action.
The summary table and the artifact are here: [fork run
36136811640](https://github.com/goutamadwant/seatunnel/actions/runs/36136811640).
On both `ubuntu-latest` and `windows-latest`, the summary lists
`demo.DemoTest#failsAssertion` (failure) and `demo.DemoIT#throwsError` (error).
The `test-reports-demo-8-ubuntu-latest-attempt-1` and
`test-reports-demo-8-windows-latest-attempt-1` artifacts contain both report
directories.
Not verified: behaviour on a real failing connector IT job upstream. The
first failing PR build after merge will show it.
Notes:
- This is separate from the build-level timing summary proposed in #11515,
which covers job durations, not test cases.
- Jobs cancelled by `timeout-minutes` do not match `failure()` and are not
covered here. Capturing hung tests is a separate topic (#12446).
### Check list
* [ ] If any new Jar binary package adding in your PR, please add License
Notice according
[New License
Guide](https://github.com/apache/seatunnel/blob/dev/docs/en/developer/new-license.md)
* [ ] If necessary, please update the documentation to describe the new
feature. https://github.com/apache/seatunnel/tree/dev/docs
* [ ] If necessary, please update `incompatible-changes.md` to describe the
incompatibility caused by this PR.
* [ ] If you are contributing the connector code, please check that the
following files are updated:
1. Update
[plugin-mapping.properties](https://github.com/apache/seatunnel/blob/dev/plugin-mapping.properties)
and add new connector information in it
2. Update the pom file of
[seatunnel-dist](https://github.com/apache/seatunnel/blob/dev/seatunnel-dist/pom.xml)
3. Add ci label in
[label-scope-conf](https://github.com/apache/seatunnel/blob/dev/.github/workflows/labeler/label-scope-conf.yml)
4. Add e2e testcase in
[seatunnel-e2e](https://github.com/apache/seatunnel/tree/dev/seatunnel-e2e/seatunnel-connector-v2-e2e/)
5. Update connector
[plugin_config](https://github.com/apache/seatunnel/blob/dev/config/plugin_config)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
