goutamadwant opened a new issue, #12483:
URL: https://github.com/apache/seatunnel/issues/12483
### Search before asking
I searched open issues and PRs. #11513 is the umbrella. #11515 routes
docs/UI/dependency checks and unit tests. #9976 moves long modules into
dedicated jobs. Neither changes which engines an E2E test runs on.
### Summary
On a connector-only PR, every `@TestTemplate` in the changed connector's E2E
module runs on all 7 PR engines: Zeta, Flink 1.13/1.15/1.18/1.20, and Spark
2.4/3.3. An engine-only PR runs the same shared connector suites on only 4
engines. The routing is inverted: the change that touches the engine
integration gets fewer engines than the change that does not.
### Evidence (dev @ deb16a3c3)
- `ContainerUtil.discoverTestContainers()`
(`seatunnel-e2e-common/.../util/ContainerUtil.java:385-430`) defaults both
`RUN_ALL_CONTAINER` and `RUN_ZETA_CONTAINER` to `true`. With `TEST_IN_PR=true`,
the PR set is the 7 ids marked `true` in `TestContainerId` (`FLINK_1_13`,
`FLINK_1_15`, `FLINK_1_18`, `FLINK_1_20`, `SPARK_2_4`, `SPARK_3_3`,
`SEATUNNEL`).
- `backend.yml` sets `RUN_ALL_CONTAINER=${{ api }}` and
`RUN_ZETA_CONTAINER=${{ engine }}` only on transform, all-connectors, iceberg,
hbase and JDBC jobs. For an engine-only PR that gives `false/true`, so those
jobs run Zeta, Flink 1.18, Flink 1.20 and Spark 3.3
(`ContainerUtil.java:408-419`).
- The `updated-modules-integration-test-part-N` jobs used by connector-only
PRs, and the dedicated jobs (kafka, redis, elasticsearch, …), do not set either
variable. So they always run all 7 engines.
- `ContainerUtil.java:420` falls through to `return true` when both flags
are `false`. So `RUN_ALL_CONTAINER=false, RUN_ZETA_CONTAINER=false` also means
"all 7 engines". No combination of flags means "fewer engines" except the
engine-PR one.
- Each engine invocation starts a new engine container
(`TestCaseInvocationContextProvider.java:118-160`). A 7-engine class therefore
pays 7 container starts plus 7 job submissions per test method.
### Proposal
1. Make the engine set an explicit input instead of two booleans with a
fall-through, for example `E2E_ENGINES=zeta,flink-1.18,flink-1.20,spark-3.3`.
Keep the current behaviour as the default so nothing changes until a job opts
in.
2. For connector-only PR jobs (the updated-modules shards and dedicated
connector jobs when triggered by `it-modules`), use Zeta + Flink 1.20 + Spark
3.3. Or use the same 4-engine set that engine PRs already get; that is the
minimum change.
3. Keep all 7 PR engines for `api=true` (translation, starter and API
changes) and for changes under `seatunnel-translation/**`.
4. The nightly (`TEST_IN_PR=false`) keeps running all 10 engines. This
depends on the nightly being able to finish, which #12467 addresses: the shared
concurrency group cancels it today.
### Expected effect [estimate]
Engine invocations per connector E2E method drop from 7 to 3–4, so
connector-PR E2E time drops by roughly 40–55%. The cost is not linear, because
container start and job submission dominate short tests. I'd confirm this on
one connector module per engine before any default changes.
### Risk and backstop
- A bug that only shows on Flink 1.13/1.15 or Spark 2.4 would be caught by
the nightly instead of the PR. Before proposing the default, I'd count from the
last 90 days of nightly and PR runs how many connector E2E failures were
specific to those engines and share the list here.
- Connector authors can still run all engines locally. A maintainer-applied
label (for example `full-e2e`) could restore all 7 on a PR.
### Coordination
- It builds on #11515: the scoped unit tests and routing flags stay the same.
- It does not overlap #9976, which is about job grouping, not engines per
test.
- It is independent of #11545 (JDK 11/17).
### Questions for dev@
- Which engine set should connector-only PRs keep? The minimal option is the
4 engines engine PRs already use.
- Is the nightly an acceptable backstop for the dropped engines once it can
finish reliably?
### Required checks and coordination
- `Build` stays the single required check. This only changes which legs run
inside it on ordinary PRs; the nightly (and a maintainer-applied full-CI
option) keep full coverage, per the boundary set in #11513.
- Complementary to #9900 (merged CI optimization) and #11074 (translation
test dependencies pulling unrelated connector modules into incremental IT
builds); it does not change either.
- This is a proposal for the dev@ discussion requested in #11513; no
implementation before there is agreement.
### Are you willing to submit a PR?
- [x] Yes, once the direction is agreed on dev@.
### Code of Conduct
- [x] I agree to follow this project's [Code of
Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
