Hi all,
Our PR CI currently runs the full test suite on both JDK 17 and JDK 21 for
every heavy workflow (spark, flink, java, hive, kafka-connect,
delta-conversion). This doubles PR runner-minutes on the shared ASF Actions
pool. spark-ci alone expands to 22 matrix jobs, which exceeds the infra
max-parallel ceiling of 20 and spills into a second wave.
I would like to propose gating pull_request runs on JDK 17 only (our
minimum supported version, and the JDK that already writes the shared
Gradle cache), while keeping the full JDK 17 + 21 matrix on push to main,
plus optionally a nightly scheduled full-matrix JDK 21 run. Concretely, the
jvm matrix becomes event-conditional, for example: jvm: ${{
github.event_name == 'pull_request' && fromJSON('[17]') || fromJSON('[17,
21]') }}
This roughly halves PR runner time across all of the heavy workflows and
brings spark-ci back under the 20-job ceiling in a single wave. Caching is
unaffected, since the canonical writer stays java-ci build-checks on JDK 17
on main. The tradeoff is that a JDK-21-only regression would surface at
merge time or in the nightly run rather than on the PR itself. To bound
that, we could keep a small JDK 21 smoke leg on PRs (for example core-tests
only), and/or rely on a nightly full run.
Does the project want to pursue this, and if so which variant: 17-only PRs
with a nightly 21 run, or 17-only PRs plus a small 21 smoke subset?
Thanks,
Vova Kolmakov
