wombatu-kun opened a new pull request, #19244: URL: https://github.com/apache/hudi/pull/19244
### Describe the issue this Pull Request addresses Java CI jobs intermittently fail in the "Build Project" step, before any test runs, because Maven cannot fetch a plugin or a plugin dependency from Maven Central. The most recent instance is [run 29080618811, job 86322672398](https://github.com/apache/hudi/actions/runs/29080618811/job/86322672398) (`test-spark-java17-scala-dml-tests (scala-2.13, spark4.1)` on #19241), which died on `Failed to read artifact descriptor for org.apache.maven:maven-archiver:jar:3.4.0`. Sibling jobs in the same run, on the same commit, built the same module successfully. Across the 40 most recent failed Java CI runs the same class of failure shows up three more times, and there Maven printed the underlying cause: | Run | Artifact | Cause | | --- | --- | --- | | [28840321910](https://github.com/apache/hudi/actions/runs/28840321910) | `maven-remote-resources-plugin:pom:1.5` | `status code: 403, reason phrase: Forbidden` | | [28883430014](https://github.com/apache/hudi/actions/runs/28883430014) | `scala-library:jar:2.11.11` | `Transfer failed ... 502 Bad Gateway` | | [28892322501](https://github.com/apache/hudi/actions/runs/28892322501) | `maven-remote-resources-plugin:pom:1.5` | `status code: 403, reason phrase: Forbidden` | Two things make CI unnecessarily exposed to this. First, the retry configuration is a no-op. `MVN_ARGS` passes `-Dmaven.wagon.httpconnectionManager.ttlSeconds=25 -Dmaven.wagon.http.retryHandler.count=5`, but CI runs Maven 3.9.16, which resolves artifacts through maven-resolver's native HTTP transport and never reads any `maven.wagon.*` property. Those flags did work when the build ran on Maven 3.8, whose only transport is wagon. In effect the build runs with resolver defaults, which retry only `429` and `503`, so a single `502` from Central kills a six-minute build. Second, the shared `~/.m2` cache is a stub. `actions/setup-java` derives one cache key from the hash of all `pom.xml` files, and the first job to finish wins the write. On `master` that is `validate-source`, which finishes in about 40 seconds and whose only Maven invocation is `mvn apache-rat:check`. It seals a ~10 MB local repository: `gh api repos/apache/hudi/actions/caches` reports `refs/heads/master` at 10,903,551 bytes, against 1.07 GB and 1.42 GB for two PR-scoped caches that real build jobs happened to write. Every job on every PR then restores that stub and re-downloads roughly 1 GB from Central, which is precisely the request burst that Central's CDN treats as abusive. ### Summary and Changelog CI no longer fails on a single transient gateway error from Maven Central, and it asks Central for far less. - `.github/workflows/bot.yml`, `.github/workflows/maven_artifact_validation.yml`, `.github/workflows/release_candidate_validation.yml`: replace the two ignored `maven.wagon.*` flags with their maven-resolver equivalents, and extend the retried status codes from the resolver default `429,503` to `429,500,502,503,504`. `retryHandler.count=5` and `connectionMaxTtl=25` preserve the intent of the flags they replace. - `403` is deliberately **not** retried. Central answers 403 when its CDN has blocked the runner's egress IP, often because of unrelated traffic sharing that IP, and [Sonatype's documentation](https://central.sonatype.org/faq/403-error-central/) describes continuing aggressively after a block as what escalates it. - `azure-pipelines-20230430.yml`: add the resolver flags but keep the wagon ones, since that image's Maven version is not pinned and Maven 3.8 reads only the latter. Each set is ignored by the other transport. - `.github/workflows/bot.yml`: drop `cache: maven` from `validate-source`, so the shared `~/.m2` cache on `master` gets seeded by a real build job instead of by a 40-second RAT check. - `hudi-utilities`, `hudi-kafka-connect`, `hudi-examples-flink`, `hudi-flink`: remove the hardcoded `maven-jar-plugin` version `3.1.2` so all four inherit the `3.5.0` pin from the root `pluginManagement`, matching what `hudi-aws` already does. `3.1.2` is the only reason `maven-archiver:3.4.0`, the artifact that failed above, is fetched at all; `3.5.0` uses `maven-archiver:3.6.5`, which every other module already downloads. These pins date back to 2019 and were missed by the `3.2.0` to `3.5.0` bump in #17657. Verified locally against Maven 3.9.16 with a local repository that always answers the status code under test: | Scenario | Requests before giving up | | --- | --- | | 502, resolver default list `429,503` | 1 | | 502, new list `429,500,502,503,504` | 4 (1 attempt + `count=3` retries) | | 403, new list | 1 | | 403, hypothetical list that includes 403 | 4 | `mvn help:effective-pom` on all four modules reports `maven-jar-plugin` `3.5.0` with the `test-jar` execution intact. ### Impact CI only. No runtime, public API, config, or user-facing change, and no test is added or removed. The worst case introduced by the retry change is that a genuinely unavailable artifact takes up to 45 seconds longer to fail, from the linear backoff of 3+6+9+12+15 seconds. ### Risk Level low The workflow changes are configuration only. The pom changes drop four version overrides so those modules use the already-pinned `3.5.0`, which `hudi-aws` has been building with under the same `test-jar` execution; the `-tests.jar` artifacts consumed by `hudi-cli`, `hudi-integ-test`, `hudi-kafka-connect` and `hudi-examples-flink` are unaffected. Removing `cache: maven` from `validate-source` only stops that job from writing the shared cache; the job itself resolves a single plugin. ### Documentation Update none ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable - [x] CI passed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
