goutamadwant opened a new pull request, #12468: URL: https://github.com/apache/seatunnel/pull/12468
### Purpose of this pull request Some CI jobs fail on a single transient error from `repo.maven.apache.org`, which unrelated PRs then have to rerun. There are two ways this happens. **1. The wrapper jar download gets HTTP 429 for longer than `mvnw` waits.** `maven-wrapper.jar` is not committed, so every job downloads it. `mvnw` tries 5 times, 5 seconds apart, so it gives up after about 20 seconds: ``` HTTP request sent, awaiting response... 429 Too Many Requests Maven wrapper download failed, retrying in 5 seconds ... ... (5 attempts, 06:04:25 to 06:04:46) Error: Could not find or load main class org.apache.maven.wrapper.MavenWrapperMain ``` - Fork jobs: [jdbc-connectors-it-part-4](https://github.com/goutamadwant/seatunnel/actions/runs/35695254986/job/106978058855), [jdbc-connectors-it-part-1](https://github.com/goutamadwant/seatunnel/actions/runs/35695254986/job/106978058944) and [all-connectors-it-7](https://github.com/goutamadwant/seatunnel/actions/runs/35673537168/job/106586149168). - The nightly of 2026-09-20 ([35521602281](https://github.com/apache/seatunnel/actions/runs/35521602281)) failed in its `Code style` job the same way, so all later jobs were skipped. **2. Dependency downloads fail on a transient error.** The wrapper pins Maven 3.8.4, which uses the Wagon HTTP transport (wagon-http 3.4.3). By default it retries I/O errors 3 times and HTTP 429 with backoff, but it does not retry 5xx responses (`serviceUnavailableRetryStrategy.class=none`). It also reuses pooled connections for up to 300 seconds. Examples: - [all-connectors-it-2](https://github.com/goutamadwant/seatunnel/actions/runs/35673383874/job/106586061768): `Could not transfer artifact mysql:mysql-connector-java:pom:8.0.16 ... Connection reset` - [jdbc-connectors-it-part-7](https://github.com/goutamadwant/seatunnel/actions/runs/35695249537/job/106982951660): `hive-service-rpc-3.1.3.jar ... Premature end of Content-Length delimited message body` **Change** - New `.mvn/maven.config`. Maven reads it as command-line `-D` options, so it applies to every `./mvnw` call in every workflow, on Linux and Windows: ``` -Dmaven.wagon.httpconnectionManager.ttlSeconds=25 -Dmaven.wagon.http.retryHandler.count=5 -Dmaven.wagon.http.serviceUnavailableRetryStrategy.class=standard ``` - `standard` retries 408/429/500/502/503/504 up to 5 times, 1 s apart (Wagon defaults). For 429 this runs first, and Wagon's existing 429 backoff (5 s, doubling up to 180 s) still applies if these retries are used up. - `ttlSeconds=25` stops reusing pooled connections older than 25 s, so a connection dropped while idle is not reused. - These property names are read by wagon-http 3.4.3, the version bundled with Maven 3.8.4. I checked them against the class files in the 3.8.4 distribution. The `aether.*` transport properties only apply to Maven 3.9+. - `mvnw`: the wait between wrapper download attempts doubles (5, 10, 20, 40 s, about 75 s in total) instead of staying at 5 s. There are still 5 attempts. **Before vs after** (local fault-injection test described below): | Injected fault | Before | After | | --- | --- | --- | | 503 on the first 2 requests for an artifact | fails on the first 503 | succeeds on the 3rd request | | Connection reset on the first 4 requests | fails after 4 attempts (default 3 retries) | succeeds on the 5th request | | 429 on the first 2 requests | succeeds (Wagon already backs off on 429) | succeeds | | Body cut off halfway (premature end) | fails | still fails, see below | | Wrapper jar: 429 for the first 30 s | fails after 21 s, `MavenWrapperMain` not found | succeeds at 35 s (4th attempt) | ### Does this PR introduce _any_ user-facing change? No. It only affects how the Maven build downloads artifacts. Local builds that use `./mvnw` also pick up `.mvn/maven.config`. The settings only add retries and shorten connection reuse. ### How was this patch tested? - `./mvnw -v` on JDK 8 (1.8.0_172) and JDK 11 (11.0.19): Maven 3.8.4 starts. `help:evaluate -Dexpression=maven.wagon.http.serviceUnavailableRetryStrategy.class` prints `standard`, so the file is read. - The same Maven step as the `Code style` job, on JDK 8 and JDK 11 with the new config: `./mvnw -B -T 1 clean test -D"license.skipAddThirdParty"=true -pl seatunnel-ci-tools -am --no-snapshot-updates`. - Fault injection, JDK 8 and JDK 11, run once without and once with `.mvn/maven.config`: - A local HTTP server served artifacts from a warm local repository and injected the faults above for `junit` paths. - It was used as a `mirrorOf *`, with an empty `-Dmaven.repo.local`, running `./mvnw -f <tiny pom> dependency:resolve`. - Results per run are in the table. - One JDK 11 run of the cut-off-body case hung in the local test server and was stopped. It is not counted. - Wrapper: the old and new `mvnw` against a local server that answers 429 for the first 30 s. - On a fork push this change does not trigger the connector or engine matrix. `.mvn/` and `mvnw` are not in any route. The fork run exercises the change in the `Code style` job (wrapper download plus a Maven build with the new config). Not covered: a response body that breaks partway through (`Premature end of Content-Length`, or a reset during the body) is still not retried. Neither Wagon nor resolver 1.6.3 retries that case. The Windows wrapper script `mvnw.cmd` has no retry loop, and I left it unchanged. The `setup-java` Maven cache (key on all poms, no restore key) is also unchanged here. That change would touch every job, so it is better done separately. ### Check list * [ ] If any new Jar binary package adding in your PR, please add License Notice according [New License Guide](https://github.com/apache/seatunnel/blob/dev/docs/en/developer/new-license.md) * [ ] If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs * [ ] If necessary, please update `incompatible-changes.md` to describe the incompatibility caused by this PR. * [ ] If you are contributing the connector code, please check that the following files are updated: 1. Update [plugin-mapping.properties](https://github.com/apache/seatunnel/blob/dev/plugin-mapping.properties) and add new connector information in it 2. Update the pom file of [seatunnel-dist](https://github.com/apache/seatunnel/blob/dev/seatunnel-dist/pom.xml) 3. Add ci label in [label-scope-conf](https://github.com/apache/seatunnel/blob/dev/.github/workflows/labeler/label-scope-conf.yml) 4. Add e2e testcase in [seatunnel-e2e](https://github.com/apache/seatunnel/tree/dev/seatunnel-e2e/seatunnel-connector-v2-e2e/) 5. Update connector [plugin_config](https://github.com/apache/seatunnel/blob/dev/config/plugin_config) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
