dwsmith1983 opened a new issue, #7702:
URL: https://github.com/apache/datafusion-comet/issues/7702
### Describe the bug
Since the pull request tier moved to contributors' forks (#7311), most fork
runs fail before building anything because Maven cannot fetch dependencies from
Maven Central. The usual first failure is the root parent POM:
```
[FATAL] Non-resolvable parent POM for
org.apache.datafusion:comet-parent-spark${spark.version.short}_${scala.binary.version}:1.3.0-SNAPSHOT:
The following artifacts could not be resolved: org.apache:apache:pom:23
(absent): Could not transfer artifact org.apache:apache:pom:23
from/to central (https://repo.maven.apache.org/maven2): status code: 429,
reason phrase: Too Many Requests
```
Some runs show `Could not find artifact org.apache:apache:pom:23 in central`
instead. The Lint Scala job fails the same way inside coursier
(`CantDownloadModule: Error downloading org.scala-sbt:sbt:latest.stable`).
It isn't limited to one fork. Over the past week the same errors show up in
fork runs from at least six contributors, and they hit lint, rust-test, the
Spark SQL and Iceberg shards, Celeborn and TPC-DS jobs more or less at random.
In ten failed runs on my fork today, every failed job was a download failure
and none had a test failure. Runs on apache's own runners (main, merge queue)
are unaffected.
The cause looks like cache scoping. Maven repository caches are restored
everywhere but saved only on push to main (see "Large caches are written on
main only" in `.github/workflows/README.md`). A fork's main never runs that
job, so the fork never has a `Linux-java-maven-*` entry. Every Maven job in
every fork run logs `Cache not found for input keys: Linux-java-maven-...` and
downloads the full dependency set from Central, roughly 30 cold jobs per push.
The four retries in `maven-bootstrap` don't survive throttling that lasts for
hours.
### Steps to reproduce
Push a branch to a fork and look at the fork run of `Comet CI (fork run)`,
or at any `CI (fork)` check that failed this week.
### Expected behavior
Fork runs restore a warm Maven repository and touch Central only when
`pom.xml` changes.
### Additional context
#1507 reported the same 429 in 2025, before fork CI. Some possible
directions:
- On fork runs, let one job per run resolve dependencies and save the Maven
repository, and have the other jobs restore it. A fork's caches count against
the fork's own budget, so the main-only rule's reason (protecting apache's
shared budget) doesn't apply there.
- Run the cache-refresh-only build on push to a fork's main, so a synced
fork has warm entries.
- Point fork runs at a Central mirror.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]