dwsmith1983 opened a new issue, #7702:
URL: https://github.com/apache/datafusion-comet/issues/7702

   ### Describe the bug
   
   Since the pull request tier moved to contributors' forks (#7311), most fork 
runs fail before building anything because Maven cannot fetch dependencies from 
Maven Central. The usual first failure is the root parent POM:
   
   ```
   [FATAL] Non-resolvable parent POM for 
org.apache.datafusion:comet-parent-spark${spark.version.short}_${scala.binary.version}:1.3.0-SNAPSHOT:
   The following artifacts could not be resolved: org.apache:apache:pom:23 
(absent): Could not transfer artifact org.apache:apache:pom:23
   from/to central (https://repo.maven.apache.org/maven2): status code: 429, 
reason phrase: Too Many Requests
   ```
   
   Some runs show `Could not find artifact org.apache:apache:pom:23 in central` 
instead. The Lint Scala job fails the same way inside coursier 
(`CantDownloadModule: Error downloading org.scala-sbt:sbt:latest.stable`).
   
   It isn't limited to one fork. Over the past week the same errors show up in 
fork runs from at least six contributors, and they hit lint, rust-test, the 
Spark SQL and Iceberg shards, Celeborn and TPC-DS jobs more or less at random. 
In ten failed runs on my fork today, every failed job was a download failure 
and none had a test failure. Runs on apache's own runners (main, merge queue) 
are unaffected.
   
   The cause looks like cache scoping. Maven repository caches are restored 
everywhere but saved only on push to main (see "Large caches are written on 
main only" in `.github/workflows/README.md`). A fork's main never runs that 
job, so the fork never has a `Linux-java-maven-*` entry. Every Maven job in 
every fork run logs `Cache not found for input keys: Linux-java-maven-...` and 
downloads the full dependency set from Central, roughly 30 cold jobs per push. 
The four retries in `maven-bootstrap` don't survive throttling that lasts for 
hours.
   
   ### Steps to reproduce
   
   Push a branch to a fork and look at the fork run of `Comet CI (fork run)`, 
or at any `CI (fork)` check that failed this week.
   
   ### Expected behavior
   
   Fork runs restore a warm Maven repository and touch Central only when 
`pom.xml` changes.
   
   ### Additional context
   
   #1507 reported the same 429 in 2025, before fork CI. Some possible 
directions:
   
   - On fork runs, let one job per run resolve dependencies and save the Maven 
repository, and have the other jobs restore it. A fork's caches count against 
the fork's own budget, so the main-only rule's reason (protecting apache's 
shared budget) doesn't apply there.
   - Run the cache-refresh-only build on push to a fork's main, so a synced 
fork has warm entries.
   - Point fork runs at a Central mirror.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to