dwsmith1983 opened a new pull request, #7706:
URL: https://github.com/apache/datafusion-comet/pull/7706

   ## Which issue does this PR close?
   
   Part of #7702. #7705 closes it; the two are independent and can merge in 
either order.
   
   ## Rationale for this change
   
   The three Iceberg Spark test jobs that run `./mvnw install` 
(`iceberg-spark`, `iceberg-spark-extensions`, `iceberg-spark-runtime`) restore 
no Maven cache. Every run downloads the full dependency set from Maven Central, 
upstream and on forks alike, and Central has been throttling those downloads 
with HTTP 429.
   
   ## What changes are included in this PR?
   
   - A restore-only `actions/cache/restore` of `~/.m2/repository` before 
`./mvnw install` in each of the three jobs, with the `java-maven` key that 
main's TPC-H/TPC-DS jobs save.
   - A second restore key, `spark-sql-`, which `setup-spark-builder` saves from 
the nightly. The nightly's Delta caches often evict main's `java-maven` entry 
from the cache budget, and the `spark-sql` entry usually survives.
   - No save step, per "Large caches are written on main only" in the workflows 
README.
   
   ## How are these changes tested?
   
   `check-ci-config.py` and actionlint pass. On my fork, with a `java-maven` 
entry on the fork's `main`, all six Iceberg jobs restored it and passed with no 
429s 
([run](https://github.com/dwsmith1983/datafusion-comet/actions/runs/44750207091)).
 Without this change, the same jobs on the same fork downloaded everything, and 
shard-2 failed on 429s 
([run](https://github.com/dwsmith1983/datafusion-comet/actions/runs/44740834808)).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to