dwsmith1983 opened a new pull request, #7705:
URL: https://github.com/apache/datafusion-comet/pull/7705

   ## Which issue does this PR close?
   
   Closes #7702.
   
   Related: PR2_LINK adds a Maven restore to the Iceberg jobs, which have no 
cache step. The two are independent and can merge in either order.
   
   ## Rationale for this change
   
   Fork runs never have a Maven repository cache. Maven and cargo caches are 
saved only from `main`, and a sync of a fork's `main` adds nothing to upstream, 
so it selects no job and writes nothing. Every Maven job in a fork branch run 
then downloads its dependencies from Maven Central, which has been answering 
with HTTP 429 and failing most fork runs.
   
   ## What changes are included in this PR?
   
   - `compute()` in `dev/ci/compute-changes.py`: a fork push to 
`refs/heads/main` with no change against upstream (a sync) selects only 
`build_linux`, so `ci.yml` runs the Linux build in cache-refresh-only mode, as 
a push to `main` does upstream. That saves `Linux-java-maven-*` and `cargo-ci` 
on the fork's default branch, which every branch of the fork can restore. A 
fork `main` with its own commits keeps the PR tier.
   - `event_from_env` reads a new `REF`, which `ci.yml` sets from `github.ref`.
   - The workflows README, the contributor CI page and a few workflow comments 
describe the sync, its cost (about 40 runner-minutes once warm), and what it 
leaves cold (other lint profiles, Celeborn clients, sbt, coursier and gradle 
downloads).
   
   Contributors get the warm cache once they sync their fork's `main` after 
this merges.
   
   ## How are these changes tested?
   
   `check-ci-config.py` gains `COMPUTE_CASES`, which call `compute()` with 
empty and non-empty file lists for fork and upstream pushes, a fork branch, a 
fork scheduled run and a `fix/main` branch. The sync case fails without the 
change, and removing or weakening any guard in the new condition fails a case.
   
   On my fork, this commit pushed to `main` saved `Linux-java-maven-*` and 
`cargo-ci` there 
([run](https://github.com/dwsmith1983/datafusion-comet/actions/runs/44729423068));
 it ran the PR tier, since it differs from upstream, and the empty-diff sync 
path only runs once this is upstream. A branch push of the same commit then 
restored those entries in all 22 jobs with a Maven or cargo cache step, with no 
429s 
([run](https://github.com/dwsmith1983/datafusion-comet/actions/runs/44740834808)).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to