felipepessoto commented on PR #12388: URL: https://github.com/apache/gluten/pull/12388#issuecomment-5145623546
@philo-he @zhouyuan This is green now and I think it's ready — the pipeline is fully self-contained and the last run passed end to end (8/8 shards, slowest 102 min, ~10 job-hours). What's in this PR: - **Own workflow.** Delta CI no longer runs inside `velox_backend_x86.yml`, so a change to the Delta pipeline no longer drags in the ~50-job TPC-H/DS + Spark-UT matrix (~1750 runner-min). Per-PR it's gated by a `paths:` filter, plus a nightly run at 05:00 UTC for everything the filter skips. - **8 shards x 4 forks**, matching delta-io/delta's own `spark_test.yaml`. Measured: slowest shard 163 → 102 min (-37%) for +5% billed hours. - Baseline at 735 known failures, 0 regressions. Proposed follow-ups, happy to file issues for these: 1. **Share the native lib** between the two workflows (~10 min on `gluten-delta` PRs, which currently build it twice). Needs a cache key covering `cpp/**`, build flags and the base image — a stale hit there means silently testing the wrong binary. 2. **Spark 3.5 leg**, nightly-only at first. The blocker is the baseline: each Spark/Delta combination needs its own file and a bootstrap run. Once 3.5 is covered here, the imported Delta suite copies @malinjawi identified become removable. 3. **`run-delta-ci` label**, if you want the opt-in back. I dropped it since it can't be expressed as a `paths:` filter and the label didn't exist on the repo; `workflow_dispatch` covers the same need today. Does this look right to you both? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
