Ruifeng Zheng created SPARK-61446:
-------------------------------------
Summary: Remove per-shard compilation fallback in Maven and hosted
Python CI
Key: SPARK-61446
URL: https://issues.apache.org/jira/browse/SPARK-61446
Project: Spark
Issue Type: Improvement
Components: Project Infra
Affects Versions: 5.0.0
Reporter: Ruifeng Zheng
The reusable Maven and hosted PySpark CI workflows allow shared precompilation
failures to fall back to full compilation in each matrix shard. A deterministic
compiler error is then repeated across the shards, consuming additional GitHub
Actions runners without producing test coverage.
For example, the shared hosted PySpark precompile failed on AddCheckConstraint
pattern compiler errors, after which all ten downstream shards attempted local
compilation and failed:
https://github.com/apache/spark/actions/runs/41736782201/job/126267913955
Require successful shared precompilation and artifact download/extraction
before running the test matrices, and remove the per-shard full-build fallback.
Preserve the Maven Connect assembly build required by its tests. Transient
artifact failures should fail the affected shard and require a rerun.
Draft PR: https://github.com/apache/spark/pull/60328
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]