zhengruifeng commented on code in PR #58274:
URL: https://github.com/apache/spark/pull/58274#discussion_r3985609840
##########
.github/workflows/build_and_test.yml:
##########
@@ -155,7 +155,7 @@ jobs:
docs=true
java25=true
else
- pyspark_install=false
+ pyspark_periodic=false
Review Comment:
By “factor out,” I meant moving it out of the precompiled-artifact-dependent
PySpark CI matrix, not moving the test files. These tests could run in a
lightweight Python-only job without waiting for Spark precompile, downloading
and extracting its large artifact, or setting up Java/SBT.
However, I missed that `pyspark-periodic` also enables `dev/run-pip-tests`,
which packages assembly jars and runs `spark-submit`. That part does need the
compiled artifact. A clean split would move the upstream/import/install tests
to the lightweight job while keeping the packaging test artifact-dependent.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
