anew commented on PR #56686:
URL: https://github.com/apache/spark/pull/56686#issuecomment-4928712422
The CI failure in test_import_spark_libraries
(pyspark.tests.test_import_spark.ImportSparkTest) is unrelated to this PR —
it's a pre-existing regression on master.
That test asserts import pyspark pulls in only stdlib + py4j (to keep import
fast). It now fails with:
> AssertionError: Unexpected 3rd party package 'numpy' imported during
'import pyspark'
The cause is [SPARK-57760], which moved import numpy as np from inside a
method to module top-level in
python/pyspark/sql/streaming/stateful_processor_api_client.py.
The failure reproduces on a clean checkout of current master
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
