HyukjinKwon opened a new pull request, #48404: URL: https://github.com/apache/spark/pull/48404
### What changes were proposed in this pull request? This PR is a followup of https://github.com/apache/spark/pull/47086 that makes PySpark tests skipped if tests classes are unavailable. ### Why are the changes needed? `./build/sbt package` should be able to run PySpark tests according to https://spark.apache.org/developer-tools.html and https://spark.apache.org/docs/latest/api/python/development/testing.html ### Does this PR introduce _any_ user-facing change? No, test-only. ### How was this patch tested? Manually tested: ```bash build/sbt -Phive clean package python/run-tests -k --python-executables python3 --testnames 'pyspark.sql.tests.test_dataframe' ``` ``` ... Starting test(python3): pyspark.sql.tests.test_dataframe (temp output: /.../spark/python/target/33eca5b9-23e8-4e95-9eca-9f09ce333336/python3__pyspark.sql.tests.test_dataframe__3saz2ymf.log) Finished test(python3): pyspark.sql.tests.test_dataframe (21s) ... 1 tests were skipped Tests passed in 21 seconds Skipped tests in pyspark.sql.tests.test_dataframe with python3: test_df_merge_into (pyspark.sql.tests.test_dataframe.DataFrameTests.test_df_merge_into) ... skip (0.001s) ``` ```bash build/sbt -Phive clean test:package python/run-tests -k --python-executables python3 --testnames 'pyspark.sql.tests.test_dataframe' ``` ``` Starting test(python3): pyspark.sql.tests.test_dataframe (temp output: /.../spark/python/target/710cf488-f39d-49b4-8b04-70044318ea02/python3__pyspark.sql.tests.test_dataframe___95hp4wt.log) Finished test(python3): pyspark.sql.tests.test_dataframe (23s) Tests passed in 23 seconds ``` ### Was this patch authored or co-authored using generative AI tooling? No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
