[
https://issues.apache.org/jira/browse/SPARK-59906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-59906:
-----------------------------------
Labels: pull-request-available (was: )
> Run relevant tests when JVM-launched Python worker scripts change under smart
> test selection
> --------------------------------------------------------------------------------------------
>
> Key: SPARK-59906
> URL: https://issues.apache.org/jira/browse/SPARK-59906
> Project: Spark
> Issue Type: Bug
> Components: PySpark
> Affects Versions: 5.0.0
> Reporter: Tian Gao
> Priority: Major
> Labels: pull-request-available
>
> PySpark smart test selection (PySparkBaseTestCase, driven by
> PYSPARK_CHANGED_FILES) keeps a test only when the changed file's module is
> reachable from the test module in the static import graph built by grimp.
> Scripts that the JVM launches as separate processes (pyspark.worker,
> pyspark.daemon, pyspark.worker_util, and the pyspark.sql.worker.* runners)
> are never imported by any test module, so a change touching only those files
> is treated as irrelevant to every test and all of them are skipped --
> including the UDF, pandas, Arrow, UDTF, and Python data source tests that
> actually exercise the changed code. This maps each such entry-point script to
> the API module whose use triggers it (the worker/daemon files are treated as
> relevant to every test), so the corresponding tests are selected.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
