Hyukjin Kwon created SPARK-22959:
------------------------------------
Summary: Configuration to select the modules for daemon and worker
in PySpark
Key: SPARK-22959
URL: https://issues.apache.org/jira/browse/SPARK-22959
Project: Spark
Issue Type: Improvement
Components: PySpark
Affects Versions: 2.3.0
Reporter: Hyukjin Kwon
We are now forced to use {{pyspark/daemon.py}} and {{pyspark/worker.py}} in
PySpark tests.
This doesn't allow a custom modification for it and it's sometimes hard to
debug what happens in Python worker processes.
This is actually related with SPARK-7721 too as somehow Coverage is unable to
detect the coverage from {{os.fork}}.
This is also related with SPARK-20368. With this simple workaround, advanced
users will be able to do a lot of pluggable supports.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
