Davies Liu created SPARK-5091:
---------------------------------
Summary: Hooks for PySpark tasks
Key: SPARK-5091
URL: https://issues.apache.org/jira/browse/SPARK-5091
Project: Spark
Issue Type: New Feature
Components: PySpark
Reporter: Davies Liu
Currently, it's not convenient to add package on executor to PYTHONPATH (we did
not assume the environment of driver an executor are identical).
It will be nice to have a hook to called before/after every tasks, then user
could manipulate sys.path by pre-task-hooks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
