GitHub user holdenk opened a pull request:
https://github.com/apache/spark/pull/12233
[SPARK-13687][PYTHON] Cleanup PySpark parallelize temporary files
## What changes were proposed in this pull request?
Eagerly cleanup PySpark's temporary parallelize cleanup files rather than
waiting for shut down.
## How was this patch tested?
Unit tests
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/holdenk/spark
SPARK-13687-cleanup-pyspark-temporary-files
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/12233.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #12233
----
commit fbb0b60d6e11acb8f3aaf565f19a84a54912ee07
Author: Holden Karau <[email protected]>
Date: 2016-04-06T01:01:14Z
Eagerily unlink the tempfile we use for parallelize
commit 2e56c132355d0c9b4bab93395ca9c3ca4ad9c407
Author: Holden Karau <[email protected]>
Date: 2016-04-06T19:16:46Z
Switch to try finally, not really needed per-se but probably slightly
better practice
commit 85481396b9d5c17feaa923f4f55985e9b6158f02
Author: Holden Karau <[email protected]>
Date: 2016-04-06T19:38:39Z
Merge branch 'master' into SPARK-13687-cleanup-pyspark-temporary-files
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
