Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/1680#issuecomment-50803845
Another thing to test: a crash in a PySpark worker should never crash the
daemon:
```python
>>> rdd = sc.parallelize(range(10000))
>>> rdd.count() # Force the workers / daemon to launch
# Crash the workers abruptly, with no error-propagation back to the JVM
>>> import os
>>> rdd.mapPartitions(lambda x: os._exit(-1)).count()
```
This seems to be mis-handled in my current code, since a worker crash
throws a RuntimeError from the SIGCHLD handler, killing the daemon.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
