HyukjinKwon edited a comment on pull request #22480:
URL: https://github.com/apache/spark/pull/22480#issuecomment-639338265
@pquentin, there's no Python thread running in the parent `daemon.py` before
the `fork()`. Also the forked workers are usually reused once they are on in
most cases.
HyukjinKwon edited a comment on pull request #22480:
URL: https://github.com/apache/spark/pull/22480#issuecomment-639212453
@pquentin, yes, it's kind of difficult to avoid in PySpark side for now. The
problem isn't solely because we use `fork()` but it binds to other conditions.
I didn't
