Github user zsxwing commented on the pull request:
https://github.com/apache/spark/pull/11257#issuecomment-186343946
> @zsxwing Most documentations on phantom reference suggests a separate
daemon thread to do the cleanup. I can try adding the phantom reference removal
from the queue in the same function without a thread. Would that work?
The problem is it will create one thread for each PythonBroadcast. If a
task has hundreds of `PythonBroadcast`s, it will create hundreds of threads.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
