Github user holdenk commented on the issue:
https://github.com/apache/spark/pull/15178
@viirya so what I mean is right now the I think executors will fetch the
blocks and they might not get cleaned up once the broadcast is destroyed. You
could add a test to see if the blocks are everywhere after unpersist. The other
question is if someone broadcasts a cached RDD and then unpersists it, I'm
worried it might clean up the blocks on the executor. You could add a test here
to see if you can use the broadcast after unpersist of the backing RDD (or if
we don't want to support that use case add a note about it to the docs and make
sure it fails in a clear manner).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
