Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21322
I think users are responsible to call `Broadcast#destroy`, which unpersist
broadcast blocks from block manager and run user-defined driver side cleanup.
It is a valid use case to allow users to define some executor side cleanup
via `AutoCloseable`. However, I don't think we should always detect
`AutoCloseable` when removing a block, as it may break existing program and
cause perf regression. We should only do it for broadcast blocks.
A good place to do it seems to be `BlockManager.removeBroadcast`
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
