Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/4021#issuecomment-75313991
Overall, this looks pretty good to me, since it's a huge improvement over
the status quo of retaining accumulators indefinitely. I left one minor
comment regarding cleanup of the accumulator map entries themselves. The
actual memory-leak addressed by my ContextCleaner suggestion isn't as big a
problem as the original issue fixed here, but using a ContextCleaner cleanup
task also gives us the chance to output a useful log message when accumulators
are garbage collected.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
