Github user srowen commented on the issue:
https://github.com/apache/spark/pull/16527
@wangyum one more though. I note that there are actually three instances in
this file where a collection is trimmed to match a max retained number of
things. See this quite related change:
https://github.com/apache/spark/pull/10846/files
Originally, these trims always reduced the collection by 10% of the max
retained size (which is quite consistent with your change). This could be too
_few_ in some cases and that issue fixed it. Your change goes farther to always
remove at least this amount.
I was going to suggest that all three of these should be consistent. But
perhaps the instance you are changing is the only one where repeatedly removing
just 1 element is expensive?
CC @rajeshbalamohan and @ajbozarth -- is it fairly safe to delete a bit
more than is necessary here -- it seems like it's something that is there to
support the UI for informational purposes, and hasn't mattered so much exactly
how much is retained.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
