Github user squito commented on the issue:
https://github.com/apache/spark/pull/14617
> AFAIK we don't record block update events in history server, so we could
not calculate the used memory from event log.
good point, sorry I had totally forgotten about. Seems like this could be
fixed w/ the info already in `TaskEnd` events though? separate from this
change, but would be a nice improvement.
I'd still like to see some test of non-zero values -- how about here:
https://github.com/apache/spark/blob/a8a765b3f302c078cb9519c4a17912cd38b9680c/core/src/test/scala/org/apache/spark/ui/UISeleniumSuite.scala#L111
you could add caching of an off-heap rdd too, and update that to test for
non-zero sizes in the right places.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
