Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/14617
Thanks @squito .
Regarding showing memory usage in history server. My major concern is that
putting so many block update event into event log will significantly increase
the file size and delay the replay, that's why in the current code we
deliberately bypass the block update event. And IIUC in history server it is
not necessary to show the change of used memory, only the final memory usage
before application finished will be shown on the UI. So instead of recording
and replaying all the block update events, just recording the final memory
usage of each executor is enough.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
