AngersZhuuuu edited a comment on issue #28036: [SPARK-26341][CORE]Expose executor memory metrics at the stage level, in the Stages tab URL: https://github.com/apache/spark/pull/28036#issuecomment-610725617 > These metrics you want to add are calculated per Executor rather than per Stage. > So I don't think it's not proper to add those metrics to StagePage. > > `Peak Execution Memory` is already shown in the task table, which is calculated per task. > `On/Off Heap Storage Memory` is also already shown in ExecutorsPage. These metrics are calculated per Executor so it's proper to be shown in ExecutorsPage. But in executor page, it make the peak during the app entire running duration. Here we can know executor peak during this current stage, it's helpful for user to know running status.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
