squito commented on issue #27329: [SPARK-27324][DOC][CORE] Document configurations related to executor metrics and rename a configuration. URL: https://github.com/apache/spark/pull/27329#issuecomment-580449307 ok, yes I also see @rezasafi 's point now. The procfs metrics are only gathered if event log stage executor metrics are turned on: https://github.com/apache/spark/blob/f59685acaa3e9c227f14fe4d8f9e94a1ac664b05/core/src/main/scala/org/apache/spark/executor/ProcfsMetricsGetter.scala#L61-L65 Which makes the current name more reasonable. But that is strange behavior, particularly with https://issues.apache.org/jira/browse/SPARK-27189 -- I should have thought to change that at the same time. @wypoon I do think that there should also be behavior change as part of this, to remove that check in procfsmetricsgetter.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
