cloud-fan commented on PR #47516: URL: https://github.com/apache/spark/pull/47516#issuecomment-2261835755
I don't see any problem with the current `SQLMetric` framework. Let me give a summary here: 1. `SQLMetric` has an initial value. When executing the SQL operator on the worker side, and the metrics are not updated at all, the metrics value will remain as the initial value and be filtered and not sent back to the driver. 2. The initial value is not always 0, as 0 can be valid as well, for max/min stats. 3. UI is not the only way to access the metrics. People can access `SQLMetric` instances in physical plan programmatically. We need to hide `-1` as it's an internal value for filtering metrics values at the worker side. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
