Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/19693
Whether shuffle write time should include the file open/close time is
debatable, also we don't know whether the actual open action is lazy or not
(depends on OS). But one downside of this change is that it makes this metric
incomparable to the old one, because we changed the semantics now.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
