jin xing created SPARK-19937:
--------------------------------
Summary: Collect metrics of block sizes when shuffle.
Key: SPARK-19937
URL: https://issues.apache.org/jira/browse/SPARK-19937
Project: Spark
Issue Type: Improvement
Components: Shuffle
Affects Versions: 2.1.0
Reporter: jin xing
This improvement is related to SPARK-19659 and also documented there. Metrics
of blocks sizes(when shuffle) should be collected for later analysis. This is
helpful for analysis when skew situations or OOM happens(though
maxBytesInFlight is set).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
