Raajay Viswanathan created SPARK-21334:
------------------------------------------
Summary: Fix metrics for external shuffle service
Key: SPARK-21334
URL: https://issues.apache.org/jira/browse/SPARK-21334
Project: Spark
Issue Type: Bug
Components: Shuffle
Affects Versions: 2.1.1
Reporter: Raajay Viswanathan
SPARK-16405 introduced metrics for external shuffle service. However, as it is
currently there are two issues.
1. The shuffle service metrics system does not report values ever.
2. The current method for determining "blockTransferRate" is incorrect. The
entire block is assumed to be transferred once the OpenBlocks message if
processed. The actual data fetch from the disk and the succeeding transfer over
the wire happens much later when MessageEncoder invokes encode on
ChunkFetchSuccess message.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
