[
https://issues.apache.org/jira/browse/SPARK-21334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-21334:
------------------------------------
Assignee: (was: Apache Spark)
> Fix metrics for external shuffle service
> ----------------------------------------
>
> Key: SPARK-21334
> URL: https://issues.apache.org/jira/browse/SPARK-21334
> Project: Spark
> Issue Type: Bug
> Components: Shuffle
> Affects Versions: 2.1.1
> Reporter: Raajay Viswanathan
> Labels: external-shuffle-service
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> SPARK-16405 introduced metrics for external shuffle service. However, as it
> is currently there are two issues.
> 1. The shuffle service metrics system does not report values ever.
> 2. -The current method for determining "blockTransferRate" is incorrect. The
> entire block is assumed to be transferred once the OpenBlocks message if
> processed. The actual data fetch from the disk and the succeeding transfer
> over the wire happens much later when MessageEncoder invokes encode on
> ChunkFetchSuccess message.-
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
