[
https://issues.apache.org/jira/browse/SPARK-25642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Parth Gandhi updated SPARK-25642:
---------------------------------
Description: Recently, the ability to expose the metrics for YARN Shuffle
Service was added as part of
[SPARK-18364|[https://github.com/apache/spark/pull/22485]]. We need to add some
metrics to be able to determine the number of active connections as well as
open connections to the external shuffle service to benchmark network and
connection issues on large cluster environments. (was: Recently, the ability
to expose the metrics for YARN Shuffle Service was added as part of
[SPARK-18364|[https://github.com/apache/spark/pull/22485].] We need to add some
metrics to be able to determine the number of active connections as well as
open connections to the external shuffle service to benchmark network and
connection issues on large cluster environments.)
> Add new Metrics in External Shuffle Service to help determine Network
> performance and Connection Handling capabilities of the Shuffle Service
> ---------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-25642
> URL: https://issues.apache.org/jira/browse/SPARK-25642
> Project: Spark
> Issue Type: Improvement
> Components: Shuffle, Spark Core
> Affects Versions: 2.4.0
> Reporter: Parth Gandhi
> Priority: Minor
>
> Recently, the ability to expose the metrics for YARN Shuffle Service was
> added as part of [SPARK-18364|[https://github.com/apache/spark/pull/22485]].
> We need to add some metrics to be able to determine the number of active
> connections as well as open connections to the external shuffle service to
> benchmark network and connection issues on large cluster environments.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
