[
https://issues.apache.org/jira/browse/SPARK-31646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17418860#comment-17418860
]
Yongjun Zhang commented on SPARK-31646:
---------------------------------------
HI [~mauzhang], BTW, in order ro do something like
{quote}
You may register your backloggedConnections in ShuffleMetrics and update it
with "registeredConenctions - activeConnections" in ShuffleMetrics#getMetrics.
{quote}
because registeredConnections is channel level (TransportChannelHandler), and
activeConnections is at RPC level (ExternalShuffleBlockHandler), one way to do
it is create a counter for numBackloggedConnections outside somewhere, and pass
it as a parameter to both TransportChannelHandler and
ExternalShuffleBlockHandler, so that this counter can be updated by both.
However, this would make the code a bit ugly.
So I will try to derive numBackLoggedConnections outside at the metrics
monitoring system. Any better suggestion?
Thanks.
> Remove unused registeredConnections counter from ShuffleMetrics
> ---------------------------------------------------------------
>
> Key: SPARK-31646
> URL: https://issues.apache.org/jira/browse/SPARK-31646
> Project: Spark
> Issue Type: Improvement
> Components: Deploy, Shuffle, Spark Core
> Affects Versions: 3.0.0
> Reporter: Manu Zhang
> Assignee: Manu Zhang
> Priority: Minor
> Fix For: 3.0.0
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
