[
https://issues.apache.org/jira/browse/SPARK-31646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17416225#comment-17416225
]
Yongjun Zhang edited comment on SPARK-31646 at 9/16/21, 4:35 PM:
-----------------------------------------------------------------
Thanks a lot for your quick response [~mauzhang]. Sorry my question was a bit
not clear earlier.
When you made the comment
{quote}
It's {{registeredConnections}} counter created in {{TransportContext}} that's
really counting the numbers and it's misleading for people who want to add new
metrics like {{registeredConnections}}.
{quote}
, you meant to use registeredConnections to mean something different than what
you reverted with this jira. Would you please explain
1. what's your interpretation of the reverted version of registeredConnections
besides "counting numbers"? is it not useful at all?
2. what your definition is for "new metrics like {{registeredConnections}}."?
is it not counting? if it's counting, why the reverted implementation doesn't
serve the purpose?
BTW, my understanding is that 1, all executors running on a given host would
register with the remote shuffle service on the same host, 2, executors only
register with shuffle service on the same host where the executors are running
at, but not shuffle service running on other hosts. Is this understanding
correct?
Thanks.
was (Author: yzhangal):
Thanks a lot for your quick response [~mauzhang]. Sorry my question was a bit
not clear earlier.
When you made the comment
{quote}
It's {{registeredConnections}} counter created in {{TransportContext}} that's
really counting the numbers and it's misleading for people who want to add new
metrics like {{registeredConnections}}.
{quote}
, you meant to use registeredConnections to mean something different than what
you reverted with this jira. Would you please explain
1. what's your interpretation of the reverted version?
2. what your definition is for "new metrics like {{registeredConnections}}."?
is it not counting? if it's counting, why the reverted implementation doesn't
serve the purpose?
BTW, my understanding is that 1, all executors running on a given host would
register with the remote shuffle service on the same host, 2, executors only
register with shuffle service on the same host where the executors are running
at, but not shuffle service running on other hosts. Is this understanding
correct?
Thanks.
> Remove unused registeredConnections counter from ShuffleMetrics
> ---------------------------------------------------------------
>
> Key: SPARK-31646
> URL: https://issues.apache.org/jira/browse/SPARK-31646
> Project: Spark
> Issue Type: Improvement
> Components: Deploy, Shuffle, Spark Core
> Affects Versions: 3.0.0
> Reporter: Manu Zhang
> Assignee: Manu Zhang
> Priority: Minor
> Fix For: 3.0.0
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
