[ https://issues.apache.org/jira/browse/SPARK-31646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17416225#comment-17416225 ]
Yongjun Zhang edited comment on SPARK-31646 at 9/16/21, 4:36 PM: ----------------------------------------------------------------- Thanks a lot for your quick response [~mauzhang]. Sorry my question was a bit not clear earlier. When you made the comment {quote} It's {{registeredConnections}} counter created in {{TransportContext}} that's really counting the numbers and it's misleading for people who want to add new metrics like {{registeredConnections}}. {quote} , you meant to use registeredConnections to mean something different than what you reverted with this jira. Would you please explain # what's your interpretation of the reverted version of registeredConnections besides "counting numbers"? is it not useful at all? # what your definition is for "new metrics like {{registeredConnections}}."? is it not counting? if it's counting, why the reverted implementation doesn't serve the purpose? BTW, my understanding is that 1, all executors running on a given host would register with the remote shuffle service on the same host, 2, executors only register with shuffle service on the same host where the executors are running at, but not shuffle service running on other hosts. Is this understanding correct? Thanks. was (Author: yzhangal): Thanks a lot for your quick response [~mauzhang]. Sorry my question was a bit not clear earlier. When you made the comment {quote} It's {{registeredConnections}} counter created in {{TransportContext}} that's really counting the numbers and it's misleading for people who want to add new metrics like {{registeredConnections}}. {quote} , you meant to use registeredConnections to mean something different than what you reverted with this jira. Would you please explain 1. what's your interpretation of the reverted version of registeredConnections besides "counting numbers"? is it not useful at all? 2. what your definition is for "new metrics like {{registeredConnections}}."? is it not counting? if it's counting, why the reverted implementation doesn't serve the purpose? BTW, my understanding is that 1, all executors running on a given host would register with the remote shuffle service on the same host, 2, executors only register with shuffle service on the same host where the executors are running at, but not shuffle service running on other hosts. Is this understanding correct? Thanks. > Remove unused registeredConnections counter from ShuffleMetrics > --------------------------------------------------------------- > > Key: SPARK-31646 > URL: https://issues.apache.org/jira/browse/SPARK-31646 > Project: Spark > Issue Type: Improvement > Components: Deploy, Shuffle, Spark Core > Affects Versions: 3.0.0 > Reporter: Manu Zhang > Assignee: Manu Zhang > Priority: Minor > Fix For: 3.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org