[GitHub] [spark] mridulm commented on a diff in pull request #37638: [SPARK-33573][SHUFFLE][YARN] Shuffle server side metrics for Push-based shuffle

2023-01-11 Thread GitBox
mridulm commented on code in PR #37638: URL: https://github.com/apache/spark/pull/37638#discussion_r1067641509 ## docs/monitoring.md: ## @@ -1421,6 +1421,21 @@ Note: applies to the shuffle service - shuffle-server.usedDirectMemory - shuffle-server.usedHeapMemory +Note:

[GitHub] [spark] mridulm commented on a diff in pull request #37638: [SPARK-33573][SHUFFLE][YARN] Shuffle server side metrics for Push-based shuffle

2023-01-11 Thread GitBox
mridulm commented on code in PR #37638: URL: https://github.com/apache/spark/pull/37638#discussion_r1067505986 ## docs/monitoring.md: ## @@ -1421,6 +1421,21 @@ Note: applies to the shuffle service - shuffle-server.usedDirectMemory - shuffle-server.usedHeapMemory +Note:

[GitHub] [spark] mridulm commented on a diff in pull request #37638: [SPARK-33573][SHUFFLE][YARN] Shuffle server side metrics for Push-based shuffle

2023-01-05 Thread GitBox
mridulm commented on code in PR #37638: URL: https://github.com/apache/spark/pull/37638#discussion_r1063072468 ## common/network-shuffle/src/test/java/org/apache/spark/network/shuffle/RemoteBlockPushResolverSuite.java: ## @@ -257,6 +274,7 @@ public void

[GitHub] [spark] mridulm commented on a diff in pull request #37638: [SPARK-33573][SHUFFLE][YARN] Shuffle server side metrics for Push-based shuffle

2023-01-05 Thread GitBox
mridulm commented on code in PR #37638: URL: https://github.com/apache/spark/pull/37638#discussion_r1063072468 ## common/network-shuffle/src/test/java/org/apache/spark/network/shuffle/RemoteBlockPushResolverSuite.java: ## @@ -257,6 +274,7 @@ public void

[GitHub] [spark] mridulm commented on a diff in pull request #37638: [SPARK-33573][SHUFFLE][YARN] Shuffle server side metrics for Push-based shuffle

2022-12-27 Thread GitBox
mridulm commented on code in PR #37638: URL: https://github.com/apache/spark/pull/37638#discussion_r1057887707 ## common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java: ## @@ -1904,4 +1951,52 @@ long getPos() { return pos;

[GitHub] [spark] mridulm commented on a diff in pull request #37638: [SPARK-33573][SHUFFLE][YARN] Shuffle server side metrics for Push-based shuffle

2022-12-27 Thread GitBox
mridulm commented on code in PR #37638: URL: https://github.com/apache/spark/pull/37638#discussion_r1057886779 ## common/network-shuffle/src/test/java/org/apache/spark/network/shuffle/RemoteBlockPushResolverSuite.java: ## @@ -235,6 +251,7 @@ public void

[GitHub] [spark] mridulm commented on a diff in pull request #37638: [SPARK-33573][SHUFFLE][YARN] Shuffle server side metrics for Push-based shuffle

2022-12-27 Thread GitBox
mridulm commented on code in PR #37638: URL: https://github.com/apache/spark/pull/37638#discussion_r1057885497 ## common/network-shuffle/src/test/java/org/apache/spark/network/shuffle/RemoteBlockPushResolverSuite.java: ## @@ -257,6 +274,7 @@ public void

[GitHub] [spark] mridulm commented on a diff in pull request #37638: [SPARK-33573][SHUFFLE][YARN] Shuffle server side metrics for Push-based shuffle

2022-12-27 Thread GitBox
mridulm commented on code in PR #37638: URL: https://github.com/apache/spark/pull/37638#discussion_r1057885497 ## common/network-shuffle/src/test/java/org/apache/spark/network/shuffle/RemoteBlockPushResolverSuite.java: ## @@ -257,6 +274,7 @@ public void

[GitHub] [spark] mridulm commented on a diff in pull request #37638: [SPARK-33573][SHUFFLE][YARN] Shuffle server side metrics for Push-based shuffle

2022-12-27 Thread GitBox
mridulm commented on code in PR #37638: URL: https://github.com/apache/spark/pull/37638#discussion_r1057878241 ## common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java: ## @@ -1220,6 +1260,7 @@ public void onData(String streamId,

[GitHub] [spark] mridulm commented on a diff in pull request #37638: [SPARK-33573][SHUFFLE][YARN] Shuffle server side metrics for Push-based shuffle

2022-12-15 Thread GitBox
mridulm commented on code in PR #37638: URL: https://github.com/apache/spark/pull/37638#discussion_r1050430333 ## common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java: ## @@ -1197,15 +1230,15 @@ public void onData(String streamId,

[GitHub] [spark] mridulm commented on a diff in pull request #37638: [SPARK-33573][SHUFFLE][YARN] Shuffle server side metrics for Push-based shuffle

2022-12-15 Thread GitBox
mridulm commented on code in PR #37638: URL: https://github.com/apache/spark/pull/37638#discussion_r1050427667 ## common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java: ## @@ -593,6 +607,9 @@ public void onData(String streamId,

[GitHub] [spark] mridulm commented on a diff in pull request #37638: [SPARK-33573][SHUFFLE][YARN] Shuffle server side metrics for Push-based shuffle

2022-12-03 Thread GitBox
mridulm commented on code in PR #37638: URL: https://github.com/apache/spark/pull/37638#discussion_r1038914958 ## common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java: ## @@ -1197,15 +1230,15 @@ public void onData(String streamId,

[GitHub] [spark] mridulm commented on a diff in pull request #37638: [SPARK-33573][SHUFFLE][YARN] Shuffle server side metrics for Push-based shuffle

2022-12-03 Thread GitBox
mridulm commented on code in PR #37638: URL: https://github.com/apache/spark/pull/37638#discussion_r1038914901 ## common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java: ## @@ -1197,15 +1230,15 @@ public void onData(String streamId,

[GitHub] [spark] mridulm commented on a diff in pull request #37638: [SPARK-33573][SHUFFLE][YARN] Shuffle server side metrics for Push-based shuffle

2022-12-03 Thread GitBox
mridulm commented on code in PR #37638: URL: https://github.com/apache/spark/pull/37638#discussion_r1038914754 ## common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java: ## @@ -1904,4 +1941,42 @@ long getPos() { return pos;

[GitHub] [spark] mridulm commented on a diff in pull request #37638: [SPARK-33573][SHUFFLE][YARN] Shuffle server side metrics for Push-based shuffle

2022-12-03 Thread GitBox
mridulm commented on code in PR #37638: URL: https://github.com/apache/spark/pull/37638#discussion_r1038914754 ## common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java: ## @@ -1904,4 +1941,42 @@ long getPos() { return pos;

[GitHub] [spark] mridulm commented on a diff in pull request #37638: [SPARK-33573][SHUFFLE][YARN] Shuffle server side metrics for Push-based shuffle

2022-12-03 Thread GitBox
mridulm commented on code in PR #37638: URL: https://github.com/apache/spark/pull/37638#discussion_r1038914434 ## common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java: ## @@ -593,6 +607,9 @@ public void onData(String streamId,

[GitHub] [spark] mridulm commented on a diff in pull request #37638: [SPARK-33573][SHUFFLE][YARN] Shuffle server side metrics for Push-based shuffle

2022-10-25 Thread GitBox
mridulm commented on code in PR #37638: URL: https://github.com/apache/spark/pull/37638#discussion_r1003656427 ## common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java: ## @@ -593,6 +607,9 @@ public void onData(String streamId,

[GitHub] [spark] mridulm commented on a diff in pull request #37638: [SPARK-33573][SHUFFLE][YARN] Shuffle server side metrics for Push-based shuffle

2022-10-24 Thread GitBox
mridulm commented on code in PR #37638: URL: https://github.com/apache/spark/pull/37638#discussion_r1003658008 ## common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java: ## @@ -1904,4 +1941,42 @@ long getPos() { return pos;

[GitHub] [spark] mridulm commented on a diff in pull request #37638: [SPARK-33573][SHUFFLE][YARN] Shuffle server side metrics for Push-based shuffle

2022-10-24 Thread GitBox
mridulm commented on code in PR #37638: URL: https://github.com/apache/spark/pull/37638#discussion_r1003656427 ## common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java: ## @@ -593,6 +607,9 @@ public void onData(String streamId,

[GitHub] [spark] mridulm commented on a diff in pull request #37638: [SPARK-33573][SHUFFLE][YARN] Shuffle server side metrics for Push-based shuffle

2022-10-24 Thread GitBox
mridulm commented on code in PR #37638: URL: https://github.com/apache/spark/pull/37638#discussion_r1003638095 ## common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java: ## @@ -1197,15 +1230,15 @@ public void onData(String streamId,

[GitHub] [spark] mridulm commented on a diff in pull request #37638: [SPARK-33573][SHUFFLE][YARN] Shuffle server side metrics for Push-based shuffle

2022-10-24 Thread GitBox
mridulm commented on code in PR #37638: URL: https://github.com/apache/spark/pull/37638#discussion_r1003635126 ## common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java: ## @@ -1197,15 +1230,15 @@ public void onData(String streamId,

[GitHub] [spark] mridulm commented on a diff in pull request #37638: [SPARK-33573][SHUFFLE][YARN] Shuffle server side metrics for Push-based shuffle

2022-10-24 Thread GitBox
mridulm commented on code in PR #37638: URL: https://github.com/apache/spark/pull/37638#discussion_r1003564958 ## common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java: ## @@ -1904,4 +1941,42 @@ long getPos() { return pos;

[GitHub] [spark] mridulm commented on a diff in pull request #37638: [SPARK-33573][SHUFFLE][YARN] Shuffle server side metrics for Push-based shuffle

2022-10-24 Thread GitBox
mridulm commented on code in PR #37638: URL: https://github.com/apache/spark/pull/37638#discussion_r1003562799 ## common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java: ## @@ -593,6 +607,9 @@ public void onData(String streamId,

[GitHub] [spark] mridulm commented on a diff in pull request #37638: [SPARK-33573][SHUFFLE][YARN] Shuffle server side metrics for Push-based shuffle

2022-09-27 Thread GitBox
mridulm commented on code in PR #37638: URL: https://github.com/apache/spark/pull/37638#discussion_r981843162 ## common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java: ## @@ -593,6 +607,9 @@ public void onData(String streamId,

[GitHub] [spark] mridulm commented on a diff in pull request #37638: [SPARK-33573][SHUFFLE][YARN] Shuffle server side metrics for Push-based shuffle

2022-08-30 Thread GitBox
mridulm commented on code in PR #37638: URL: https://github.com/apache/spark/pull/37638#discussion_r958962174 ## common/network-yarn/src/main/java/org/apache/spark/network/yarn/YarnShuffleService.java: ## @@ -296,10 +301,15 @@ protected void serviceInit(Configuration

[GitHub] [spark] mridulm commented on a diff in pull request #37638: [SPARK-33573][SHUFFLE][YARN] Shuffle server side metrics for Push-based shuffle

2022-08-30 Thread GitBox
mridulm commented on code in PR #37638: URL: https://github.com/apache/spark/pull/37638#discussion_r958962174 ## common/network-yarn/src/main/java/org/apache/spark/network/yarn/YarnShuffleService.java: ## @@ -296,10 +301,15 @@ protected void serviceInit(Configuration