dongjoon-hyun commented on a change in pull request #32388:
URL: https://github.com/apache/spark/pull/32388#discussion_r628799385
##########
File path:
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ExternalBlockHandler.java
##########
@@ -291,9 +294,18 @@ public ShuffleMetrics() {
allMetrics.put("openBlockRequestLatencyMillis",
openBlockRequestLatencyMillis);
allMetrics.put("registerExecutorRequestLatencyMillis",
registerExecutorRequestLatencyMillis);
allMetrics.put("finalizeShuffleMergeLatencyMillis",
finalizeShuffleMergeLatencyMillis);
+ allMetrics.put("blockTransferRate", blockTransferRate);
allMetrics.put("blockTransferRateBytes", blockTransferRateBytes);
+ allMetrics.put("blockTransferAvgSize_1min", new RatioGauge() {
+ @Override
+ protected Ratio getRatio() {
+ return Ratio.of(
+ blockTransferRateBytes.getOneMinuteRate(),
+ blockTransferRate.getOneMinuteRate());
+ }
+ });
allMetrics.put("registeredExecutorsSize",
- (Gauge<Integer>) () ->
blockManager.getRegisteredExecutorsSize());
+ (Gauge<Integer>) blockManager::getRegisteredExecutorsSize);
Review comment:
Is there a reason why we touch another `registeredExecutorsSize` metric
at a PR claiming to add new different metrics? Apparently, this looks like
orthogonal to me and the PR description doesn't mention this at all either. If
you need this for a different reason, shall we make a separate PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
