maobaolong commented on code in PR #1758:
URL:
https://github.com/apache/incubator-uniffle/pull/1758#discussion_r1704806720
##########
server/src/main/java/org/apache/uniffle/server/ShuffleServerGrpcService.java:
##########
@@ -506,14 +506,23 @@ public void reportShuffleResult(
"appId[" + appId + "], shuffleId[" + shuffleId + "], taskAttemptId[" +
taskAttemptId + "]";
try {
+ int expectedBlockCount =
partitionToBlockIds.values().stream().mapToInt(x -> x.length).sum();
LOG.info(
- "Report "
- + partitionToBlockIds.size()
- + " blocks as shuffle result for the task of "
- + requestInfo);
- shuffleServer
- .getShuffleTaskManager()
- .addFinishedBlockIds(appId, shuffleId, partitionToBlockIds,
bitmapNum);
+ "Accepted blockIds report for {} blocks across {} partitions as
shuffle result for task {}",
+ expectedBlockCount,
+ partitionToBlockIds.size(),
+ request);
+ int updatedBlockCount =
+ shuffleServer
+ .getShuffleTaskManager()
+ .addFinishedBlockIds(appId, shuffleId, partitionToBlockIds,
bitmapNum);
+ if (expectedBlockCount != updatedBlockCount) {
+ LOG.warn(
+ "Existing {} duplicated blockIds on blockId report for appId: {},
shuffleId: {}",
Review Comment:
Unfortunately, we encountered this warn level log these days, it make us
stressed.
Is there indicate something wrong? Can you explain what rare condition
scenario can make updatedBlockCount less than expectedBlockCount?
Why there are duplicated blockIds on blockId report?
##########
server/src/main/java/org/apache/uniffle/server/ShuffleServerGrpcService.java:
##########
@@ -506,14 +506,23 @@ public void reportShuffleResult(
"appId[" + appId + "], shuffleId[" + shuffleId + "], taskAttemptId[" +
taskAttemptId + "]";
try {
+ int expectedBlockCount =
partitionToBlockIds.values().stream().mapToInt(x -> x.length).sum();
LOG.info(
- "Report "
- + partitionToBlockIds.size()
- + " blocks as shuffle result for the task of "
- + requestInfo);
- shuffleServer
- .getShuffleTaskManager()
- .addFinishedBlockIds(appId, shuffleId, partitionToBlockIds,
bitmapNum);
+ "Accepted blockIds report for {} blocks across {} partitions as
shuffle result for task {}",
Review Comment:
This log is verbose and fill server log, is there a better way to reduce it
or do you think `DEBUG` level can be better?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
