This is an automated email from the ASF dual-hosted git repository.
SteNicholas pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/celeborn.git
The following commit(s) were added to refs/heads/main by this push:
new 8d473c5af [CELEBORN-2318] Miss increment to
WRITE_DATA_HARD_SPLIT_COUNT on returning HARD_SPLIT in handlePushData
8d473c5af is described below
commit 8d473c5af5a754a5b0329a805298ce9e8f0d27e7
Author: Kartikay Bhutani <[email protected]>
AuthorDate: Fri May 8 15:42:51 2026 +0800
[CELEBORN-2318] Miss increment to WRITE_DATA_HARD_SPLIT_COUNT on returning
HARD_SPLIT in handlePushData
### What changes were proposed in this pull request?
Missing increment to `WRITE_DATA_HARD_SPLIT_COUNT` on returning HARD_SPLIT
### Why are the changes needed?
- The post-restart detection branch in `handlePushData` (Case2: shuffleKey
in storageManager but not in shuffleMapperAttempts) returns HARD_SPLIT without
incrementing `WRITE_DATA_HARD_SPLIT_COUNT`
- The sibling Case1 branch (line 398) and all other HARD_SPLIT return paths
already increment it
- This makes Case2 invisible to monitoring during rolling restarts
### Does this PR resolve a correctness bug?
No
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Existing UTs
Closes #3676 from kaybhutani/kartikay/missing-hard-split-metric.
Authored-by: Kartikay Bhutani <[email protected]>
Signed-off-by: SteNicholas <[email protected]>
---
.../org/apache/celeborn/service/deploy/worker/PushDataHandler.scala | 1 +
1 file changed, 1 insertion(+)
diff --git
a/worker/src/main/scala/org/apache/celeborn/service/deploy/worker/PushDataHandler.scala
b/worker/src/main/scala/org/apache/celeborn/service/deploy/worker/PushDataHandler.scala
index 50f085bf8..cc1dc257f 100644
---
a/worker/src/main/scala/org/apache/celeborn/service/deploy/worker/PushDataHandler.scala
+++
b/worker/src/main/scala/org/apache/celeborn/service/deploy/worker/PushDataHandler.scala
@@ -217,6 +217,7 @@ class PushDataHandler(val workerSource: WorkerSource)
extends BaseMessageHandler
// after worker restart, some tasks still push data to this
HARD_SPLIT partition.
logDebug(s"[Case2] Receive push data for committed hard split
partition of " +
s"(shuffle $shuffleKey, map $mapId attempt $attemptId)")
+ workerSource.incCounter(WorkerSource.WRITE_DATA_HARD_SPLIT_COUNT)
callbackWithTimer.onSuccess(ByteBuffer.wrap(Array[Byte](StatusCode.HARD_SPLIT.getValue)))
} else {
logWarning(s"While handle PushData, Partition location wasn't found
for " +
