This is an automated email from the ASF dual-hosted git repository.
ethanfeng pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/celeborn.git
The following commit(s) were added to refs/heads/main by this push:
new 062db5b38 [CELEBORN-1921][FOLLOWUP] Log the
GetReducerFileGroupResponse size to provide insights
062db5b38 is described below
commit 062db5b38b631043a64a51e4f001b5f31ffdd70b
Author: Wang, Fei <[email protected]>
AuthorDate: Fri May 16 14:42:30 2025 +0800
[CELEBORN-1921][FOLLOWUP] Log the GetReducerFileGroupResponse size to
provide insights
### What changes were proposed in this pull request?
Always log the size of GetReducerFileGroupResponse, which is the most
heaviest RPC in client end.
### Why are the changes needed?
To provide more insights if meet rpc timeout for
GetReducerFileGroupResponse.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
GA.
Closes #3262 from turboFei/broadcast_size.
Authored-by: Wang, Fei <[email protected]>
Signed-off-by: mingji <[email protected]>
---
.../apache/celeborn/client/commit/ReducePartitionCommitHandler.scala | 2 ++
1 file changed, 2 insertions(+)
diff --git
a/client/src/main/scala/org/apache/celeborn/client/commit/ReducePartitionCommitHandler.scala
b/client/src/main/scala/org/apache/celeborn/client/commit/ReducePartitionCommitHandler.scala
index 5bdd1c550..a65f7e1eb 100644
---
a/client/src/main/scala/org/apache/celeborn/client/commit/ReducePartitionCommitHandler.scala
+++
b/client/src/main/scala/org/apache/celeborn/client/commit/ReducePartitionCommitHandler.scala
@@ -366,6 +366,8 @@ class ReducePartitionCommitHandler(
val serializedMsg =
context.asInstanceOf[RemoteNettyRpcCallContext].nettyEnv.serialize(returnedMsg)
+ logInfo(
+ s"Shuffle $shuffleId GetReducerFileGroupResponse size " +
serializedMsg.capacity())
if (getReducerFileGroupResponseBroadcastEnabled &&
serializedMsg.capacity() >=
getReducerFileGroupResponseBroadcastMiniSize) {
