This is an automated email from the ASF dual-hosted git repository.
zhouky pushed a commit to branch branch-0.3
in repository https://gitbox.apache.org/repos/asf/incubator-celeborn.git
The following commit(s) were added to refs/heads/branch-0.3 by this push:
new 54a206d79 [CELEBORN-779] Fix sorted file size summary overflow
54a206d79 is described below
commit 54a206d79250b82f0e6cc01fb8dc45a1e14b11a2
Author: mingji <[email protected]>
AuthorDate: Mon Jul 10 11:29:47 2023 +0800
[CELEBORN-779] Fix sorted file size summary overflow
### What changes were proposed in this pull request?
To fix sorted file size summary counter overflow. Sorted file size summary
can be larger than the integer's max value. It should be applied to branch-0.3,
branch-0.2, and main branch.
### Why are the changes needed?
This graph is incorrect.
<img width="1500"
alt="lQLPJx4a6A76xo7NBOHNC7iwLBx50tXnv5IEoQTQxMAMAA_3000_1249"
src="https://github.com/apache/incubator-celeborn/assets/4150993/8d288087-64d7-4569-ba04-05bb1915118a";>
### Does this PR introduce _any_ user-facing change?
It will cause user-facing changes. Celeborn worker's metric for sorted
file summary is incorrect and this patch will correct it.
### How was this patch tested?
UT.
Closes #1694 from FMX/CELEBORN-779.
Authored-by: mingji <[email protected]>
Signed-off-by: zky.zhoukeyong <[email protected]>
(cherry picked from commit a7ff28c26d9701ae3958513bdcc2b42df1a4c0af)
Signed-off-by: zky.zhoukeyong <[email protected]>
---
.../celeborn/service/deploy/worker/storage/PartitionFilesSorter.java | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git
a/worker/src/main/java/org/apache/celeborn/service/deploy/worker/storage/PartitionFilesSorter.java
b/worker/src/main/java/org/apache/celeborn/service/deploy/worker/storage/PartitionFilesSorter.java
index 1d88069a8..016290513 100644
---
a/worker/src/main/java/org/apache/celeborn/service/deploy/worker/storage/PartitionFilesSorter.java
+++
b/worker/src/main/java/org/apache/celeborn/service/deploy/worker/storage/PartitionFilesSorter.java
@@ -154,8 +154,8 @@ public class PartitionFilesSorter extends
ShuffleRecoverHelper {
return sortedFileCount.get();
}
- public int getSortedSize() {
- return (int) sortedFilesSize.get();
+ public long getSortedSize() {
+ return sortedFilesSize.get();
}
public FileInfo getSortedFileInfo(
