This is an automated email from the ASF dual-hosted git repository.
zhouky pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-celeborn.git
The following commit(s) were added to refs/heads/main by this push:
new a7ff28c26 [CELEBORN-779] Fix sorted file size summary overflow
a7ff28c26 is described below
commit a7ff28c26d9701ae3958513bdcc2b42df1a4c0af
Author: mingji <[email protected]>
AuthorDate: Mon Jul 10 11:29:47 2023 +0800
[CELEBORN-779] Fix sorted file size summary overflow
### What changes were proposed in this pull request?
To fix sorted file size summary counter overflow. Sorted file size summary
can be larger than the integer's max value. It should be applied to branch-0.3,
branch-0.2, and main branch.
### Why are the changes needed?
This graph is incorrect.
<img width="1500"
alt="lQLPJx4a6A76xo7NBOHNC7iwLBx50tXnv5IEoQTQxMAMAA_3000_1249"
src="https://github.com/apache/incubator-celeborn/assets/4150993/8d288087-64d7-4569-ba04-05bb1915118a";>
### Does this PR introduce _any_ user-facing change?
It will cause user-facing changes. Celeborn worker's metric for sorted
file summary is incorrect and this patch will correct it.
### How was this patch tested?
UT.
Closes #1694 from FMX/CELEBORN-779.
Authored-by: mingji <[email protected]>
Signed-off-by: zky.zhoukeyong <[email protected]>
---
.../celeborn/service/deploy/worker/storage/PartitionFilesSorter.java | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git
a/worker/src/main/java/org/apache/celeborn/service/deploy/worker/storage/PartitionFilesSorter.java
b/worker/src/main/java/org/apache/celeborn/service/deploy/worker/storage/PartitionFilesSorter.java
index 1d88069a8..016290513 100644
---
a/worker/src/main/java/org/apache/celeborn/service/deploy/worker/storage/PartitionFilesSorter.java
+++
b/worker/src/main/java/org/apache/celeborn/service/deploy/worker/storage/PartitionFilesSorter.java
@@ -154,8 +154,8 @@ public class PartitionFilesSorter extends
ShuffleRecoverHelper {
return sortedFileCount.get();
}
- public int getSortedSize() {
- return (int) sortedFilesSize.get();
+ public long getSortedSize() {
+ return sortedFilesSize.get();
}
public FileInfo getSortedFileInfo(
