This is an automated email from the ASF dual-hosted git repository.
rexxiong pushed a commit to branch branch-0.5
in repository https://gitbox.apache.org/repos/asf/celeborn.git
The following commit(s) were added to refs/heads/branch-0.5 by this push:
new 0607bcae5 [CELEBORN-1765] Fix NPE when removeFileInfo in StorageManager
0607bcae5 is described below
commit 0607bcae5da93184e4e911f6bac170b557393807
Author: zhengtao <[email protected]>
AuthorDate: Tue Dec 10 14:08:54 2024 +0800
[CELEBORN-1765] Fix NPE when removeFileInfo in StorageManager
### What changes were proposed in this pull request?
As title.
### Why are the changes needed?
The file may not have been committed when removeFileInfo in
gracefulshutdown condition.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Existing ut.
Closes #2982 from zaynt4606/clb1765.
Authored-by: zhengtao <[email protected]>
Signed-off-by: Shuang <[email protected]>
(cherry picked from commit 22ee8bfed5b3a20922768a7d9e9701e371143433)
Signed-off-by: Shuang <[email protected]>
---
.../celeborn/service/deploy/worker/storage/StorageManager.scala | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git
a/worker/src/main/scala/org/apache/celeborn/service/deploy/worker/storage/StorageManager.scala
b/worker/src/main/scala/org/apache/celeborn/service/deploy/worker/storage/StorageManager.scala
index 851458dfa..c189f93bd 100644
---
a/worker/src/main/scala/org/apache/celeborn/service/deploy/worker/storage/StorageManager.scala
+++
b/worker/src/main/scala/org/apache/celeborn/service/deploy/worker/storage/StorageManager.scala
@@ -483,7 +483,9 @@ final private[worker] class StorageManager(conf:
CelebornConf, workerSource: Abs
if (diskShuffleMap != null) {
if (workerGracefulShutdown) {
val committedFileInfoMap = committedFileInfos.get(shuffleKey)
- committedFileInfoMap.remove(fileName)
+ if (committedFileInfoMap != null) {
+ committedFileInfoMap.remove(fileName)
+ }
}
return diskShuffleMap.remove(fileName)
}
