This is an automated email from the ASF dual-hosted git repository. xianjingfeng pushed a commit to branch branch-0.8 in repository https://gitbox.apache.org/repos/asf/incubator-uniffle.git
commit 2320e394c5987a684ad15f87a6fd63681043f183 Author: Zhen Wang <[email protected]> AuthorDate: Wed Dec 13 14:09:53 2023 +0800 [MINOR] fix(spark): Fix NPE for ShuffleWriteClientImpl.unregisterShuffle (#1367) ### What changes were proposed in this pull request? Fix NPE for ShuffleWriteClientImpl.unregisterShuffle. ### Why are the changes needed? Since the `id` of `RssShuffleManager` in Executor is assigned in `getWriter`, this means that the `id` may be null before `getWriter`, so this may cause NPE to occur in `ShuffleWriteClientImpl.unregisterShuffle`. error log: ``` 23/12/12 14:25:47 ERROR RssShuffleManager: Errors on closing shuffle write client java.lang.NullPointerException at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936) at org.apache.uniffle.client.impl.ShuffleWriteClientImpl.unregisterShuffle(ShuffleWriteClientImpl.java:751) at org.apache.spark.shuffle.RssShuffleManager.stop(RssShuffleManager.java:591) at org.apache.spark.SparkEnv.stop(SparkEnv.scala:90) at org.apache.spark.executor.Executor.stop(Executor.scala:335) at org.apache.spark.executor.CoarseGrainedExecutorBackend$$anonfun$receive$1$$anon$1.run(CoarseGrainedExecutorBackend.scala:203) ``` ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? (cherry picked from commit f17e060bc2ecf9ad7b8f17562f4cad6ffe66321f) --- .../java/org/apache/uniffle/client/impl/ShuffleWriteClientImpl.java | 3 +++ 1 file changed, 3 insertions(+) diff --git a/client/src/main/java/org/apache/uniffle/client/impl/ShuffleWriteClientImpl.java b/client/src/main/java/org/apache/uniffle/client/impl/ShuffleWriteClientImpl.java index d261b35b5..d90669106 100644 --- a/client/src/main/java/org/apache/uniffle/client/impl/ShuffleWriteClientImpl.java +++ b/client/src/main/java/org/apache/uniffle/client/impl/ShuffleWriteClientImpl.java @@ -997,6 +997,9 @@ public class ShuffleWriteClientImpl implements ShuffleWriteClient { @Override public void unregisterShuffle(String appId) { + if (appId == null) { + return; + } Map<Integer, Set<ShuffleServerInfo>> appServerMap = shuffleServerInfoMap.get(appId); if (appServerMap == null) { return;
