Repository: samza
Updated Branches:
refs/heads/master 51729ac68 -> 2e461a880
SAMZA-1699: Fix NPE in ClusterResourceManager
When the ClusterResourcedManager receives a notification that a container is
started, it moves the container from the "pending queue" to its "running queue".
In the meanwhile, it's possible for another thread to remove the mapping for
the key. Here's an example:
NMCallbackThread-1:```
pendingYarnContainers.remove(key);```
NMCallbackThread-2:
```
for (String key : pendingYarnContainers.keySet()) {
yarnContainer = pendingYarnContainers.get(key); <-- could be null depending
on whether the removal happened before it.
}```
Author: Jagadish <[email protected]>
Reviewers: Prateek M<[email protected]>
Closes #504 from vjagadish/npe-fix-async
Project: http://git-wip-us.apache.org/repos/asf/samza/repo
Commit: http://git-wip-us.apache.org/repos/asf/samza/commit/2e461a88
Tree: http://git-wip-us.apache.org/repos/asf/samza/tree/2e461a88
Diff: http://git-wip-us.apache.org/repos/asf/samza/diff/2e461a88
Branch: refs/heads/master
Commit: 2e461a8804ec568ff52de2d6e81a8edd4865ce12
Parents: 51729ac
Author: Jagadish <[email protected]>
Authored: Fri May 4 14:02:35 2018 -0700
Committer: Jagadish <[email protected]>
Committed: Fri May 4 14:02:35 2018 -0700
----------------------------------------------------------------------
.../java/org/apache/samza/job/yarn/YarnClusterResourceManager.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/samza/blob/2e461a88/samza-yarn/src/main/java/org/apache/samza/job/yarn/YarnClusterResourceManager.java
----------------------------------------------------------------------
diff --git
a/samza-yarn/src/main/java/org/apache/samza/job/yarn/YarnClusterResourceManager.java
b/samza-yarn/src/main/java/org/apache/samza/job/yarn/YarnClusterResourceManager.java
index 407768c..79a9083 100644
---
a/samza-yarn/src/main/java/org/apache/samza/job/yarn/YarnClusterResourceManager.java
+++
b/samza-yarn/src/main/java/org/apache/samza/job/yarn/YarnClusterResourceManager.java
@@ -712,7 +712,7 @@ public class YarnClusterResourceManager extends
ClusterResourceManager implement
private String getPendingSamzaContainerId(ContainerId containerId) {
for (String samzaContainerId: state.pendingYarnContainers.keySet()) {
YarnContainer yarnContainer =
state.pendingYarnContainers.get(samzaContainerId);
- if (yarnContainer.id().equals(containerId)) {
+ if (yarnContainer != null && yarnContainer.id().equals(containerId)) {
return samzaContainerId;
}
}
