[
https://issues.apache.org/jira/browse/MAPREDUCE-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18043479#comment-18043479
]
Ryan Carter commented on MAPREDUCE-7507:
----------------------------------------
Thanks for spotting this subtle concurrency issue!
I agree that the interleaving can cause the exception, but since this is a hot
path, we can’t add synchronization for performance reasons.
Do you have any ideas for a lightweight or lock-free fix?
> Interleaving of double execution getTask() can cause a NullPointerException
> ----------------------------------------------------------------------------
>
> Key: MAPREDUCE-7507
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7507
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client
> Affects Versions: 3.4.1
> Reporter: John Doe
> Priority: Major
> Attachments: TestViolation.java
>
>
> If the method `getTask()` in the class `TaskAttemptListenerImpl` is executed
> twice, the interleaving can cause a concurrency bug (atomicity) and throw a
> null pointer exception.
> Suppose both executions reach the point before `
> jvmIDToActiveAttemptMap.remove()`, the first executed one will behave
> normally, but for the latter one, its `task` will be null and then get a null
> pointer exception.
> {code:java}
> if (!jvmIDToActiveAttemptMap.containsKey(wJvmID)) {
> LOG.info("JVM with ID: " + jvmId + " is invalid and will be killed.");
> jvmTask = TASK_FOR_INVALID_JVM;
> } else {
> if (!launchedJVMs.contains(wJvmID)) {
> jvmTask = null;
> LOG.info("JVM with ID: " + jvmId
> + " asking for task before AM launch registered. Given null
> task");
> } else {
> // remove the task as it is no more needed and free up the memory.
> // Also we have already told the JVM to process a task, so it is no
> // longer pending, and further request should ask it to exit.
> org.apache.hadoop.mapred.Task task =
> jvmIDToActiveAttemptMap.remove(wJvmID);
> launchedJVMs.remove(wJvmID);
> LOG.info("JVM with ID: " + jvmId + " given task: " +
> task.getTaskID());
> task.setEncryptedSpillKey(encryptedSpillKey);
> jvmTask = new JvmTask(task, false);
> }
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
