[
https://issues.apache.org/jira/browse/MAPREDUCE-3519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13166218#comment-13166218
]
Robert Joseph Evans commented on MAPREDUCE-3519:
------------------------------------------------
I like the patch. I also traced down from where the synchronized call is
happening looking for a potential race condition between reads and writes. The
only place I found that uses conf from
LocalDirsHandlerService.getLocalPathForWrite is LocalDirAllocator.confChanged.
Inside that code we access conf three times, once to get the local file system,
so I don't see that as being an issue. But we call it twice to get the value
of the contextConfigItemName. The first to see if the value has changed, and
the second to turn that value into an array. It would be better to change the
code so that it only calls conf.get once for that value, and then to turn it
into a trimmed array of Strings call StringUtils.getTrimmedStrings, which is
what the second method does internally.
> Deadlock in LocalDirsHandlerService and ShuffleHandler
> ------------------------------------------------------
>
> Key: MAPREDUCE-3519
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3519
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, nodemanager
> Affects Versions: 0.23.1, 0.24.0
> Reporter: Ravi Gummadi
> Assignee: Ravi Gummadi
> Fix For: 0.23.1
>
> Attachments: 3519.patch, 3519.v1.patch, deadlock.txt
>
>
> MAPREDUCE-3121 cloned Configuration object in LocalDirsHandlerService.init()
> to avoid others to access that configuration object. But since it is used in
> local FileSystem object creation in LocalDirAllocator.AllocatorPerContext and
> the same FileSystem object is used in
> ShuffleHandler.Shuffle.localDirAllocator, this is causing a deadlock when
> accessing this configuration object from LocalDirsHandlerService and
> ShuffleHandler along with AllocatorPerContext object.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
