[
https://issues.apache.org/jira/browse/HADOOP-10048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15339794#comment-15339794
]
Jason Lowe commented on HADOOP-10048:
-------------------------------------
Technically this is a performance optimization improvement rather than a bug
fix. Typically we wouldn't backport since those other releases are in
maintenance and should only receive fixes rather than features/improvements to
reduce the risk of destabilizing those releases. If we feel this is an
important enough performance improvement and the rewards outweigh the risks
then I'm OK with it. It doesn't pick cleanly, but it would if we also
backported HADOOP-8436, HADOOP-8437, and HADOOP-12252 which are all bug fixes.
> LocalDirAllocator should avoid holding locks while accessing the filesystem
> ---------------------------------------------------------------------------
>
> Key: HADOOP-10048
> URL: https://issues.apache.org/jira/browse/HADOOP-10048
> Project: Hadoop Common
> Issue Type: Improvement
> Affects Versions: 2.3.0
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Fix For: 2.8.0
>
> Attachments: HADOOP-10048.003.patch, HADOOP-10048.004.patch,
> HADOOP-10048.005.patch, HADOOP-10048.006.patch, HADOOP-10048.patch,
> HADOOP-10048.trunk.patch
>
>
> As noted in MAPREDUCE-5584 and HADOOP-7016, LocalDirAllocator can be a
> bottleneck for multithreaded setups like the ShuffleHandler. We should
> consider moving to a lockless design or minimizing the critical sections to a
> very small amount of time that does not involve I/O operations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
