[
https://issues.apache.org/jira/browse/HADOOP-7016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15894854#comment-15894854
]
Hadoop QA commented on HADOOP-7016:
-----------------------------------
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s{color}
| {color:red} HADOOP-7016 does not apply to trunk. Rebase required? Wrong
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HADOOP-7016 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12459964/HADOOP-7016-v3.Yahoo.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/11758/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Optimize locking codepath in LocalDirAllocator.getLocalPathToRead() and
> reduce creating many Path objects
> ---------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-7016
> URL: https://issues.apache.org/jira/browse/HADOOP-7016
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
> Affects Versions: 0.20.2
> Reporter: Rajesh Balamohan
> Assignee: Rajesh Balamohan
> Priority: Minor
> Attachments: HADOOP-7016-v2.Yahoo.patch, HADOOP-7016-v3.Yahoo.patch,
> HADOOP-7016-Yahoo.patch
>
>
> LocalDirAllocator.getLocalPathToRead() is called from TaskTracker to retrieve
> temporary map output location. This method is synchronized and amount of time
> spent on this method directly affects the performance of TaskTracker's
> ability to serve faster. This JIRA is created to reduce the additional fat
> in getLocalPathToRead() and reduce creating the number of Path obejcts which
> are expensive.
> 1. getLocalPathToRead() internally calls RawLocalFileSystem.exists() in order
> to check the existence of a local file. RawLocalFileSystem.exists()
> internally creates a FileStatus object which needs to be populated with
> lastModified, length, directory details etc. This might not be necessary for
> just checking file existence.
> 2. Creating a Path object out of a string requires fair amount of processing.
> It might be worth to store "localDirsPath" in LocalDirAllocator instead of
> storing "localDirs". This would avoid the number of times Path() object is
> created in LocalDirAllocator.
> Any other codepath using LocalDirAllocator would also benefit from the above
> 2 optimizations.
> The attached patch addresses the above 2 issues.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
