[
https://issues.apache.org/jira/browse/MAPREDUCE-1981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12973817#action_12973817
]
Hairong Kuang commented on MAPREDUCE-1981:
------------------------------------------
Min, thanks for trying this patch.
For the first bug, my patch does not transfer LocatedFileStatus over the wire.
Did you modify my patch to use LocatedFileStatus instead?
The second is truely a bug. I will get it fixed.
BTW, I have a slightly different implementation of getListing for our internal
branch, which can avoid type casting, and therefore provides better
performance. I will share this with the community hopefully soon.
> Improve getSplits performance by using listFiles, the new FileSystem API
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-1981
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1981
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: job submission
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.22.0
>
> Attachments: mapredListFiles.patch, mapredListFiles1.patch,
> mapredListFiles2.patch, mapredListFiles3.patch, mapredListFiles4.patch,
> mapredListFiles5.patch
>
>
> This jira will make FileInputFormat and CombinedFileInputForm to use the new
> API, thus reducing the number of RPCs to HDFS NameNode.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
