[
https://issues.apache.org/jira/browse/MAPREDUCE-1981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12973539#action_12973539
]
Min Zhou commented on MAPREDUCE-1981:
-------------------------------------
Lines listed below will caused a NullPointerException because EMPTY_BLOCK_LOCS
will return null when calling blocks.getLocatedBlocks()
{noformat}
/** a default LocatedBlocks object, its content should not be changed */
private final static LocatedBlocks EMPTY_BLOCK_LOCS = new LocatedBlocks();
{noformat}
here is an example of this exception
{noformat}
java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.hdfs.DFSUtil.locatedBlocks2Locations(DFSUtil.java:84)
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.getListing(FSDirectory.java:731)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getListing(FSNamesystem.java:2015)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.getLocatedListing(NameNode.java:494)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:481)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:894)
{noformat}
> Improve getSplits performance by using listFiles, the new FileSystem API
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-1981
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1981
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: job submission
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.22.0
>
> Attachments: mapredListFiles.patch, mapredListFiles1.patch,
> mapredListFiles2.patch, mapredListFiles3.patch, mapredListFiles4.patch,
> mapredListFiles5.patch
>
>
> This jira will make FileInputFormat and CombinedFileInputForm to use the new
> API, thus reducing the number of RPCs to HDFS NameNode.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
