NameNode should not return full path name when lisitng a diretory or getting
the status of a file
-------------------------------------------------------------------------------------------------
Key: HDFS-946
URL: https://issues.apache.org/jira/browse/HDFS-946
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Hairong Kuang
Fix For: 0.22.0
FSDirectory#getListring(String src) has the following code:
int i = 0;
for (INode cur : contents) {
listing[i] = createFileStatus(srcs+cur.getLocalName(), cur);
i++;
}
So listing a directory will return an array of FileStatus. Each FileStatus
element has the full path name. This increases the return message size and adds
non-negligible CPU time to the operation.
FSDirectory#getFileInfo(String) does not need to return the file name either.
Another optimization is that in the version of FileStatus that's used in the
wire protocol, the field path does not need to be Path; It could be a String or
a byte array ideally. This could avoid unnecessary creation of the Path objects
at NameNode, thus help reduce the GC problem observed when a large number of
getFileInfo or getListing operations hit NameNode.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
