[
https://issues.apache.org/jira/browse/HADOOP-6678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eli Collins updated HADOOP-6678:
--------------------------------
Attachment: hadoop-6678-2.patch
Thanks Hairong. Patch attached.
* Removes FC#isFile and FC#isDirectory, and uses thereof.
* Refactors FC#exists per previous comment and moves to Util
* Refactors FC#checkDest. Note that it is now passed overwrite instead of
false. It (and it's callers) needs unit tests. Filed HADOOP-6730 for that.
* Adds Path#getName tests to TestPath to sanity check my understanding of
FC#checkDest.
* Modifies test classes to use the new helper methods
Still needs to be applied with HDFS-1089 (easy 4 line patch). ant test-core
passes.
> Remove FileContext#isFile, isDirectory and exists
> -------------------------------------------------
>
> Key: HADOOP-6678
> URL: https://issues.apache.org/jira/browse/HADOOP-6678
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
> Reporter: Hairong Kuang
> Assignee: Eli Collins
> Fix For: 0.21.0, 0.22.0
>
> Attachments: hadoop-6678-1.patch, hadoop-6678-2.patch
>
>
> # Add a method Iterator<FileStatus> listStatus(Path), which allows HDFS
> client not to have the whole listing in the memory, benefit more from the
> iterative listing added in HDFS-985. Move the current FileStatus[]
> listStatus(Path) to be a utility method.
> # Remove methods isFile(Path), isDirectory(Path), and exists.
> All these methods are implemented by calling getFileStatus(Path).But most
> users are not aware of this. They would write code as below:
> {code}
> FileContext fc = ..;
> if (fc.exists(path)) {
> if (fc.isFile(path)) {
> ...
> } else {
> ...
> }
> }
> {code}
> The above code adds unnecessary getFileInfo RPC to NameNode. In our
> production clusters, we often see that the number of getFileStatus calls is
> multiple times of the open calls. If we remove isFile, isDirectory, and
> exists from FileContext, users have to explicitly call getFileStatus first,
> it is more likely that they will write more efficient code as follow:
> {code}
> FileContext fc = ...;
> FileStatus fstatus = fc.getFileStatus(path);
> if (fstatus.isFile() {
> ...
> } else {
> ...
> }
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
