Attila Magyar created HADOOP-16673:
--------------------------------------
Summary: Add filter parameter to FileSystem>>listFiles
Key: HADOOP-16673
URL: https://issues.apache.org/jira/browse/HADOOP-16673
Project: Hadoop Common
Issue Type: Improvement
Components: fs
Reporter: Attila Magyar
Currently getting recursively a filtered list of files in a directory is clumsy
because filtering should happen afterwards on the result list.
Imagine we want to list all non hidden files recursively.
The non hidden files filter is defined as:
{code:java}
!name.startsWith("_") && !name.startsWith(".") {code}
Then we can do:
{code:java}
RemoteIterator<LocatedFileStatus> remoteIterator = fs.listFiles(path,
/*recursive*/true);
while (remoteIterator.hasNext()) {
LocatedFileStatus each = remoteIterator.next();
if (filter applies to all of the path elements in each) {
result.add(each);
}
}
{code}
For example each of these paths should be skipped:
* /.a/b/c
* /a/.b/c
* /a/b/.c/
It would be lot better to have a filter parameter on listFiles. This is needed
to solve HIVE-22411 effectively.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
