Attila Magyar created HADOOP-16673:

             Summary: Add filter parameter to FileSystem>>listFiles
                 Key: HADOOP-16673
             Project: Hadoop Common
          Issue Type: Improvement
          Components: fs
            Reporter: Attila Magyar

Currently getting recursively a filtered list of files in a directory is clumsy 
because filtering should happen afterwards on the result list.

Imagine we want to list all non hidden files recursively.

The non hidden files filter is defined as: 
!name.startsWith("_") && !name.startsWith(".") {code}

Then we can do:

RemoteIterator<LocatedFileStatus> remoteIterator = fs.listFiles(path, 
while (remoteIterator.hasNext()) {
 LocatedFileStatus each =;
 if (filter applies to all of the path elements in each) {

For example each of these paths should be skipped:
 * /.a/b/c
 * /a/.b/c
 * /a/b/.c/

It would be lot better to have a filter parameter on listFiles. This is needed 
to solve HIVE-22411 effectively. 

This message was sent by Atlassian Jira

To unsubscribe, e-mail:
For additional commands, e-mail:

Reply via email to