Attila Magyar created HADOOP-16673: -------------------------------------- Summary: Add filter parameter to FileSystem>>listFiles Key: HADOOP-16673 URL: https://issues.apache.org/jira/browse/HADOOP-16673 Project: Hadoop Common Issue Type: Improvement Components: fs Reporter: Attila Magyar
Currently getting recursively a filtered list of files in a directory is clumsy because filtering should happen afterwards on the result list. Imagine we want to list all non hidden files recursively. The non hidden files filter is defined as: {code:java} !name.startsWith("_") && !name.startsWith(".") {code} Then we can do: {code:java} RemoteIterator<LocatedFileStatus> remoteIterator = fs.listFiles(path, /*recursive*/true); while (remoteIterator.hasNext()) { LocatedFileStatus each = remoteIterator.next(); if (filter applies to all of the path elements in each) { result.add(each); } } {code} For example each of these paths should be skipped: * /.a/b/c * /a/.b/c * /a/b/.c/ It would be lot better to have a filter parameter on listFiles. This is needed to solve HIVE-22411 effectively. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org