Benjamin Garrett created NIFI-4524:
--------------------------------------
Summary: ListHDFS should allow for glob patterns
Key: NIFI-4524
URL: https://issues.apache.org/jira/browse/NIFI-4524
Project: Apache NiFi
Issue Type: Improvement
Components: Core Framework
Affects Versions: 1.4.0
Reporter: Benjamin Garrett
Priority: Minor
ListHDFS does not support glob patterns. If it did, then I could consolidate a
bunch of ListHDFS instances in my flow down to a single ListHDFS instance.
This is because there are some directory structures that are easier to deal
with as a glob:
/data1/avro
/data1/orc
/data2/avro
/data2/orc
If I wanted to process all 'avro' dirs, then I could use a glob like this:
/*/avro
But since ListHDFS doesn't support a glob, I need many ListHDFS instances for
this situation which is more to maintain.
Fortunately org.apache.hadoop.fs.FileSystem supports a globStatus() alternative
to listStatus().
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
