I believe the list/fetch processors could work, it is just going to take a
step in between to route the specific directory I am looking for. For
example, I want to pull back all files from the previous hour (by previous
hour I am referencing my directory structure '/in/data/yyyy/MM/dd/HH/'), so
if I put a RouteOnAttribute inbetween the List/Fetch I should be able to
match for the specific directory I need... working on this now with the
expression language to match
/in/data/${now():toNumber():minus(3600000):format('yyyy/MM/dd/HH')}On Mon, Feb 8, 2016 at 11:24 AM, Bryan Bende <[email protected]> wrote: > Hello, > > There is another processor, FetchHDFS, which supports expression language > for the file to fetch. > It is intended to be used with ListHDFS to feed the results of the listing > into FetchHDFS. > > Do the List/Fetch processors work for your use-case? > > -Bryan > > > On Mon, Feb 8, 2016 at 8:23 AM, Michael Parco <[email protected]> wrote: > > > Has there been thoughts/discussion for thoughts on the GetHDFS processor > > supporting the NiFi expression language for file retrieval? Similar to > how > > PutHDFS can utilize the NiFi expression lanaguage to dynamically generate > > directories, I see the benefit for the GetHDFS processor to do the same > in > > order to retrieve files based on say a time (YYYY;/MM/DD/hh) directory > > structure. > > > > ~ > > >
