It might take a bit more than allowing input on that processor. One thing you might want to check is how the processor retains state for the path given in the configuration. If this path changes regularly due to EL or incoming flowfiles, the state management for the processor might not be consistent. From memory, I believe the processor only stores timestamps related to the path, but not the path itself. That's most likely one of the reasons why the processor does not allow incoming flowfiles; the state of multiple directories would have to be tracked with no way to automatically clear unused/stale state, without NiFi being able to determine of a path's state is still needed. If you allow input in ListFile, you'll have to make sure that state of the various paths gets tracked properly. There's potentially a sort of memory leak at that point, although admittedly you'd have to send a lot of unique paths through that processor before it became a problem. At the moment, I don't have a good strategy when the "stale" paths could be removed from state.
On Sat, Jan 27, 2018 at 7:21 PM James McMahon <[email protected]> wrote: > This sounds promising. I can give it a try. Assuming this is the proper > place to download the code for ListFile, can you tell me how to pull the > correct source code? I've not yet done this. I would definitely go the > route of fixed folder path with Expression Language support. -Jim > > On Sat, Jan 27, 2018 at 7:09 PM, Mike Thomsen <[email protected]> > wrote: > >> It's input requirement is set to INPUT_FORBIDDEN. It shouldn't be too >> hard to set that to INPUT_ALLOWED and make it able to handle a flowfile or >> a fixed folder path (or even better, fixed folder path w/ EL support). If >> you do a patch, I'll try to find time to do a review. >> >> On Fri, Jan 26, 2018 at 4:03 PM, James McMahon <[email protected]> >> wrote: >> >>> My customer has posted a requirement for a generalized List / Fetch >>> workflow. I would not be able to express the target directory in the >>> ListFile as a fixed value because I would need to reset the input directory >>> each workflow cycle, and the target input directories will not share any >>> common parent. >>> >>> Has anyone recently explored an approach to feed the ListFile or >>> equivalent its target input directory dynamically? It does not seem >>> possible to do that. I've failed to figure this out before, not being able >>> to determine a way to precede the ListFile in the workflow. >>> >> >> >
