Hi Koji, I am seeing many issues to get this to run reliably. When running this with a few flowfiles at a time, and stepping through by switching processors on and off it works mostly fine, but running this at volume I receive many errors about "no release signal found"
I have tried to fix this in a few different ways, but the issue keeps coming back. This is also not consistent at all - different wait processors will block different flowfiles at different times, without changing any configuration. Stop/Start the flow, and different queues will fill up. Do you have any ideas what could be causing this behavior? I checked the DistributedMapCache Server/Client components, and they all appear to be working OK. Thanks, Martijn On 28 May 2018 at 05:11, Koji Kawamura <[email protected]> wrote: > Hi Martin, > > Alternative approach is using Wait/Notify processors. > I have developed similar flow using those before, and it will work > with your case I believe. > A NiFi flow template is available here. > https://gist.github.com/ijokarumawak/06b3b071eeb4d10d8a27507981422edd > > Hope this helps, > Koji > > > On Sun, May 27, 2018 at 11:48 PM, Andrew Grande <[email protected]> > wrote: > > Martijn, > > > > Here's an idea you could explore. Have the ListFile processor work as > usual > > and create a custom component (start with a scripting one to prototype) > > grouping the filenames as needed. I don't know of the number of files in > a > > set is different every time, so trying to be more robust. > > > > Once you group and count the set, you can transfer the names to the > success > > relationship. Ignore otherwise and wait until the set is full. > > > > Andrew > > > > > > On Sun, May 27, 2018, 7:29 AM Martijn Dekkers <[email protected]> > > wrote: > >> > >> Hello all, > >> > >> I am trying to work out an issue with little success. > >> > >> I need to ingest files generated by some application. I can only ingest > >> these files when a specific set exists. For example: > >> > >> file_123_456_ab.ex1 > >> file_123_456_cd.ex1 > >> file_123_456_ef.ex1 > >> file_123_456_gh.ex1 > >> file_123_456.ex2 > >> > >> Only when a set like that exists should I pick them up into the Flow. > The > >> parts I am looking for to "group" would "ab.ex1", "cd.ex1", "ef.ex1", > >> "gh.ex1", ".ex2". > >> > >> I tried to do this with some expression, but couldn't work it out. > >> > >> What would be the best way to achieve this? > >> > >> Many thanks! >
