Re: Wait from multiple inputs before ending the flow

2015-12-13 Thread Louis-Étienne Dorval
Mark, Yes I've build the project a few days ago, it was in 0.4.0.rc2 if I recall correctly. I can confirm that the Penalty setting is there with the default of 30 seconds. Sadly the system that I invoke returns me a 404 which in the code will be resolved to REL_NO_RETRY and it's the only case whe

Re: Wait from multiple inputs before ending the flow

2015-12-12 Thread Louis-Étienne Dorval
Juan, Very good advice! I've updated my loop to add a processor that will run each X seconds when a retry is needed, that way it won't block the other request : InvokeHttp --(failure)--> UpdateAttribute (increment a counter) --> RouteAttribute (if counter lower than X will retry) --(if retriable

Re: Wait from multiple inputs before ending the flow

2015-12-12 Thread Juan Sequeiros
Good afternoon, All processors have a scheduling tab or run every X time. I would set it at your last invokeHttp. If time is important look at the prioritizer settings too. So that you can send through as example a more important flowFile. On Dec 12, 2015 13:48, "Louis-Étienne Dorval" wrote: >

Re: Wait from multiple inputs before ending the flow

2015-12-12 Thread Louis-Étienne Dorval
Hi again, The MergeContent works perfectly for my case! The flow I've described in the previous email changed a bit, but still it's working as expected. The only problem, now NiFi is much faster than the existing system running in parallel (which is really good). I've done a "retry loop" describ

Re: Wait from multiple inputs before ending the flow

2015-12-06 Thread Joe Witt
Louis-Etienne, NIFI-190 isn't scheduled on anything as of yet. We had some design questions/ideas and your example informs it even further. I think the custom proc method you mention will work out well. Ultimately there will need to be one anyway to deal with the logic of merging this particular

Re: Wait from multiple inputs before ending the flow

2015-12-06 Thread ledor473
Joe, Thanks for the prompt reply. About the merge, both message will be JSON and I need some specific part from both. I'll recheck the doc to see what my options are, but I think that using FlowFile Streams and a custom processor that would do the logic might be good About the HoldProcessor,

Re: Wait from multiple inputs before ending the flow

2015-12-06 Thread Joe Witt
Louis-Etienne, My initial thought is your idea with MergeContent is the right one. However, the issue there is not just the combining of the data but the 'what does merging truly mean in that case'. So it is a bit undefined what the next step will be. Merge the content? If so, how? What is the

Wait from multiple inputs before ending the flow

2015-12-06 Thread Louis-Étienne Dorval
Hi everyone! I'm very excited to start using NiFi and I think that it will be very usefull for a some projects. I've been playing with it for some times and did a few basic flow, but I'm having a hard time figuring how to achieve a part of my flow or if NiFi will be able to do it. I'm building a