Hi Eduardo, I have created the same flow in NIFI, the only thing I am not having is GenerateONDemand. and also am confused about how to set up the release signal for notifying and wait. Can you help me with that?
On Mon, 2 Mar 2020 at 18:26, Eduardo Fontes <[email protected]> wrote: > Sorry, > > Try this link: https://pasteboard.co/IXejthc.png > > On Mon, Mar 2, 2020 at 9:35 AM Midhun Mohan <[email protected]> > wrote: > > > Hi Eduardo, > > Can you please send the Image again, I am not able to open it > > > > On Mon, 2 Mar 2020 at 18:02, Eduardo Fontes <[email protected]> > > wrote: > > > > > Hi Midhun, > > > > > > I think this flow will do what you want. > > > > > > [image: image.png] > > > > > > Be advised that use of Wait/Notify will slow down the average > throughput > > > of your flow because the aditional steps with the Cache. > > > > > > > > > > > > > > > On Sun, Mar 1, 2020 at 12:57 PM Midhun Mohan <[email protected]> > > > wrote: > > > > > >> So I would like to know your suggestions on how to implement wait and > > >> notify. > > >> > > >> In my case messages will flow one after another > > >> > > >> If a failure occurs I need to retry the failure before the orginal > ones, > > >> and notify users if it fails again. > > >> > > >> > > >> On Sun, 1 Mar 2020, 6:31 pm Emanuel Oliveira, <[email protected]> > > wrote: > > >> > > >> > Hi, > > >> > > > >> > Invokehttp have 4 exits > > >> > - success > > >> > - error retry -> loop/count retries (updateattribite) -> if retries > > > > >> max > > >> > then logmessage(error) flow stops here ff discarded Otherwise loop > > >> back to > > >> > invokehttp. > > >> > - error type 2 (bad password etc, ) > > >> > - error type 3 (connectivity problems) you may want also to loop > this > > >> one > > >> > with some delay , count retries and exit /log/stop after max. > > >> > > > >> > I just highly advise prior to invokehttp to have an updateattribute > > >> > deleting all attributes that don't start with some prefix.. to be > sure > > >> your > > >> > ff looping doesnt keep adfing different errors, attributes wriiten > by > > >> > invoke http. > > >> > > > >> > As good practice we prefix name all control attributes with > > >> flow_xxxxxxx so > > >> > our "cleanup," Updateattribute before Invokehttp have special regex > > >> sayng > > >> > "delete all attributes that don't start with flow_ > > >> > (If you interested tomorrow when im back in office i csn share the > > >> regex a > > >> > bit tricky). > > >> > > > >> > Hope this helps, i just would love all processors to be as good as > > >> > invokehttp providing hard multipke exit points for specifics + all > the > > >> nice > > >> > write attributes invokehttp adds that ptovide high details on errors > > or > > >> > success (atttibutes created/added by invokehttp to ff are prefixed > > with > > >> > invoke.xxxxx > > >> > > > >> > Cheers, > > >> > Emanuel O. > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > On Sun, 1 Mar 2020, 12:42 Midhun Mohan, <[email protected]> > > wrote: > > >> > > > >> > > Hi my use case is > > >> > > > > >> > > When a invokehttp processor fails I need to stop the incoming > > message > > >> and > > >> > > retry the failed message 2 times and if it is success start the > > >> execution > > >> > > normally or if it fails again need to notify admins and stop the > > main > > >> > > invoke http processor. > > >> > > > > >> > > > > >> > > > > >> > > Hope you can get the scenario from this > > >> > > > > >> > > On Sun, 1 Mar 2020, 5:30 pm Eduardo Fontes, < > > [email protected] > > >> > > > >> > > wrote: > > >> > > > > >> > > > Hi Midhun, > > >> > > > > > >> > > > You can use DistributedMapCacheServer which starts a cache > service > > >> > within > > >> > > > NiFi's process, but I not recommend in production. Instead you > > could > > >> > use > > >> > > > RedisConnectionPoolService. > > >> > > > > > >> > > > To use solution Wait/Notify the NiFi must know the number of > > >> flowfiles > > >> > to > > >> > > > process in one queue before start another queue. If it is a > > batch/on > > >> > > demand > > >> > > > flow I think it's easier. If it is a continuos/streamming flow > you > > >> must > > >> > > > define a window, by number of flowfiles or by time, to switch > > >> between > > >> > > > queues. > > >> > > > > > >> > > > Please let me know what is your case. > > >> > > > > > >> > > > On Fri, Feb 28, 2020 at 2:39 PM Midhun Mohan < > > >> [email protected]> > > >> > > > wrote: > > >> > > > > > >> > > > > I think wait and notify can solve my problem. I checked the > > >> > documents , > > >> > > > > which is bit unclear to me. Can anyone help me? > > >> > > > > > > >> > > > > Just to know few things like whether we need to configure a > > cache > > >> > > server, > > >> > > > > if so which one is best > > >> > > > > And also getting confused on how to configure wait and notify > > >> signals > > >> > > > > > > >> > > > > On Fri, 28 Feb 2020, 10:59 pm Eduardo Fontes, < > > >> > > [email protected]> > > >> > > > > wrote: > > >> > > > > > > >> > > > > > Hi! Maybe you should try another approach, like Wait/Notify > or > > >> > > > configure > > >> > > > > > prioritizer just before the funnel, depending on your > demand. > > >> > > > > > > > >> > > > > > On Fri, Feb 28, 2020 at 11:40 AM Midhun Mohan < > > >> > > [email protected]> > > >> > > > > > wrote: > > >> > > > > > > > >> > > > > > > Hi all, > > >> > > > > > > How to prioritize one queue before a funnel? > > >> > > > > > > > > >> > > > > > > I need to make sure one queue is emptied before another > > >> queue is > > >> > > > taken > > >> > > > > > > over for executing, I tried with a priority attribute but > it > > >> > fails > > >> > > > > when a > > >> > > > > > > queue with less priority has messages before the queue > with > > >> high > > >> > > > > > priority. > > >> > > > > > > > > >> > > > > > > -- > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > Regards, > > >> > > > > > > Midhun Mohan > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > > > > > > -- > > > > > > Regards, > > Midhun Mohan > > > -- Regards, Midhun Mohan
