Any recommended examples of successful DistributedMapCache implementations?
On Fri, Feb 26, 2021 at 11:30 AM Andrew Grande <[email protected]> wrote: > I saw it several times and I have a strong conviction this is an > anti-pattern. A dataflow must not mess with start/stop state of processors > or process groups. > > Instead, a flow is always running and one puts a conditional check to > either not get the data in or reroute/deny it. > > If you need to coordinate between processors, use wait/notify. > > If you need to coordinate across nodes, consider DistributedMapCache with > a variety of implementations. > > Finally, can use external stores directly if need to coordinate with other > systems. > > These become part of the flow design. > > Andrew > > On Fri, Feb 26, 2021, 7:42 AM Tomislav Novosel < > [email protected]> wrote: > >> Hi guys, >> >> >> >> I want to stop the processor after exceeding maximum number of retries. >> >> For that I'm using RetryFlowFile processor, after 5 times of retry, it >> routes >> >> flowfile to retries_exceeded. >> >> >> >> When that kicks in, I want to stop the processor which was retried 5 >> times. >> >> >> >> What is the best approach? I have few ones: >> >> >> >> - Execute shell script which sends request to nifi-api to set >> processor state to STOPPED >> - Put InvokeHTTP processor to send request >> >> >> >> The downside is, what if processor-id changes, e.g. deploying to another >> env or nifi restart, not sure about that. >> >> Also, it is nifi cluster with authentication and SSL, so it complicates >> the things. >> >> >> >> Maybe someone has much simpler approach, with backpressure or something. >> >> >> >> Regards, >> >> Tom >> >
