Is it possible to interrupt/suspend the whole flow (upstream as well as downstream) when a plugin encounters an error?
Michael > On 17 Nov 2015, at 16:46, Oleg Zhurakousky <[email protected]> > wrote: > > Perhaps we may need to start explaining ‘administrative yield” as Circuit > Breaker because that ’s what it is from known pattern perspective > > Question to DEV; Should we consider going as far as deprecating the name and > incorporating CB in it somehow? That would definitely help to avoid the > confusion > Cheers > Oleg > >> On Nov 17, 2015, at 11:19 AM, Mark Payne <[email protected]> wrote: >> >> Michael, >> >> Hello. First, thanks for your interest in NiFi! I'm very happy to see that >> you're already starting to dig in pretty deep here! >> >> I admit that the wording here can be a little confusing. Let me try to >> explain what's going on here and how yielding >> works. When you are developing a Processor in NiFi, especially if reaching >> out to some external resource, >> a common occurrence is to realize that there is no work that can really be >> done at this point in time (for >> example, because the remote resource is not accessible). In that case, the >> processor can call ProcessContext.yield() >> and as a result, the framework will stop scheduling that processor to run >> for the configured amount of time, so that >> we aren't wasting resources when we know there is nothing that we can >> accomplish. This is where the configured >> Yield Duration comes in. >> >> There is, however, a concept of "administratively yielding" a processor. The >> idea here is that the Processor throws >> some unexpected exception. Since the Exception was not of type >> ProcessException, the framework considers this >> a bug and will perform an administrative yield to avoid continually >> triggering a processor that may be in a bad state. >> >> Perhaps using the term "administratively yielding" was a bad idea, as it can >> clearly lead to confusion. >> If you feel that something is not behaving properly, though, or you believe >> that the user experience can be improved >> in some way, please do let us know, as we strive to continually improve the >> user experience. >> >> Thanks >> -Mark >> >> >>> On Nov 17, 2015, at 10:34 AM, Michael De Courci <[email protected]> wrote: >>> >>> FYI >>> >>>> Begin forwarded message: >>>> >>>> From: Michael De Courci <[email protected]> >>>> Date: 16 November 2015 at 15:41:27 GMT >>>> To: [email protected], [email protected], Kefah >>>> Issa <[email protected]> >>>> Subject: NiFI Custom Plugin - Yield Duration Ignored >>>> >>>> I wrote a NAR custom plugin to verify NoFi Exception handling. - version; >>>> nifi-0.3.0 >>>> >>>> The plugin was written to always through a RunTimeException for any >>>> exception including ProcessException. >>>> >>>> I created a simple flow; GetFile->MyPlugin->PutFile >>>> >>>> and set the Yield duration to 60 seconds. >>>> >>>> The log shows that the yield duration is always 1 sec >>>> >>>> I debugged the NiFi framework code and see that the yield duration always >>>> defaults to 1 sec; >>>> See; >>>> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent >>>> >>>> Setting the duration updates the class >>>> org.apache.nifi.controller.StandardProcessorNode >>>> but this does not communicate the change to TimerDrivenSchedulingAgent >>>> >>>> Evidence from log; >>>> >>>> 80eb-552a-471c-8ffd-f2509d71fd2e] failed to process session due to >>>> java.lang.RuntimeException: java.lang.RuntimeException >>>> >>>> 2015-11-15 16:17:58,417 WARN [Timer-Driven Process Thread-7] >>>> c.b.n.p.SampleErrorHandlerProcessor >>>> SampleErrorHandlerProcessor[id=8e8f80eb-552a-471c-8ffd-f2509d71fd2e] >>>> Processor Administratively Yielded for 1 sec due to processing failure >>>> >>>> 2015-11-15 16:17:58,417 WARN [Timer-Driven Process Thread-7] >>>> o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding >>>> SampleErrorHandlerProcessor[id=8e8f80eb-552a-471c-8ffd-f2509d71fd2e] due >>>> to uncaught Exception: java.lang.RuntimeException >>>> >>>> 2015-11-15 16:17:58,419 WARN [Timer-Driven Process Thread-7] >>>> o.a.n.c.t.ContinuallyRunProcessorTask >>>> >>>> java.lang.RuntimeException: null >>>> >>>> at >>>> com.barclays.nifi.processor.SampleErrorHandlerProcessor.onTrigger(SampleErrorHandlerProcessor.scala:103) >>>> ~[na:na] >>>> >>>> >>>> >>>> With Kind Regards >>>> Michael de Courci >>> >> >> >
