All, Please review/merge pull request https://github.com/apache/nifi-minifi-cpp/pull/40 which adds event-based scheduling.
Regards, Andy P.S. Thanks for the info Mark (missed your email earlier). I did notice the bored duration seemed to be implemented in minifi-cpp. Thankfully, we now have both. This "true" event scheduler based on native C++ threading constructs should get us significantly lower latency than even 10ms. ________________________________________ From: Mark Payne <[email protected]> Sent: Friday, January 6, 2017 10:59:01 AM To: [email protected] Subject: Re: MiNiFi event-based scheduling? Andy, Of note - with NiFi, the way that the Timer-Based Scheduling agent works, if you set a schedule of "0 secs" NiFi will determine whether or not there is any work that can be done by the processor. If the processor has no work to do, it is considered "bored" and isn't scheduled again for 10 milliseconds (this is configurable in nifi.properties, using the "nifi.bored.yield.duration" property, of which the default is 10 milliseconds). This is designed to avoid hammering the CPU. Off the top of my head, I am not sure if MiNiFi supports this feature or not, though. Any idea, Aldrin? Thanks -Mark > On Jan 6, 2017, at 10:51 AM, Aldrin Piri <[email protected]> wrote: > > Great! Thanks for taking the time to do so, much appreciated. > > On Fri, Jan 6, 2017 at 10:48 AM, Andrew Christianson < > [email protected]> wrote: > >> Got it. Added this as MINIFI-182 with a link to the wiki for design ideas. >> >> ________________________________________ >> From: Aldrin Piri <[email protected]> >> Sent: Friday, January 6, 2017 10:03:06 AM >> To: dev >> Subject: Re: MiNiFi event-based scheduling? >> >> Hey Andy, >> >> Your inspection would prove correct and it does not appear we have a JIRA >> as of yet either. If you would be interested in creating one, please do, >> otherwise I shall see to it that it gets added. >> >> We are still working to get some of the more foundational framework >> components established but certainly can see how this would be helpful. >> Definitely should track this so that it isn't lost in the shuffle and the >> framework can be refactored to support it. >> >> Thanks! >> --aldrin >> >> On Fri, Jan 6, 2017 at 9:56 AM, Andrew Christianson < >> [email protected]> wrote: >> >>> We noticed that MiNiFi currently only supports timer-based scheduling. >>> Looking at the code, one could set the timer interval to 0, but this >> would >>> result in a hot loop burning 100% of a core. As such, there seems to be >> no >>> way presently to have flow file records processed as soon as incoming >>> records populate the queue. Further, it looks like the FlowController >> code >>> is hard-coded against the timer-based scheduling agent, and unfortunately >>> there is no obvious way to plug-in an alternative scheduler. >>> >>> Are there any near/mid-term plans to implement event-based scheduling? >>> >>> -Andy >>> >>
