Hi Joe, This, indeed, looks like something we should fix. Please create a JIRA and I'll try to have a look into it.
Pierre 2016-09-30 12:01 GMT+02:00 Joe Gresock <[email protected]>: > I forgot to mention the bigger problem with the way this prioritization > works. If three queues lead to a single component, and all 3 are full due > to back pressure, it is easily observable that once Queue A starts to > drain, it will continue to drain as long as it has incoming flow files. > This means that if there's a constant flow of incoming flow files to Queue > A, the other two queues will never be drained (at least, that's my theory > based on an hour of observation). > > I'm going to create a JIRA ticket to track this. > > On Thu, Sep 29, 2016 at 4:02 PM, Joe Gresock <[email protected]> wrote: > > > In my flow, I receive flow files from multiple sources that eventually > get > > funneled into one processor. The connections use OldestFirstPrioritizer, > > and while there is no backpressure, the cross-queue prioritization > appears > > to work as designed (i.e., each queue that leads into the funnel appears > to > > be fairly drained, resulting in essentially a round robin distribution > > among the funneled queues). > > > > However, as soon as backpressure engages, NiFi appears to want to fully > > drain one of the funneled queues, leaving the others untouched. This > > happens even when the undrained queues have flow files older than the > ones > > that are drained first. Is this known behavior in NiFi? If so, do you > > guys think it should be addressed? > > > > -- > > I know what it is to be in need, and I know what it is to have plenty. I > > have learned the secret of being content in any and every situation, > > whether well fed or hungry, whether living in plenty or in want. I can > > do all this through him who gives me strength. *-Philippians 4:12-13* > > > > > > -- > I know what it is to be in need, and I know what it is to have plenty. I > have learned the secret of being content in any and every situation, > whether well fed or hungry, whether living in plenty or in want. I can do > all this through him who gives me strength. *-Philippians 4:12-13* >
