[ https://issues.apache.org/jira/browse/NIFI-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536700#comment-15536700 ]
Pierre Villard commented on NIFI-2848: -------------------------------------- After looking into it, I believe this could be related to NIFI-2751. Indeed in processors such as UpdateAttribute or LogAttribute we are getting batch of flow files. Will look further into it. > Queues aren't fairly drained when leading to a single component > --------------------------------------------------------------- > > Key: NIFI-2848 > URL: https://issues.apache.org/jira/browse/NIFI-2848 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework > Affects Versions: 1.0.0, 0.7.0 > Reporter: Joseph Gresock > Attachments: Backpressure_prioritization_test.xml, dominate_1.png, > dominate_2.png, queue_drain.png > > > Consider the scenario where multiple queues lead to a single component and > all of them are full due to back pressure. With the attached template, it is > easily observable that once a single queue starts to drain due to relieved > back pressure, it will continue to drain as long as it has incoming flow > files. This means that if there's a constant flow of incoming flow files to > this queue, the other queues will never be drained (at least, that's my > theory based on several hours of observation). > To reproduce this: > # Load the template into NiFi 1.0.0 > # Play all three GenerateFlowFile processors, but not the UpdateAttribute > processor (this simulates backpressure). Wait until each queue has 1,000 > flow files (max backpressure) > # Stop the GenerateFlowFile processors, and play the UpdateAttribute > processor (this relieves the backpressure) > # Observe which queue has started to drain, and start its GenerateFlowFile > processor > # Observe that the other two queues remain full indefinitely, while the > draining queue continues to replenish and be drained indefinitely -- This message was sent by Atlassian JIRA (v6.3.4#6332)