Alexander Stewart created NIFI-8681:
---------------------------------------
Summary: Allow users to configure FlowFile Concurrency more
specifically on ProcessGroup
Key: NIFI-8681
URL: https://issues.apache.org/jira/browse/NIFI-8681
Project: Apache NiFi
Issue Type: Improvement
Components: Core Framework, Core UI, Documentation & Website
Reporter: Alexander Stewart
With NIFI-7476, users gained the ability to allow a single FlowFile to pass
through a ProcessGroup before the next one is allowed through. I propose that
we expand this concept to allow for grouping via FlowFile attribute. That is
that there would be some hardcoded FlowFile attribute that incoming FlowFile's
would be grouped via, and then only a single FlowFile from a grouping is
allowed to be processed by the ProcessGroup at a time. This would essentially
work as if there were multiple input queues that the ProcessGroup is pulling
from - one for each unique FlowFile attribute value - and a FlowFile can only
be pulled off the queue if there is no file from that group currently being
processed.
It is often the case that one would want to equitably share resources across
some grouping, to prevent one grouping from starving out the others. This
suggestion would provide an easy solution to such problems as setting the
correct grouping would allow for every group to get the same resources without
things getting starved out by one group having sent all of its FlowFiles first.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
