@Bryan - Correct, everything is still per queue, just with that convenience feature.
Totally agree with @Salvatore too. I hadn't even thought of nested process groups. On Tue, Apr 12, 2022 at 10:14 PM Salvatore <[email protected]> wrote: > #2 would definitely be convenient. Maybe also include an option whether to > recurse down through nested process groups, or just apply to the selected > process group. > > On Wed, 13 Apr 2022 at 06:05, Bryan Bende <[email protected]> wrote: > > > Makes sense. For # 2, it is still per queue with an "Apply All" > > convenience right? Just trying to differentiate with prioritizing > > across all queues. > > > > On Tue, Apr 12, 2022 at 3:22 PM Ryan Hendrickson > > <[email protected]> wrote: > > > > > > I see two things as particularly useful... > > > > > > 1) Default Prioritizer for new Relationships (Bound to a process > group, > > > similar to how the "Default FlowFile Expiration" can be changed). > > > 2) Applying a prioritizer to an entire Process Group as a one-time > > action. > > > > > > Some background... I'm hand-converting two super-legacy v0.7.3 canvases > > to > > > v1.15.3. Part of that is applying flow priorities all over the place > in > > > the new system. Probably not a common task, but I could see this > feature > > > being useful for other week-to-week work too. > > > > > > Ryan > > > > > > On Tue, Apr 12, 2022 at 1:32 PM Bryan Bende <[email protected]> wrote: > > > > > > > I think there are two different concepts here... The original > > > > discussion is just about default settings for new connections. The > > > > idea in NIFI-6831 is about prioritizing data across multiple queues, > > > > either for all of nifi or within a given process group. > > > > > > > > > > > > On Tue, Apr 12, 2022 at 1:13 PM Mark Bean <[email protected]> > > wrote: > > > > > > > > > > We experimented with the idea of a custom "Global Prioritizer". One > > of > > > > the > > > > > problems with this approach is that it ran the risk of breaking the > > > > > multi-tenancy philosophy. If there were a truly global priority, it > > would > > > > > affect all flows, each may have different priority rules. However, > if > > > > this > > > > > could be applied only at the process group level, it might have > legs. > > > > > > > > > > You can follow the initial approach to such a mechanism in the JIRA > > > > ticket. > > > > > https://issues.apache.org/jira/browse/NIFI-6831 > > > > > > > > > > > > > > > On Tue, Apr 12, 2022 at 12:06 PM Ryan Hendrickson < > > > > > [email protected]> wrote: > > > > > > > > > > > I just went to the config button in my process group, hoping to > > set all > > > > > > relationships in there to priority first.... Lots of right > > clicking & > > > > > > dragging instead. > > > > > > > > > > > > +1 for an approach like that. > > > > > > > > > > > > On Tue, Apr 12, 2022 at 11:44 AM Joe Witt <[email protected]> > > wrote: > > > > > > > > > > > > > Hello > > > > > > > > > > > > > > Certainly the spirit of this is a good idea. Would likely need > > to > > > > > > approach > > > > > > > it at a more flow/process group centric level. > > > > > > > > > > > > > > Thanks > > > > > > > > > > > > > > On Tue, Apr 12, 2022 at 8:34 AM Ryan Hendrickson < > > > > > > > [email protected]> wrote: > > > > > > > > > > > > > > > This would be very helpful. > > > > > > > > > > > > > > > > Ryan > > > > > > > > > > > > > > > > On Fri, Feb 19, 2021 at 4:51 PM Salvatore Foss < > > > > > > [email protected]> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > Do you see much value in being able to specify an > > instance-wide > > > > (or > > > > > > > > > cluster-wide) default prioritizer for all connections that > > do not > > > > > > have > > > > > > > > one > > > > > > > > > manually set? > > > > > > > > > > > > > > > > > > Along with the the following properties in nifi.properties: > > > > > > > > > nifi.queue.backpressure.count=10000 > > > > > > > > > nifi.queue.backpressure.size=1 GB > > > > > > > > > > > > > > > > > > I'd like to see see something like > > > > > > > > > nifi.queue.prioritizer.default=org.apache.nifi.prioritize. > > > > > > > > > PriorityAttributePrioritizer > > > > > > > > > > > > > > > > > > Thoughts? My only concern would be if connection > prioritizers > > > > have a > > > > > > > > > noticeable impact on system resources. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
