Hi Mark, I searched and didn't see a ticket. I created one here: https://issues.apache.org/jira/browse/NIFI-9974 | Default Prioritizer for new Relationships
Thanks, Ryan On Fri, Apr 15, 2022 at 10:24 AM Mark Bean <mark.o.b...@gmail.com> wrote: > I like the idea. I'm thinking about how it would be implemented in the UI > (and API). Specifically, I'm comparing this feature of setting the > prioritizer to setting default connection properties via the configuration > of a process group. For the default connection properties, updates to the > process group configuration only change the default settings for any new > connection created; it does not affect existing connections. This was > intentional because it seemed heavy-handed to apply such settings to all > existing connections - especially expiration settings which could result in > data loss. > > However, the recommendation here is for actively changing the prioritizer > in all existing connections (and potentially nested connections in child > process groups). I understand the benefit and use case, but it seems the > two modification styles (new connections or existing connections) would > easily become confused. > > Would a checkbox for "apply prioritizer to all existing connections" be > appropriate? And, if so, we still need to somehow make it clear that this > applies to just prioritizer settings. I do not believe we want the other > connection settings to be recursively applied to existing connections. > > Do we want to introduce a new context menu item for process groups, > "Connection Settings" or something similar? > > Is there a JIRA ticket for this proposed new feature? > > On Tue, Apr 12, 2022 at 11:48 PM Ryan Hendrickson < > ryan.andrew.hendrick...@gmail.com> wrote: > > > @Bryan - Correct, everything is still per queue, just with that > convenience > > feature. > > > > Totally agree with @Salvatore too. I hadn't even thought of nested > process > > groups. > > > > On Tue, Apr 12, 2022 at 10:14 PM Salvatore <salvatoref...@gmail.com> > > wrote: > > > > > #2 would definitely be convenient. Maybe also include an option whether > > to > > > recurse down through nested process groups, or just apply to the > selected > > > process group. > > > > > > On Wed, 13 Apr 2022 at 06:05, Bryan Bende <bbe...@gmail.com> wrote: > > > > > > > Makes sense. For # 2, it is still per queue with an "Apply All" > > > > convenience right? Just trying to differentiate with prioritizing > > > > across all queues. > > > > > > > > On Tue, Apr 12, 2022 at 3:22 PM Ryan Hendrickson > > > > <ryan.andrew.hendrick...@gmail.com> wrote: > > > > > > > > > > I see two things as particularly useful... > > > > > > > > > > 1) Default Prioritizer for new Relationships (Bound to a process > > > group, > > > > > similar to how the "Default FlowFile Expiration" can be changed). > > > > > 2) Applying a prioritizer to an entire Process Group as a one-time > > > > action. > > > > > > > > > > Some background... I'm hand-converting two super-legacy v0.7.3 > > canvases > > > > to > > > > > v1.15.3. Part of that is applying flow priorities all over the > place > > > in > > > > > the new system. Probably not a common task, but I could see this > > > feature > > > > > being useful for other week-to-week work too. > > > > > > > > > > Ryan > > > > > > > > > > On Tue, Apr 12, 2022 at 1:32 PM Bryan Bende <bbe...@gmail.com> > > wrote: > > > > > > > > > > > I think there are two different concepts here... The original > > > > > > discussion is just about default settings for new connections. > The > > > > > > idea in NIFI-6831 is about prioritizing data across multiple > > queues, > > > > > > either for all of nifi or within a given process group. > > > > > > > > > > > > > > > > > > On Tue, Apr 12, 2022 at 1:13 PM Mark Bean <mark.o.b...@gmail.com > > > > > > wrote: > > > > > > > > > > > > > > We experimented with the idea of a custom "Global Prioritizer". > > One > > > > of > > > > > > the > > > > > > > problems with this approach is that it ran the risk of breaking > > the > > > > > > > multi-tenancy philosophy. If there were a truly global > priority, > > it > > > > would > > > > > > > affect all flows, each may have different priority rules. > > However, > > > if > > > > > > this > > > > > > > could be applied only at the process group level, it might have > > > legs. > > > > > > > > > > > > > > You can follow the initial approach to such a mechanism in the > > JIRA > > > > > > ticket. > > > > > > > https://issues.apache.org/jira/browse/NIFI-6831 > > > > > > > > > > > > > > > > > > > > > On Tue, Apr 12, 2022 at 12:06 PM Ryan Hendrickson < > > > > > > > ryan.andrew.hendrick...@gmail.com> wrote: > > > > > > > > > > > > > > > I just went to the config button in my process group, hoping > to > > > > set all > > > > > > > > relationships in there to priority first.... Lots of right > > > > clicking & > > > > > > > > dragging instead. > > > > > > > > > > > > > > > > +1 for an approach like that. > > > > > > > > > > > > > > > > On Tue, Apr 12, 2022 at 11:44 AM Joe Witt < > joe.w...@gmail.com> > > > > wrote: > > > > > > > > > > > > > > > > > Hello > > > > > > > > > > > > > > > > > > Certainly the spirit of this is a good idea. Would likely > > need > > > > to > > > > > > > > approach > > > > > > > > > it at a more flow/process group centric level. > > > > > > > > > > > > > > > > > > Thanks > > > > > > > > > > > > > > > > > > On Tue, Apr 12, 2022 at 8:34 AM Ryan Hendrickson < > > > > > > > > > ryan.andrew.hendrick...@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > This would be very helpful. > > > > > > > > > > > > > > > > > > > > Ryan > > > > > > > > > > > > > > > > > > > > On Fri, Feb 19, 2021 at 4:51 PM Salvatore Foss < > > > > > > > > salvatoref...@gmail.com> > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > > > Do you see much value in being able to specify an > > > > instance-wide > > > > > > (or > > > > > > > > > > > cluster-wide) default prioritizer for all connections > > that > > > > do not > > > > > > > > have > > > > > > > > > > one > > > > > > > > > > > manually set? > > > > > > > > > > > > > > > > > > > > > > Along with the the following properties in > > nifi.properties: > > > > > > > > > > > nifi.queue.backpressure.count=10000 > > > > > > > > > > > nifi.queue.backpressure.size=1 GB > > > > > > > > > > > > > > > > > > > > > > I'd like to see see something like > > > > > > > > > > > > > nifi.queue.prioritizer.default=org.apache.nifi.prioritize. > > > > > > > > > > > PriorityAttributePrioritizer > > > > > > > > > > > > > > > > > > > > > > Thoughts? My only concern would be if connection > > > prioritizers > > > > > > have a > > > > > > > > > > > noticeable impact on system resources. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
