These are great ideas! I do love Adam's dynamic relationship idea over creating a failure relationship that is auto-terminated. This would make flow migrations in Registry and NiFi easier.
After some more pondering, I (slowly) realized that this problem affects more than UpdateAttribute, though. You can easily get expression language (explang) exceptions anywhere that explang is used. I'm sure all of us can create a RouteOnAttribute configuration that causes an explang exception which rolls back the flowfile. If we spend so much effort on a solution it would be a shame for that to only apply to UpdateAttribute. For this reason I would favor Matt's idea of a try() or trycatch() explang method. How it might work isn't intuitive, though. Would we have to make it aware of a data type to return? RouteOnAttribute expects boolean but UpdateAttribute expects string (or convertible to string). I can see why the rollback/admin yield solution has been status quo for so long. -- Mike On Fri, Feb 9, 2024 at 10:03 AM u...@moosheimer.com <u...@moosheimer.com> wrote: > 👍 > --Uwe > > > Am 09.02.2024 um 13:50 schrieb Mike Thomsen <mikerthom...@gmail.com>: > > > > How about a third option which is to provide three options: > > > > 1) Default - status quo, exceptions cause it to yield > > 2) Exception = moves forward to success w/ an error attribute, an error > log > > statement that triggers a bulletin, etc to let data manages know what's > > happening. > > 3) Exception = moves to a failure relationship that is otherwise > > autoterminated > > > >> On Thu, Feb 8, 2024 at 7:12 PM Matt Burgess <mattyb...@apache.org> > wrote: > >> > >> Mike's option #2 seems solid but would take a lot of work and there will > >> always be inputs we don't account for. I support that work but in code > >> sometimes we just do a "catch(Throwable)" just so it doesn't blow up. > What > >> about a subjectless "try" or "trycatch" function you can wrap around > your > >> whole expression? If no exception is thrown, the evaluated value will be > >> returned but if one is thrown, you can provide some alternate value that > >> you can check downstream. As this is optional it would retain the > current > >> behavior unless you use it, and then it takes the place of all those > >> ifElse(isXYZValid()) calls we'd need throughout the expression. > >> > >> Regards, > >> Matt > >> > >> > >> On Wed, Feb 7, 2024 at 8:11 PM Phillip Lord <phillord0...@gmail.com> > >> wrote: > >> > >>> IMO... UpdateAttribute has been around since the beginning of time, I > >> can't > >>> see adding a failure relationship. At the same time I understand the > want > >>> for such exceptions to be handled more gracefully rather than rolling > >> back > >>> indefinitely. > >>> I'd vote in favor of considering Moser's option #2... and being able to > >>> implement an "if this then that" logic within your flow. > >>> > >>> Also just thinking... for every UA failure you have to consider a good > >>> failure-management strategy, which MIGHT add a lot of noise to the > flow. > >>> Something that might otherwise easily be identified in a downstream > >>> component and/or database/etc. > >>> > >>> My 2 cents ** > >>> Phil > >>> > >>> > >>> > >>> > >>> > >>>> On Wed, Feb 7, 2024 at 5:18 PM Adam Taft <a...@adamtaft.com> wrote: > >>> > >>>> Or better, the failure relationship just doesn't even exist until the > >>>> property "Has Failure Relationship" is set to True. This involves > >>> updating > >>>> UpdateAttribute to have dynamic relationships (the failure > >> relationships > >>>> appearing on true), which isn't hard to do in processor code. > >>>> > >>>> This has the advantage of being backwards compatible for existing > users > >>> and > >>>> allows the failure relationship to exist for new configurations. > >>> Obviously > >>>> the processor would need an update to catch Expression Language > >>> exceptions > >>>> and then route conditionally to failure. > >>>> > >>>> Just thinking out loud. > >>>> /Adam > >>>> > >>>> > >>>> > >>>> On Wed, Feb 7, 2024 at 1:48 PM u...@moosheimer.com <u...@moosheimer.com > > > >>>> wrote: > >>>> > >>>>> Hi Mike, > >>>>> > >>>>> How about the option of introducing a new property that decides > >> whether > >>>> to > >>>>> route to the 'failure' relationship in the event of an error? > >>>>> If this property is set to false, then the 'failure' relationship is > >>>>> automatically set to 'terminate' (since nothing is routed there > >>> anyway). > >>>>> > >>>>> Then everyone can decide whether and where they want to use this new > >>>>> feature or not. > >>>>> All other options would still be possible with such a solution. > >>>>> > >>>>> -- Uwe > >>>>> > >>>>>> Am 07.02.2024 um 22:15 schrieb Michael Moser <moser...@gmail.com>: > >>>>>> > >>>>>> Hi Dan, > >>>>>> > >>>>>> This has been discussed in the past, as you found with those two > >> Jira > >>>>>> tickets. Personally, I'm still not sure whether a new failure > >>>>> relationship > >>>>>> on UpdateAttribute in 2.0 is a good approach. I have heard from > >> some > >>>>>> dataflow managers that would not want to go through their entire > >>> graph > >>>>> when > >>>>>> upgrading to 2.0 and update every UpdateAttribute configuration. > >>>>>> > >>>>>> I have heard some alternatives to a 'failure' relationship that I > >>> would > >>>>>> like to share as options. > >>>>>> > >>>>>> 1) Add a new property to UpdateAttribute that controls whether a > >>>> flowfile > >>>>>> that causes an expression language exception either yields and > >> rolls > >>>>> back, > >>>>>> or silently fails to update the attribute and sends the flowfile to > >>>>>> success. I personally don't like this, because the use case for > >>>> "silent > >>>>>> failure" seems really like a rarely needed edge case. > >>>>>> > >>>>>> 2) Identify all expression language methods that can throw an > >>> exception > >>>>> and > >>>>>> document that fact in the Expression Language Guide (some methods > >>>> already > >>>>>> mention they can throw an "exception bulletin"). Then implement > >> new > >>>>>> expression methods to check if an expression could fail, and use > >> that > >>>> in > >>>>>> UpdateAttribute advanced rules. For example, if the format() and > >>>>>> formatInstant() methods can fail on a negative number, we create a > >>> new > >>>>>> method such as isValidMilliseconds(). This already exists for some > >>>>> cases, > >>>>>> such as isJson() which can do a quick check of some value before > >>>> calling > >>>>>> jsonPathDelete() on it. > >>>>>> > >>>>>> I'm curious to hear more thoughts on this. > >>>>>> > >>>>>> -- Mike > >>>>>> > >>>>>> > >>>>>> > >>>>>>> On Wed, Jan 31, 2024 at 11:02 AM Dan S <dsti...@gmail.com> wrote: > >>>>>>> > >>>>>>> My team is requesting a failure relationship for UpdateAttribute > >> as > >>>>> seen in > >>>>>>> NIFI-5448 <https://issues.apache.org/jira/browse/NIFI-5448> and > >>>>> NIFI-6344 > >>>>>>> <https://issues.apache.org/jira/browse/NIFI-6344> as we are > >>>>>>> experiencing the same problem where a NIFI Expression Language is > >>>>> throwing > >>>>>>> an exception. In the PR for NIFI-5448 it was mentioned this > >> feature > >>>>> would > >>>>>>> have to wait until NIFI 2.0.0. I wanted to know if there is any > >>> active > >>>>> work > >>>>>>> regarding this and whether eventually there will be a failure > >>>>> relationship > >>>>>>> added to UpdateAttribute? > >>>>>>> > >>>>> > >>>>> > >>>> > >>> > >> >
