NiFi (as of the last couple releases I think) has the ability to set auto-terminating relationships; this IMO is one of those use cases (for NiFi 1.x). If new relationships are added, they could default to auto-terminate; then the existing processors should remain valid. However we might want an "omnibus Jira" to capture those relationships we'd like to remove the auto-termination from in NiFi 2.0.
Regards, Matt On Thu, Oct 25, 2018 at 10:12 PM Peter Wicks (pwicks) <[email protected]> wrote: > > Mark, > > I agree with you that this is the best option in general terms. After > thinking about it some more I think the biggest use case is for > troubleshooting. If a file routes to failure, you need to be watching the UI > to see what the exception was. An admin may have access to the NiFi log files > and could grep the error out, but a normal user who checks in on the flow and > sees a FlowFile in the error queue will not know what the cause was; this is > especially frustrating if retrying the file works without failure the second > time... Capturing the error message in an attribute makes this easy to find. > > One thing I worry about too is adding new relationships to core processors. > After an upgrade, won't users need to go to each instance of that processor > and handle the new relationship? Right now I'd swagger we have at least five > thousand ExecuteSQL processors in our environment; and while we have strong > scripting skills in my NiFi team, I would not want to encounter this without > that. > > Thanks, > Peter > > -----Original Message----- > From: Mark Payne [mailto:[email protected]] > Sent: Thursday, October 25, 2018 10:38 PM > To: [email protected] > Subject: [EXT] Re: New Standard Pattern - Put Exception that caused failure > in an attribute > > I agree - the notion of adding a "failure.reason" attribute is, in my > opinion, an anti-pattern that should be avoided. Relationships are not a > workaround but rather the preferred approach in this scenario - an attribute > I would consider a workaround. This is due to the fact that not only is it > brittle and complex to add processors that route on such things, but there's > no reason at all to assume that from release to release (even bug > fix/increment releases) that the Exception type or message will be the same, > so the flow could stop working at any time after upgrading nifi. > Relationships offer a well-defined way to explicitly indicate "these are the > possible outcomes," > similar IMO to Java Exception classes vs. throwing Strings in C. > > > > On Oct 25, 2018, at 9:47 AM, Bryan Bende <[email protected]> wrote: > > > > I think processors should really have well defined relationships for > > the error scenarios that need to be handled. Having the exception > > message is ok for a human who wants to see it, but in order to do > > anything with it in the flow you will have to have a bunch of > > parsing/interpreting of the message with a bunch of routing > > processors, which seems more brittle than just having the appropriate > > relationships. > > On Thu, Oct 25, 2018 at 1:36 AM Peter Wicks (pwicks) <[email protected]> > > wrote: > >> > >> When a FlowFile is routed to failure, frequently there is no clear reason > >> without looking into the actual error message. > >> Some processors work around this by creating many different relationships, > >> but even then frequently the generic Failure relationship also provides > >> little guidance. > >> > >> I've seen a few cases recently where processors are including the > >> exception message as an attribute on the FlowFile when routing to failure > >> (ExecuteStreamCommand, new PR for ExecuteSQL). Should this be a standard > >> pattern so that it's easier for users to route failures? > >> > >> --Peter >
