RE: [EXT] Re: New Standard Pattern - Put Exception that caused failure in an attribute

2018-11-09 Thread Peter Wicks (pwicks)
. Thanks, Peter -Original Message- From: Joe Witt [mailto:joe.w...@gmail.com] Sent: Friday, November 9, 2018 2:36 PM To: dev@nifi.apache.org Subject: Re: [EXT] Re: New Standard Pattern - Put Exception that caused failure in an attribute Peter Ok cool. So i think we agree on the state

Re: [EXT] Re: New Standard Pattern - Put Exception that caused failure in an attribute

2018-11-09 Thread Joe Witt
ove the fragileness into NiFi, where it's hard coded until > the next release. > > Thank you, > Peter > > -Original Message- > From: Joe Witt [mailto:joe.w...@gmail.com] > Sent: Friday, November 9, 2018 12:23 PM > To: dev@nifi.apache.org > Subject: Re: [EXT]

RE: [EXT] Re: New Standard Pattern - Put Exception that caused failure in an attribute

2018-11-09 Thread Peter Wicks (pwicks)
> Sent: Friday, November 2, 2018 11:54 AM > To: dev@nifi.apache.org > Subject: RE: [EXT] Re: New Standard Pattern - Put Exception that > caused failure in an attribute > > Dev Team, > > I don’t think we've reached a conclusion on this discussion, but would like > too. I had

Re: [EXT] Re: New Standard Pattern - Put Exception that caused failure in an attribute

2018-11-09 Thread Joe Witt
: dev@nifi.apache.org > Subject: RE: [EXT] Re: New Standard Pattern - Put Exception that caused > failure in an attribute > > Dev Team, > > I don’t think we've reached a conclusion on this discussion, but would like > too. I had not done enough research when I originally suggeste

RE: [EXT] Re: New Standard Pattern - Put Exception that caused failure in an attribute

2018-11-09 Thread Peter Wicks (pwicks)
A one week bump on this thread. --Peter -Original Message- From: Peter Wicks (pwicks) Sent: Friday, November 2, 2018 11:54 AM To: dev@nifi.apache.org Subject: RE: [EXT] Re: New Standard Pattern - Put Exception that caused failure in an attribute Dev Team, I don’t think we've reached

RE: [EXT] Re: New Standard Pattern - Put Exception that caused failure in an attribute

2018-11-02 Thread Peter Wicks (pwicks)
rom: James Srinivasan [mailto:james.sriniva...@gmail.com] Sent: Tuesday, October 30, 2018 3:00 PM To: dev@nifi.apache.org Subject: Re: [EXT] Re: New Standard Pattern - Put Exception that caused failure in an attribute Apologies if I've missed this in the discussion so far - we use the InvokeHTTP proce

Re: [EXT] Re: New Standard Pattern - Put Exception that caused failure in an attribute

2018-10-30 Thread James Srinivasan
he exception text could be included in an attribute. > > Thanks, > Peter > > -Original Message----- > From: u...@moosheimer.com [mailto:u...@moosheimer.com] > Sent: Saturday, October 27, 2018 10:46 AM > To: dev@nifi.apache.org > Subject: Re: [EXT] Re: New Standa

RE: [EXT] Re: New Standard Pattern - Put Exception that caused failure in an attribute

2018-10-30 Thread Peter Wicks (pwicks)
ache.org Subject: Re: [EXT] Re: New Standard Pattern - Put Exception that caused failure in an attribute Do you really want to mix provenance and data lineage with logging/error information? Writing exception information/logging information within an attribute is not a bad idea in my opinion. If

Re: [EXT] Re: New Standard Pattern - Put Exception that caused failure in an attribute

2018-10-27 Thread u...@moosheimer.com
t;> files and could grep the error out, but a normal user who checks in on the >> flow and sees a FlowFile in the error queue will not know what the cause >> was; this is especially frustrating if retrying the file works without >> failure the second time... Capturing

Re: [EXT] Re: New Standard Pattern - Put Exception that caused failure in an attribute

2018-10-26 Thread Pierre Villard
ng the error message in an attribute > makes this easy to find. > > > > > > > > One thing I worry about too is adding new relationships to core > processors. After an upgrade, won't users need to go to each instance of > that processor and handle the new relation

Re: [EXT] Re: New Standard Pattern - Put Exception that caused failure in an attribute

2018-10-26 Thread Koji Kawamura
parate this into its own relationship, I do not think > > most users would appreciate it being a condition silently not handled by > > the normal failure path. > > > > Thanks, > > Peter > > > > -----Original Message- > > From: Matt Burgess [mailto:mat

Re: [EXT] Re: New Standard Pattern - Put Exception that caused failure in an attribute

2018-10-25 Thread Matt Burgess
NiFi team, I would not want to encounter this > > without that. > > > > Thanks, > > Peter > > > > -Original Message- > > From: Mark Payne [mailto:marka...@hotmail.com] > > Sent: Thursday, October 25, 2018 10:38 PM > > To: dev@nifi.ap

RE: [EXT] Re: New Standard Pattern - Put Exception that caused failure in an attribute

2018-10-25 Thread Peter Wicks (pwicks)
Burgess [mailto:mattyb...@apache.org] Sent: Friday, October 26, 2018 10:18 AM To: dev@nifi.apache.org Subject: Re: [EXT] Re: New Standard Pattern - Put Exception that caused failure in an attribute NiFi (as of the last couple releases I think) has the ability to set auto-terminating

Re: [EXT] Re: New Standard Pattern - Put Exception that caused failure in an attribute

2018-10-25 Thread Matt Burgess
ripting skills in my NiFi team, I would not want to encounter this without > that. > > Thanks, > Peter > > -Original Message- > From: Mark Payne [mailto:marka...@hotmail.com] > Sent: Thursday, October 25, 2018 10:38 PM > To: dev@nifi.apache.org > Subject:

RE: [EXT] Re: New Standard Pattern - Put Exception that caused failure in an attribute

2018-10-25 Thread Peter Wicks (pwicks)
Pattern - Put Exception that caused failure in an attribute I agree - the notion of adding a "failure.reason" attribute is, in my opinion, an anti-pattern that should be avoided. Relationships are not a workaround but rather the preferred approach in this scenario - an attribute I woul

Re: New Standard Pattern - Put Exception that caused failure in an attribute

2018-10-25 Thread Mark Payne
I agree - the notion of adding a "failure.reason" attribute is, in my opinion, an anti-pattern that should be avoided. Relationships are not a workaround but rather the preferred approach in this scenario - an attribute I would consider a workaround. This is due to the fact that not only is it

Re: New Standard Pattern - Put Exception that caused failure in an attribute

2018-10-25 Thread Bryan Bende
I think processors should really have well defined relationships for the error scenarios that need to be handled. Having the exception message is ok for a human who wants to see it, but in order to do anything with it in the flow you will have to have a bunch of parsing/interpreting of the message

New Standard Pattern - Put Exception that caused failure in an attribute

2018-10-24 Thread Peter Wicks (pwicks)
When a FlowFile is routed to failure, frequently there is no clear reason without looking into the actual error message. Some processors work around this by creating many different relationships, but even then frequently the generic Failure relationship also provides little guidance. I've seen