Thanks Matt. As a workaround, is there a processor that does not modify the
flow file (content or attribute) at all, and I can use it to delay the
self-referencing flow files to hit the main processor again immediately?
From: Matt Burgess [mailto:mattyb...@apache.org]
Sent: Friday, October 14, 2016 10:25 PM
Subject: Re: Penalize Flow File on Failure
The use of penalize(), yield(), etc. is not enforced by the framework,
so processors can have different behavior, sometimes on purpose, and
sometimes inadvertently. The Developer's Guide has guidance on when
to use such methods , and reviewers often check the submissions to
see if they exhibit such behavior, but it is possible for processors
to handle these cases differently.
Please feel free to log bugs/improvements for such processors, I think
consistent behavior in this vein (when prudent) is a good idea.
On Fri, Oct 14, 2016 at 12:38 PM, Manish Gupta 8 <mgupt...@sapient.com> wrote:
> Hello Everyone,
> In some of the processors I have seen that flow files on failure are not
> being penalized. For example – Kite processors like ConvertJsonToAvro. Is
> there some specific reason why some processors have different behavior?
> I think every processor should penalize every non-success relationship. This
> way, we can hold those failed files (using a self-referencing loop and some
> decent penalty duration) in the queue itself and debug and resolve later.