Boris Tyukin commented on NIFI-90:

I am surprised this Jira has not got any traction after 3 years...Having used 
Apache Airflow for a while, I am looking to retry capabilities in NiFi and it 
comes down to "building your own" flows, that would handle retries in a loop 
and then sleeping for some time. The best solution I found, suggested by 
[Alessio Palma|https://community.hortonworks.com/users/16011/ozw1z5rd.html] 
 but still would be nice to have re-try capabilities like Airflow folks. You 
can specify global re-try behavior for a flow or override per task/processor. 
This helps a lot to deal with intermittent issues, like losing network 
connection or source database system, being down for maintenance. 

> Replace explicit penalization with automatic penalization/back-off
> ------------------------------------------------------------------
>                 Key: NIFI-90
>                 URL: https://issues.apache.org/jira/browse/NIFI-90
>             Project: Apache NiFi
>          Issue Type: Improvement
>          Components: Core Framework
>            Reporter: Joseph Witt
>            Priority: Minor
> Rather than having users configure explicit penalization periods and 
> requiring developers to implement it in their processors we can automate 
> this.  Perhaps keep a LinkedHashMap<Connection ID, Counter> of size 5 or so 
> in the FlowFileRecord construct.  When a FlowFile is routed to a Connection, 
> the counter is incremented.  If the counter exceeds 3 visits to the same 
> connection, the FlowFile will be automatically penalized.  This protects us 
> "5 hops out" so that if we have something like DistributeLoad -> PostHTTP 
> with failure looping back to DistributeLoad, we will still penalize when 
> appropriate.
> In addition, we will remove the configuration option from the UI, setting the 
> penalization period to some default such as 5 seconds.

This message was sent by Atlassian JIRA

Reply via email to