Just found this Jira
https://issues.apache.org/jira/browse/NIFI-90

I am surprised it has not got any traction after 3 years...Having used
Apache Airflow for a while, I am looking to retry capabilities in NiFi and
it seems it comes down to "build your own" flow approach, that would handle
retries in a loop and then sleeping for some time. The best alternative
solution I found was suggested by Alessio Palma
<https://community.hortonworks.com/users/16011/ozw1z5rd.html>
https://community.hortonworks.com/questions/56167/is-there-wait-processor-in-nifi.html


IMHO it still would be nice to have retry capabilities like with Apache
Airflow. You can specify a global retry behavior for a flow or specify
retry options per task/processor. This helps a lot to deal with
intermittent issues, like losing network connection or source database
system, being down for maintenance. Airflow can also send an email on retry
and supports a bunch of other parameters around retries:

https://airflow.apache.org/code.html#baseoperator


   - *retries* (*int*) – the number of retries that should be performed
   before failing the task
   - *retry_delay* (*timedelta*) – delay between retries
   - *retry_exponential_backoff* (*bool*) – allow progressive longer waits
   between retries by using exponential backoff algorithm on retry delay
   (delay will be converted into seconds)
   - *max_retry_delay* (*timedelta*) – maximum delay interval between
   retries


   - *on_retry_callback* – much like the on_failure_callback except that it
   is executed when retries occur.

Is everyone using UpdateAttribute and RouteOnAttribute and Sleep method to
implement retries?

thanks,
Boris

Reply via email to