Hey Boris,

Using the UpdateAttribute and RouteOnAttribute approach is only necessary when 
you want
to retry N number of times (or for some time period) and after that elapses to 
treat the data
differently. Most of the time, though, what is used is to simply loop the 
'failure' relationship back
to the processor itself. So failures would simply remain in the flow, trying 
indefinitely. When a processor
is unable to communicate with some external service due to some intermittent 
issue, that processor
generally should "yield", meaning that the processor will not be triggered for 
some amount of time
(by default it is 1 second).

So in this way, it's very simple to just say "keep trying until you're 
successful." You could also set "age-off"
to occur so that if the data is more than say 1 hour old you can have nifi 
automatically just discard the data.

There are some situations, though, in which users will need to try for say 10 
times and then route the data differently.
We could definitely improve that experience instead of having to use 
UpdateAttribute / RouteOnAttribute. But from
my experience simply looping until successful is the most common scenario and 
so that's probably why we've not
really seen much traction there.


On Mar 6, 2018, at 3:02 PM, Boris Tyukin 
<bo...@boristyukin.com<mailto:bo...@boristyukin.com>> wrote:

Just found this Jira

I am surprised it has not got any traction after 3 years...Having used Apache 
Airflow for a while, I am looking to retry capabilities in NiFi and it seems it 
comes down to "build your own" flow approach, that would handle retries in a 
loop and then sleeping for some time. The best alternative solution I found was 
suggested by Alessio 

IMHO it still would be nice to have retry capabilities like with Apache 
Airflow. You can specify a global retry behavior for a flow or specify retry 
options per task/processor. This helps a lot to deal with intermittent issues, 
like losing network connection or source database system, being down for 
maintenance. Airflow can also send an email on retry and supports a bunch of 
other parameters around retries:


  *   retries (int) – the number of retries that should be performed before 
failing the task
  *   retry_delay (timedelta) – delay between retries
  *   retry_exponential_backoff (bool) – allow progressive longer waits between 
retries by using exponential backoff algorithm on retry delay (delay will be 
converted into seconds)
  *   max_retry_delay (timedelta) – maximum delay interval between retries

  *   on_retry_callback – much like the on_failure_callback except that it is 
executed when retries occur.

Is everyone using UpdateAttribute and RouteOnAttribute and Sleep method to 
implement retries?


Reply via email to