[
https://issues.apache.org/jira/browse/NIFI-4805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16653454#comment-16653454
]
Ed Berezitsky edited comment on NIFI-4805 at 10/17/18 12:32 PM:
----------------------------------------------------------------
Some time ago I have implemented such functionality using scripted processor.
I've designed it as following:
* attribute for counter with unique ID (similar to Mark's suggestion)
* max number of retries (null/empty for infinite)
* input property for sequence of times to penalize. For instance, "1,60, 3600"
- means: first time wait 1 sec, seconds time - 1 min, third time - 1 hour (and
same for each additional retry till max). I didn't want to make it exponential
infinite because at some point you don't want to gradually increase the waiting
time, it just doesn't make sense.
My relationships were defined as following:
* retry - continue (aka success) after penalization
* expired - continue after max retry count reached (also was dropping counter
to 0 to avoid wrapping loops
This implementation also gave me a simple "PenalizeFlowFile" functionality by
simply not looping, or specifying max count to 0
My reasons were to minimize number of processors for a retry flow, and make it
simple as much as possible.
[~alopresto], [~alfonz], [~markap14], [~patricker],
what do you think about this design?
(tagged [~patricker] as he implemented initial version, which I think needs to
be enhanced to include at least features that have been discussed above by
Andy, Martin and Mark).
was (Author: bdesert):
Some time ago I have implemented such functionality using scripted processor.
I've designed it as following:
* attribute for counter with unique ID (similar to Mark's suggestion)
* max number of retries (null/empty for infinite)
* input property for sequence of times to penalize. For instance, "1,60, 3600"
- means: first time wait 1 sec, seconds time - 1 min, third time - 1 hour (and
same for each additional retry till max). I didn't want to make it exponential
infinite because at some point you don't want to gradually increase the waiting
time, it just doesn't make sense.
My relationships were defined as following:
* retry - continue (aka success) after penalization
* expired - continue after max retry count reached (also was dropping counter
to 0 to avoid wrapping.
This implementation also gave me a simple "PenalizeFlowFile" functionality by
simply not looping, or specifying max count to 0
My reasons were to minimize number of processors for a retry flow, and make it
simple as much as possible.
[~alopresto], [~alfonz], [~markap14], [~patricker],
what do you think about this design?
(tagged [~patricker] as he implemented initial version, which I think needs to
be enhanced to include at least features that have been discussed above by
Andy, Martin and Mark).
> allow delayed transfer
> ----------------------
>
> Key: NIFI-4805
> URL: https://issues.apache.org/jira/browse/NIFI-4805
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Reporter: Martin Mucha
> Assignee: Peter Wicks
> Priority: Minor
> Attachments: retry.xml
>
>
> Nifi has concept of penalization, but this penalization has fixed delay, and
> there isn't way how to change it dynamically.
> If we want to implement retry flow, where FlowFile flows in loop, we can
> either lower performance of Processor via yielding it, or we can do active
> waiting. And this is actually recommended as a correct way how to do that.
> It seems, that we can easily implement better RetryProcessor, all we missing
> is `session.penalize` which accepts `penalizationPeriod`. Processor then can
> gradually prolong waiting time after each failure.
>
> Would it be possible to make such method visible?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
