GitHub user martin-mucha opened a pull request:
https://github.com/apache/nifi/pull/2429
Allow delayed transfer
Nifi has concept of penalization, but this penalization has fixed delay,
and there isn't way how to change it dynamically.
If we want to implement retry flow, where FlowFile flows in loop, we can
either lower performance of Processor via yielding it, or we can do active
waiting. And this is actually recommended as a correct way how to do that.
It seems, that we can easily implement better RetryProcessor, all we
missing is `session.penalize` which accepts `penalizationPeriod`. Processor
then can gradually prolong waiting time after each failure.
---
Pull request providing both core changes + exemplary processor
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/martin-mucha/nifi allowDelayedTransfer
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/2429.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2429
----
commit 3fd6e4ce73e2dc9f747378ac0015f2389824968d
Author: Martin Mucha <alfonz19@...>
Date: 2018-01-23T19:31:18Z
NIFI-4805: allow delayed transfer, parameterize session.penalize
Nifi has concept of penalization, but this penalization has fixed
delay, and there isn't way how to change it dynamically.
If we want to implement retry flow, where FlowFile flows in loop, we
can either lower performance of Processor via yielding it, or we can
do active waiting. And this is actually recommended as a correct way
how to do that.
It seems, that we can easily implement better RetryProcessor, all we
missing is `session.penalize` which accepts `penalizationPeriod`.
Processor then can gradually prolong waiting time after each failure.
Signed-off-by: Martin Mucha <[email protected]>
commit 013b89e6e67e622e06b03d56eb9ec9e0a616718e
Author: Martin Mucha <alfonz19@...>
Date: 2018-01-23T20:14:15Z
NIFI-4805: Retry waiting, LogarithmicallyPenalizeFlowFileProcessor
processor, which can be used to create retry loop with logarithmical
waiting.
Signed-off-by: Martin Mucha <[email protected]>
----
---
