Re: Delay Processor

2016-11-15 Thread Andrew Grande
Joe, It's good to know some thinking went into this feature before. Basically, I'm trying to put a spotlight on these 2 areas: 1. Making it, potentially, more generic than a retry loop. E.g. enhance the ControlRate processor. 2. Making these policies more explicit, so a user wouldn't

Re: Delay Processor

2016-11-15 Thread Andrew Grande
Oleg, I'll break my response in 2 threads. I understand your use case of 'delay until X', but frankly would design it differently if we're talking about long-term transactions or schedules. It may involve systems external to NiFi. Anyway, I'd like to keep this use case out of scope for the delay

Re: Delay Processor

2016-11-15 Thread Joe Witt
The concept of flowfile penalization exists to support many of the requirements for delay. This is something which can be set, by a component (processor), on a flow file for some given period of time and which will be honored on the connection it is placed on such that things without this

Re: Delay Processor

2016-11-15 Thread Oleg Zhurakousky
I am +1 on this as I’ve seen many cases in the field where it is applicable and as you mention exponential back off is one of the common one. That said, I am wondering if that has to be a processor at all? Actually let me answer my own question. There are definitely cases where it has to be a

Delay Processor

2016-11-15 Thread Andrew Grande
Hi, I'd lime to check where discussions are on this, ir propose the new component otherwise. Use case: make delay strategies explicit, easier to use. E.g. think of a failure retry loop. Currently, ControlRate is somewhat related, but can be improved. E.g. introduce delay strategies a la