Hi Luca, Sorry to hear that you are having an issue with Wait processor.
By looking at the code and testing it locally, I couldn't find a cause of the issue nor reproduce it. However, theoretically such a situation can happen when there are too many queued FlowFiles in the connection in front of the Wait processor exceeding the processor's throughput. Especially where the Wait processor is busy processing the recently added FlowFiles and not being able to take care of older FlowFiles. By default, the order of processing queued FlowFiles is undefined. To eliminate one of the possible uncertainties, I'd recommend using OldestFlowFileFirstPrioritizer at the incoming connection, so that older FlowFiles can be processed first. Hope this helps, Koji On Thu, Dec 17, 2020 at 1:36 AM Luca Giovannini <[email protected]> wrote: > > > > Hi All, > > > > I am using the Wait/Notify couple of processors, with a setting of the Wait > processor as follows: > > Wait mode = keep in the upstream connection > Expiration duration = 10 min > Wait penalty duration = 30 sec > > > > I use the Wait/Notify in a split/merge situation where al the split parts > have to be processed before the Wait is released. > > Occasionally, one of the branches of the split fails or takes too much time > and that is taken care with an “expired” branch of the Wait processor. > > > > The flow runs well for days but then slowly the incoming connection to the > Wait processor starts to grow, and checking the “queued duration” and > “lineage duration” of the flowfiles in the queue list I see that they are > much older than the “Expiration duration” setting (days vs. 10 minutes). I > have also noticed that, when this happens, if I change the “expiration > duration” setting to a smaller value the Wait processor starts to clean up > the queue by routing the flowfiles to expired. I have seen this problem > happening with all sorts of different values for “Expiration duration” and > “Wait penalty”, the ones that I provided above are just an example. > > I am using NiFi 1.11.4. > > > > What can be happening? > > This unpredictable behaviour is blocking the whole flow and making the Wait > processor unusable for me ☹ > > > > Thank you very much for your help and support! > > > > Luca > > > > Le informazioni contenute in questo messaggio di posta elettronica sono > riservate e confidenziali e ne e' vietata la diffusione in qualsiasi modo o > forma. Qualora Lei non fosse la persona destinataria del presente messaggio, > La invitiamo a non diffonderlo e ad eliminarlo, dandone gentilmente > comunicazione al mittente. > > The information included in this e-mail and any attachments are confidential > and may also be privileged. If you are not the correct recipient, you are > kindly requested to notify the sender immediately, to cancel it and not to > disclose the contents to any other person.
