Hi Arne, Generally, the approach that is used in such a situation would be to route failure back to the PublishJMS processor itself (without diverting first to a LogAttribute processor). The PublishJMS processors itself should be logging an error with the FlowFile's identity. Then, troubleshooting can be done by inspecting the queue (right-click, List Queue) or via Data Provenance [1]. When a processor encounters backpressure, it still will continue to process data that comes in on self-looping connections. So the failure relationship would still get processed.
Does this help? Thanks -Mark [1] http://nifi.apache.org/docs/nifi-docs/html/user-guide.html#data_provenance On Oct 23, 2017, at 6:46 AM, Arne Degenring <[email protected]<mailto:[email protected]>> wrote: Hi, We came across a situation when we experience a kind of “back pressure dead lock”. In our setup, this occurs around PublishJMS when the target JMS queue is full. Please find attached a screenshot of the relevant flow. The failure relation we route to a logging component, and then back to PublishJMS for retry. Sooner or later, the failure and retry queues will become full and produce backpressure towards the main input (which is good). The problem is that the same back pressure is also applied to the retry queue. In this situation, PublishJMS will not be called at all any longer. Even when the JMS problem resolves, the whole thing stays deadlocked. Is there a recommended way to avoid such situation? Obviously, an admin can temporarily increase the back pressure threshold of the failure connection, once the JMS problem is resolved. But it would be nicer if the problem could resolve automatically, i.e. PublishJMS should keep retrying somehow. Any hints? Thanks, Arne <backpressure-deadlock.png>
