[
https://issues.apache.org/jira/browse/NIFI-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16565917#comment-16565917
]
ASF GitHub Bot commented on NIFI-3531:
--------------------------------------
GitHub user mosermw opened a pull request:
https://github.com/apache/nifi/pull/2931
NIFI-3531 Moved session.recover in JMSConsumer to exceptional situations
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mosermw/nifi nifi-3531
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/2931.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2931
----
commit 8309747889c3187005a0ce4e4c0c11b1e3281e10
Author: Mike Moser <mosermw@...>
Date: 2018-08-01T20:11:35Z
NIFI-3531 Moved session.recover in JMSConsumer to exceptional situations
----
> modify session.recover behaviour in nifi-jms-processors to cope with
> high-traffic JMS
> -------------------------------------------------------------------------------------
>
> Key: NIFI-3531
> URL: https://issues.apache.org/jira/browse/NIFI-3531
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Reporter: Dominik Benz
> Priority: Major
>
> As described in this mailing list post
> http://apache-nifi-developer-list.39713.n7.nabble.com/session-recover-behaviour-in-nifi-jms-processor-td14940.html
> the current implementation of nifi-jms-processor, especially of JMSConsumer
> https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/JMSConsumer.java
> causes frequent re-delivery of JMS messages due to the following behaviour:
> 1) several threads perform the JMS session callback in parallel
> 2) each callback performs a session.recover
> 3) during high traffic, the situation arises that the ACKs from another
> thread may not (yet) have arrived at the JMS server
> 4) this implies that the pointer of session.recover will reconsume the
> not-yet-acked message from another thread
> I understood Nifi prefers message duplication over message loss - however, in
> our case the number of re-delivered messages is "piling up" over time,
> leading to a growing number of un-acked messages in JMS and finally to
> storage problems on the JMS server side.
> It would be great to modify the nifi-jms-processor package to reliably cope
> with higher-traffic JMS sources. Ideas from my side would be / include:
> 1) make synchronous / asynchronous delivery configurable
> 2) add configuration option for custom ACKing modes (i.e. non-standard JMS
> modes like individual ACKing or NO_ACK)
> 3) add configuration option for JMS message selectors
> From other projects, I found the JMS communication options in this
> spark-jms-receiver package
> https://github.com/tbfenet/spark-jms-receiver
> very helpful (in which situations are synchronous / asynchronous approaches
> reliable, how to buffer messages before acking them, how/when to ack, ...).
> Here's also a java port of a subset:
> https://github.com/bernhardschaefer/spark-jms-receiver-java
> For any mentioned option I'm also happy to help!
> Best & thanks for a great piece of software,
> Dominik
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
