[ https://issues.apache.org/jira/browse/NIFI-2865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15546630#comment-15546630 ]
ASF GitHub Bot commented on NIFI-2865: -------------------------------------- GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/1097 NIFI-2865: Refactored PublishKafka and PublishKafka_0_10 to allow bat… …ching of FlowFiles within a single publish and to let messages timeout if not acknowledged You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-2865 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1097.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1097 ---- commit 5b10a50ff34150c2642b544e4ee7a855c080b285 Author: Mark Payne <marka...@hotmail.com> Date: 2016-09-08T23:37:35Z NIFI-2865: Refactored PublishKafka and PublishKafka_0_10 to allow batching of FlowFiles within a single publish and to let messages timeout if not acknowledged ---- > Address issues of PublishKafka blocking when having trouble communicating > with Kafka broker and improve performance > ------------------------------------------------------------------------------------------------------------------- > > Key: NIFI-2865 > URL: https://issues.apache.org/jira/browse/NIFI-2865 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions > Reporter: Mark Payne > Assignee: Mark Payne > Fix For: 1.1.0 > > > When NiFi is unable to communicate properly with the Kafka broker, we see the > NiFi threads occasionally block. This should be resolvable by calling the > wakeup() method of the client. Additionally, if Kafka takes too long to > respond, we should be able to route the FlowFile to failure and move on. > PublishKafka has a nice feature that allows a demarcated stream to be sent as > separate messages, so that a large number of messages can be sent as a single > FlowFile. However, in the case of individual messages per FlowFile, the > performance could be improved by batching together multiple FlowFiles per > session -- This message was sent by Atlassian JIRA (v6.3.4#6332)