[ https://issues.apache.org/jira/browse/NIFI-2865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15552019#comment-15552019 ]
ASF subversion and git services commented on NIFI-2865: ------------------------------------------------------- Commit 92cca96d49042f9898f93b3a2d2210b924708e52 in nifi's branch refs/heads/master from [~markap14] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=92cca96 ] NIFI-2865: Refactored PublishKafka and PublishKafka_0_10 to allow batching of FlowFiles within a single publish and to let messages timeout if not acknowledged This closes #1097. Signed-off-by: Bryan Bende <bbe...@apache.org> > Address issues of PublishKafka blocking when having trouble communicating > with Kafka broker and improve performance > ------------------------------------------------------------------------------------------------------------------- > > Key: NIFI-2865 > URL: https://issues.apache.org/jira/browse/NIFI-2865 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions > Reporter: Mark Payne > Assignee: Mark Payne > Fix For: 1.1.0 > > > When NiFi is unable to communicate properly with the Kafka broker, we see the > NiFi threads occasionally block. This should be resolvable by calling the > wakeup() method of the client. Additionally, if Kafka takes too long to > respond, we should be able to route the FlowFile to failure and move on. > PublishKafka has a nice feature that allows a demarcated stream to be sent as > separate messages, so that a large number of messages can be sent as a single > FlowFile. However, in the case of individual messages per FlowFile, the > performance could be improved by batching together multiple FlowFiles per > session -- This message was sent by Atlassian JIRA (v6.3.4#6332)