[
https://issues.apache.org/jira/browse/NIFI-1097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14989598#comment-14989598
]
Mark Payne commented on NIFI-1097:
----------------------------------
[~trixpan] - Gotcha. Thanks for the clarification!
The way that I addressed this is to use the option to throw an Exception if the
buffer is full. In this event, I route the FlowFile to failure and 'yield' the
processor (effectively, pause the Processor for some amount of time; the
default duration is 1 second). If we are using a Message Delimiter so that we
are sending multiple messages per incoming FlowFile, and at least one message
is queued up, we route the messages that couldn't be added to the buffer to
'failure' and then enqueue the messages that we could. We do have documented
that if using the Message Delimiter, the messages may be split up into multiple
outbound FlowFiles if some are sent successfully while others are not.
This approach gives us the ability to avoid blocking while still routing the
messages to the appropriate relationship.
Thanks
-Mark
> PutKafka should be changed to use the new 0.8.2 Producer API
> ------------------------------------------------------------
>
> Key: NIFI-1097
> URL: https://issues.apache.org/jira/browse/NIFI-1097
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Reporter: Mark Payne
> Assignee: Mark Payne
> Priority: Critical
> Fix For: 0.4.0
>
> Attachments:
> 0001-NIFI-1097-Rewrite-PutKafka-to-use-the-new-producer-a.patch
>
>
> Currently, PutKafka uses the old Producer API, which provides several
> limitations. The most critical of these limitations is that the asynchronous
> send does not provide an indication of its success. As a result, when
> PutKafka is configured to send asynchronously, it will always route the
> FlowFIle to 'success' even if it does not send the data to Kafka.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
