[
https://issues.apache.org/jira/browse/NIFI-10780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17631911#comment-17631911
]
ASF subversion and git services commented on NIFI-10780:
--------------------------------------------------------
Commit 588e04cd0771d9895554e106ef401e24f096bc96 in nifi's branch
refs/heads/main from David Handermann
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=588e04cd07 ]
NIFI-10780 Improved Event Server to avoid dropping messages
- Improved ByteArrayMessageChannelHandler to call BlockingQueue.offer() while
the server is not shutting down
- Improved NettyEventServer to use ChannelFuture.await() for closing the
channel to allow shutdown to proceed
- Added test method for EventDroppedException handling
Signed-off-by: Nathan Gough <[email protected]>
This closes #6634.
> Avoid Dropping Events in Listen Processors for Full Queues
> ----------------------------------------------------------
>
> Key: NIFI-10780
> URL: https://issues.apache.org/jira/browse/NIFI-10780
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Reporter: David Handermann
> Assignee: David Handermann
> Priority: Major
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Multiple listening Processors, including {{ListenSyslog}} and
> {{{}ListenTCP{}}}, depend on the {{nifi-event-transport}} module and the
> {{NettyEventServer}} to receiving incoming messages. These components use the
> {{ByteArrayMessageChannelHandler}} to add parsed messages to a configured
> queue for subsequent processing when the framework triggers the Processor.
> {{ListenSyslog}} and {{ListenTCP}} use a {{LinkedBlockingQueue}} with a
> configurable maximum size as an internal buffer, and
> {{LinkedBlockingQueue.add()}} throws an {{IllegalStateException}} if the
> caller attempts to add an element that exceeds the maximum queue size. As a
> result of this approach, these Processors can drop messages when the maximum
> queue size is too low.
> The supporting {{ByteArrayMessageChannelHandler}} should be improved to call
> a method other than {{LinkedBlockingQueue.add()}} to avoid dropping messages
> while waiting for the Processor to handle queued events.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
