[
https://issues.apache.org/jira/browse/NIFI-11341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17706056#comment-17706056
]
Mark Payne commented on NIFI-11341:
-----------------------------------
Marking as related to NIFI-10887. NIFI-10887 introduces a
NonFlushableOutputStream into the StandardProcessSession. That change did not
cause this bug. However, it did expose the bug. Namely, the bug was due to the
underlying OutputStream not having an idempotent close() method. Introducing
this new NonFlushableOutputStream caused that underlying stream to have its
close() method now called twice. This should have been ok but since the close()
method was not idempotent, it caused issues.
> ListenUDPRecord produces invalid FlowFiles due to Content Repository issues
> ---------------------------------------------------------------------------
>
> Key: NIFI-11341
> URL: https://issues.apache.org/jira/browse/NIFI-11341
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Affects Versions: 1.20.0
> Reporter: Peter Kimberley
> Assignee: Mark Payne
> Priority: Major
> Fix For: 2.0.0, 1.21.0
>
> Attachments: NiFi_Flow.json, image.png
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> In our environment, we use {{ListenUDPRecord}} to collect Syslog messages.
> This processor is followed by a {{PartitionRecord}} processor that populates
> an attribute for routing.In release {*}1.19.0{*}, this flow worked without
> issue. In *1.20.0* though, I am seeing intermittent message truncation in
> {{{}PartitionRecord{}}}, with bulletin messages like the following appearing
> regularly:
> {noformat}
> PartitionRecord[id=03ea67a7-0b9c-1c9f-ffff-ffff8d7e5185] Failed to partition
> FlowFile[filename=ca9c3e11-9365-4ff9-9499-29522fc0cab7]:
> com.fasterxml.jackson.core.JsonParseException: Unexpected character (','
> (code 44)): expected a value
> at [Source: (org.apache.nifi.stream.io.NonCloseableInputStream); line: 1,
> column: 381]{noformat}
>
> An example message (note the absence of a Syslog header):
> {noformat}
> itor] [Unit test] Alarm check cfg warning threshold=75 critical threshold=85
> warning alarm <...>{noformat}
> {{ListenUDPRecord}} properties are attached.
> h3. Reproduction
> The attached minimal flow illustrates this setup.
>
> To reproduce this issue, generate improperly-formatted syslog and send to
> {{ListenUDPRecord}}.
>
> In my environment, I have two syslog sources feeding this test cluster.
> Scenario is as follows:
> # First source (compliant Syslog format) feeds in.
> # Flow is OK - no bulletins.
> # Activate second source, which is of an invalid Syslog format and flows to
> the {{parse.failure}} relationship of {{ListenUDPRecord}}. This is expected -
> I deal with this gracefully.
> # Bulletins start firing in {{PartitionRecord}} and the first source starts
> getting truncated randomly.
> Overall, the majority of messages from the well-formed source make it
> through. However I'm seeing roughly 1 bulletin every few seconds, which
> indicates a small proportion of messages are getting truncated.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
