[
https://issues.apache.org/jira/browse/NIFI-14367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann resolved NIFI-14367.
-------------------------------------
Fix Version/s: 2.4.0
Assignee: Jordan Sammut
Resolution: Fixed
> Kafka Publisher fails when NDJson content is larger than 1MB
> ------------------------------------------------------------
>
> Key: NIFI-14367
> URL: https://issues.apache.org/jira/browse/NIFI-14367
> Project: Apache NiFi
> Issue Type: Bug
> Affects Versions: 2.3.0
> Environment: Docker image running NiFi 2.3.0
> Reporter: Jordan Sammut
> Assignee: Jordan Sammut
> Priority: Blocker
> Fix For: 2.4.0
>
> Attachments: image-2025-03-13-18-06-00-557.png,
> image-2025-03-13-18-06-37-150.png, image-2025-03-13-18-07-36-620.png,
> image-2025-03-14-15-43-41-942.png
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> The Kafka Publisher seemingly fails with no error message when passing NDJson
> which is larger than 1MB. There seems to be a limit somewhere. I have
> attempted to debug the code myself and it looks like the following error is
> returned on this line with the method **
> {*}Iterator<KafkaRecord> convert{*}:
> {code:java}
> readerFactory.createRecordReader(attributes, in, inputLength, logger) {code}
> Error:
> {code:java}
> java.io.IOException: Resetting to invalid mark {code}
> A basic JsonTreeReader and RecordSetWriter were utilized:
> !image-2025-03-13-18-06-00-557.png|width=314,height=210!
> !image-2025-03-13-18-06-37-150.png|width=319,height=213!
> !image-2025-03-13-18-07-36-620.png|width=325,height=215!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
