[
https://issues.apache.org/jira/browse/NIFI-14367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jordan Sammut updated NIFI-14367:
---------------------------------
Summary: Kafka Publisher fails when NDJson content is larger than 1MB
(was: Kafka Publisher fails when content is larger than 1MB)
> Kafka Publisher fails when NDJson content is larger than 1MB
> ------------------------------------------------------------
>
> Key: NIFI-14367
> URL: https://issues.apache.org/jira/browse/NIFI-14367
> Project: Apache NiFi
> Issue Type: Bug
> Affects Versions: 2.3.0
> Environment: Docker image running NiFi 2.3.0
> Reporter: Jordan Sammut
> Priority: Blocker
> Fix For: 2.4.0
>
> Attachments: image-2025-03-13-18-06-00-557.png,
> image-2025-03-13-18-06-37-150.png, image-2025-03-13-18-07-36-620.png
>
>
> The Kafka Publisher seemingly fails with no error message when passing NDJson
> which is larger than 1MB. There seems to be a limit somewhere. I have
> attempted to debug the code myself and it looks like the following error is
> returned on this line with the method **
> {*}Iterator<KafkaRecord> convert{*}:
> {code:java}
> readerFactory.createRecordReader(attributes, in, inputLength, logger) {code}
> Error:
> {code:java}
> java.io.IOException: Resetting to invalid mark {code}
> A basic JsonTreeReader and RecordSetWriter were utilized:
> !image-2025-03-13-18-06-00-557.png|width=314,height=210!
> !image-2025-03-13-18-06-37-150.png|width=319,height=213!
> !image-2025-03-13-18-07-36-620.png|width=325,height=215!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
