[
https://issues.apache.org/jira/browse/NIFI-8437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17325092#comment-17325092
]
Mark Payne commented on NIFI-8437:
----------------------------------
[~lannanml] that would not address the problem. It would avoid the Exception,
but it would end up in leaving the stream consumed. The issue appears to be
JsonPathReader. It's currently wrapping the incoming {{InputStream}} in a
{{BufferedInputStream}}. As a result, it's buffering the data into memory and
then when reset is called, it fails if it didn't buffer everything. I think we
can avoid this by simply removing the line of code in {{JsonPathReader}} that
wraps the InputStream in BufferedInputStream - just use the {{InputStream}} as
it is. It should not have been wrapped anyway because the JSON Parser is going
to handle buffering on its own as it reads the data.
> RecordReader 'Infer Schema' for large records may throw BufferedInputStream
> error
> ---------------------------------------------------------------------------------
>
> Key: NIFI-8437
> URL: https://issues.apache.org/jira/browse/NIFI-8437
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Affects Versions: 1.12.1
> Environment: Mac, BigSur 11.2.3, 32MB RAM, 8 Core Intel i9, 1 TB
> storage
> Reporter: Matthew Lannan
> Priority: Major
> Labels: easyfix, newbie
> Attachments: Nifi-stack-trace.txt,
> PDF_Encode_Decode_template-with-bug.xml
>
>
> In the RecordReader(s), if the Schema Access Strategy is set to Infer Schema
> and the record is a single large record(tested with record ~50MB), the
> InferSchemaAccessStrategy will error.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
