[
https://issues.apache.org/jira/browse/NIFI-15526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-15526:
------------------------------------
Summary: Add Fast Reader Enabled property to Avro Reader (was: OOM
triggered by AVRO org.apache.avro.fastread)
> Add Fast Reader Enabled property to Avro Reader
> -----------------------------------------------
>
> Key: NIFI-15526
> URL: https://issues.apache.org/jira/browse/NIFI-15526
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Affects Versions: 2.7.1, 2.7.2
> Reporter: Josef Zahner
> Assignee: Pierre Villard
> Priority: Critical
> Attachments: image-2026-01-29-16-14-08-609.png,
> image-2026-01-29-16-14-29-698.png, image-2026-01-29-16-16-01-232.png
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> We found an issue causes OOM (used java heap full) for NiFi > v2.5.0 if we
> use AVRO Readers/Writers. It seems that the AVRO Library used in the newest
> NiFi versions has AVRO {{{}org.apache.avro.fastread{}}}=true per default.
> This causes massive OOM problems on our side. We suspect it is due to the
> inheritance of the reader/writer schemas.
> If we disable fastread in {{{}/conf/bootstrap.conf{}}}, the memory leak is
> gone:
> {code:java}
> java.arg.28=-Dorg.apache.avro.fastread=false{code}
> Here real world outputs based on prometheus metrics from used heap for NiFi.
> The screenshot 1 (blue line) shows a max 32GB heap, which never goes down
> again, it just grows (until it would reach 32GB and crashes). Thats taken
> with NiFI 2.7.2 with nothing changed regarding AVRO fast read (default, seems
> to be enabled).
> !image-2026-01-29-16-14-08-609.png!
> Screenshot 2 (orange line) shows another cluster with more or less the same
> use case, but with fastread=false. No memory leak anymore. So it's clearly
> related to the AVRO fastread feature.
> !image-2026-01-29-16-14-29-698.png!
> Yourkit Profiler example of the FastReaderBuilder object and memory usage. 1
> Object consumes about 10GB memory (fastread is on default, so enabled) .
> !image-2026-01-29-16-16-01-232.png!
>
> Would be great if someone of the devs, which is more familiar with all this
> topics, could investigate it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
