[
https://issues.apache.org/jira/browse/NIFI-14314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Leathwood updated NIFI-14314:
----------------------------------
Attachment: image001.png
> Improve Kafka3 client performance when reading Avro
> ---------------------------------------------------
>
> Key: NIFI-14314
> URL: https://issues.apache.org/jira/browse/NIFI-14314
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Affects Versions: 2.2.0
> Reporter: Alex Leathwood
> Priority: Major
> Labels: performance, pull-request-available
> Attachments: image001.png
>
>
> In [NiFi-5213 |https://issues.apache.org/jira/browse/NIFI-5213]"Allow
> AvroReader to process files w embedded schema even when the access strategy
> is explicit schema" did so at the expense of [throwing and catching an
> IOException|https://github.com/apache/nifi/blob/cd25603157556d28a4bda961ae5e52d18ac71849/nifi-extension-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/avro/AvroReaderWithExplicitSchema.java#L53]
> for each and every Kafka record without an embedded schema, even when the
> processor is correctly configured.
> In high-performance use cases we have observed up to 2/3 of the cpu time ot
> the respective worker thread being used for this check.
> The exception originates for the kafka/avro client libraries and cannot
> easily be replaced with a lightweight implementation of Exception .
> We propose to either drop this earlier improvement of user experience (PR
> available) )or to reimplement it so that the correct configuration is the
> "happy path" and only the incorrect configuration case be penalized with
> exceptions and ideally a warning.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
