[
https://issues.apache.org/jira/browse/NIFI-8609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne updated NIFI-8609:
-----------------------------
Fix Version/s: 1.14.0
Status: Patch Available (was: Open)
> Improve efficiency of converting Record object to Avro GenericRecord object
> ---------------------------------------------------------------------------
>
> Key: NIFI-8609
> URL: https://issues.apache.org/jira/browse/NIFI-8609
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Reporter: Mark Payne
> Assignee: Mark Payne
> Priority: Major
> Fix For: 1.14.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> During some performance tests and profiling, I found that for a given flow,
> pushing Avro records to Kafka, one of the most expensive parts of the flow
> was converting our Record (MapRecord) object into a GenericRecord object for
> the Avro Writer.
> I created a simple unit test to determine a baseline for performance numbers
> before making any changes. The unit test creates a Record with 100 null
> String fields, half of which have a {{null}} value assigned to them. I then
> converted the record into an Avro GenericRecord via
> {{AvroTypeUtil.createAvroRecord(record, avroSchema);}} in a loop of 1,000,000
> iterations and output how long it took; this was then repeated 1,000 times in
> order to allow the JVM to warm up.
> Numbers on my Macbook Pro showed after the first few iterations that the
> amount of time needed to convert 1 million records was on the order of 4.5
> seconds.
> After updating the code, performance numbers are just under 2 seconds. So
> somewhere on the order of 2x better performance.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
