[
https://issues.apache.org/jira/browse/NIFI-7369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Simon Bence updated NIFI-7369:
------------------------------
Status: Patch Available (was: In Progress)
> Fixing precision of floating point numbers when reading Avro record
> -------------------------------------------------------------------
>
> Key: NIFI-7369
> URL: https://issues.apache.org/jira/browse/NIFI-7369
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Affects Versions: 1.9.2
> Reporter: Simon Bence
> Assignee: Simon Bence
> Priority: Major
> Fix For: 1.12.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> h3. Summary:
> When ConvertRecord reads in decimal (logical) data type from Avro record, it
> converts it into Java Double, which depending on the decimal precision, might
> cause precision loss.
> h3. In details:
> AbstractRecordProcessor (parent of ConvertRecord) works in a way it reads in
> record with RecordReader and writes it out with RecordSetWriter. Between the
> two steps, we have an internal representation uses RecordFieldTypes. In this
> example, the following transformations are happening: AVRO into internal
> Record. Internal Record into the format defined by the writer. The specific
> reader where the issue comes up is the AvroReaderWithEmbeddedSchema. This
> (based on the embedded schema) tries to figure out what could be the proper
> internal representation for the given fields, using
> AvroTypeUtil#determineDataType. This util, with the type definition specifies
> decimal [ends
> up|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-extension-utils/nifi-record-utils/nifi-avro-record-utils/src/main/java/org/apache/nifi/avro/AvroTypeUtil.java#L343-L346]
> using double (the originally read data with the reader is BigDecimal and it
> is converted). Double is not capable to correctly represent the incoming data
> when it is high precision so in further steps, a less accurate number will be
> used and written into the output.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
