I presume you're using schema inference to determine the column type in the
reader?
Interestingly, the following code:
System.out.println("As a double: " + Double.parseDouble("999999.99"));
System.out.println("As a float: " + Float.parseFloat("999999.99"));
Produces the following output:
As a double: 999999.99
As a float: 1000000.0
The CSV inference logic has the following code:
if (value.contains(".")) {
try {
final double doubleValue = Double.parseDouble(value);
if (doubleValue > Float.MAX_VALUE || doubleValue < Float.MIN_VALUE) {
return RecordFieldType.DOUBLE.getDataType();
}
return RecordFieldType.FLOAT.getDataType();
} catch (final NumberFormatException nfe) {
return RecordFieldType.STRING.getDataType();
}
}
So it determines that there's a decimal point. And the value easily fits within
a Float, so it considers the value to be a Float. We should probably improve
our inference logic to do a better job determining whether a value is a Float
or Double - or else always use a Double instead of a Float.
Thanks
-Mark
On Feb 19, 2020, at 4:42 PM, Shawn Weeks
<[email protected]<mailto:[email protected]>> wrote:
How are you defining the schema and what data type are setting for that column?
Thanks
Shawn
From: KhajaAsmath Mohammed
<[email protected]<mailto:[email protected]>>
Reply-To: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Date: Wednesday, February 19, 2020 at 3:32 PM
To: "[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Subject: NIFI Bug with Convert Record - 999999.99 changed to 1000000.0
Hi,
I am using below text to convert CSV to JSON using convert record processor.
Value of 999999.99 changed to 1000000.0 , May I know how is this possible?
OPERATING_UNIT|TEST_34_2|TEST_34_15|TEST_34_5
"141516"|"1.1"|"1.1"|"1.1"
"141517"|"1.11"|"1.11"|"1.11"
"141518"|"1101.11"|"999999.99"|"1101.11"
<image001.png>
Is this happening with CSV reader or JSON record set writer?
Thanks,
Asmath
