stevedlawrence opened a new pull request, #1133:
URL: https://github.com/apache/daffodil/pull/1133
When parsing text floats, ICU sometimes returns a BigInteger, such as the
case when parsing the maximum single precision floating-point value (i.e.
3.40282347E+38).
The current logic for checking if these parsed BigIntegers are in range of a
float is to convert it to a double and then compare that against the maximum
value of a float (promoted to a double). However, converting a BigInteger to a
double can lead to changes in precision, making this comparison inaccurate.
Looking at bit representations makes this more clear:
Float.MaxValue.doubleValue converted to bits is:
1 00011111101 111111111111111111111100000000000000000000000000000
But BigInteger(Float.MaxValue).doubleValue converted to bits is:
1 00011111101 111111111111111111111100101010011011010111111111000
The rules for converting a float to a double are different than converting a
BigInteger to a double, which leads to changes in precision and a double value
that is slightly bigger than the actual float max value, leading to out of
range errors.
Because converting a BigInteger to a double is unreliable, this adds a new
case for BigInteger, which converts it to a BigDecimal and compares that
against the max value, which avoids precision issues. Note that this also
changes the min/maxBD values to be created from a string rather than a double,
since converting a double to a BigDecimal also has related precision issues.
Additionally, when unparsing text floats, ICU does not have a way to unparse
single precision floats. Instead, it converts the float to a double and then
unparses that. However, this leads to text numbers with extra precision that
implies more accuracy than exists for floats, and leads to errors with some
round trip tests. To fix this, this converts the float to a BigDecimal using
the String constructor (which maintains prcision), and then has ICU format the
BigDecimal. This is more expensive, but leads to more correct text floats.
DAFFODIL-2867
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
