Not sure if we discussed this before, but it keeps coming up in our tests.

Several of the XQTS tests fail for extremely large or small value numbers
being converted to a double form a string. The issue lies in our custom
parse double code. (Its not a simple algorithm to convert a string to a
double without losing precision.) The cast function's argument is a byte
array that hold a UTF8 string. Java offers a function that parses a string
and returns a double. Using the java function would give us greater
precision than our current code but would require converting the byte array
to a string object.

Do you know of an parseDouble function that could be modified to fit our
use case?

Or do you think it would be ok to break our object avoidance rule in this
case for better precision?

Thanks
Preston

Reply via email to