The reason while binary and decimal digits are mixed can be ullustrated by such example. It is necessary 3 decimal fraction digits to represent exactly pbinary power pow(2,-3) : pow(2,-3)=1/8=0.125
On Thu, Sep 12, 2013 at 3:57 PM, David Chase <david.r.ch...@oracle.com>wrote: > This explanation seems to combine numbers of binary digits (1075) > and numbers of decimal digits (17), and therefore makes me a little > nervous, though I think 1100 is a conservative choice that, even if not > 100% correct, will be 99.(over 700 9s)% correct. > > David > > On 2013-09-12, at 1:17 AM, Dmitry Nadezhin <dmitry.nadez...@gmail.com> > wrote: > > When I prepared the patch I didn't try to find the optimal MAX_NDIGITS, > but > > I was sure that MAX_NDIGITS = 1100 is enough. > > Here is an expanation: > > 1) If value of decimal string is larger than pow(10,309) then it is > > converted to positive infinity before the test of nDigits; > > 2) If value of decimal string is smaller than pow(10,309) and larger than > > pow(2,53) then it rounds to Java double with ulp >= 2. > > Half-ulp >= 1. > > The patch is correct when decimal ULP of kept digits is 1 or less. It > is > > true beacuse MAX_NDIGITS = 1100 > 309; > > 3) If value of decimal string is smaller than pow(2,53) but larger than > > 0.5*Doulle.MIN_VALUE = pow(2,-1022-52-1), than > > binary ULP is a multiple of pow(2,-1074). > > Half-ULP is a multiple of pow(2,-1075). > > The patch is correct when decimal ULP of kept digits is pow(10,-1075) > of > > less. > > pow(2,53) has 17 decimal decimal digits. MAX_NDIGITS = 1100 > 1075 + > 17 . > > 4) If value of decimal string is smaller than 0.5*Double.MIN_VALUE then > it > > is converted to zero before the test of nDigits. > > > > > > You can treat replacement of 1075 + 17 by 1100 as my paranoia. > > It still was interesting for me what is the optimal truncation of decimal > > string. >
