Aleksey, I like your wording of the comment. Thank you very much. I would reformulate a little: <<< We can demonstrate (<link>) that decimal ulp should be less than 10^(-1075) to guarantee correctness === We can demonstrate (<link>) that decimal ulp less than 10^(-1075) is enough to guarantee correctness. >> because decimal ulp may be larger than 10^(-1075) for some inputs. For example, for decimal string "1.1111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111"
binary half-ulp is 2^(-53) and decimal ulp 10^(-53) is enough . On Thu, Sep 12, 2013 at 6:32 PM, Aleksey Shipilev < aleksey.shipi...@oracle.com> wrote: > On 09/12/2013 09:17 AM, Dmitry Nadezhin wrote: > > The patch is correct when decimal ULP of kept digits is pow(10,-1075) > > of less. pow(2,53) has 17 decimal decimal digits. MAX_NDIGITS = 1100 > > > 1075 + 17 . > > OK. That makes more sense. Can we please add the short comment in the > code (maybe next time around)? > > I expected to see something like "The value below is chosen as the > conservative threshold. We can demonstrate (<link>) that decimal ulp > should be less than 10^(-1075) to guarantee correctness; we also > compensate for binary mantissa which takes 53 binary digits, or 17 > decimal ones. Hence, 1075+17 =~ 1100." > > -Aleksey. >
