On Fri, Oct 12, 2018 at 11:22 PM Nikos Alexandris <[email protected]> wrote: > > > Nikos wrote: > [...] > > >1.000000000000001 is most probably a too small difference to 1 considering > >floating point precision limits. > > It is. But then again, how do I decide how "loose" it can be? > > 1.01? > 1.001? > 1.0001?
>From experience I recommend 1 + 1e-6 which is a bit lower than the minimum single precision floating point limit. Instead of assert fractions_sum < 1.000000000000001, "Sum of fractions is > 1" try assert abs(fractions_sum - 1) < 1e-6, "Sum of fractions is != 1" You should not go lower than 1e-15 because here you are hitting the double precision floating point limit. Markus M
_______________________________________________ grass-dev mailing list [email protected] https://lists.osgeo.org/mailman/listinfo/grass-dev
