Markus wrote:
>1.000000000000001 is most probably a too small difference to 1considering>floating point precision limits.
Nikos:
It is. But then again, how do I decide how "loose" it can be? 1.01? 1.001? 1.0001?
Markus:
From experience I recommend 1 + 1e-6 which is a bit lower than the minimum single precision floating point limit. Instead of assert fractions_sum < 1.000000000000001, "Sum of fractions is > 1" try assert abs(fractions_sum - 1) < 1e-6, "Sum of fractions is != 1"
That's really smart. The fractions are fragments from the same "sum", so this assertion refers exactly to this: their "sum" being close to 1. This is what I need!
You should not go lower than 1e-15 because here you are hitting the double precision floating point limit.
Kudos, Nikos
signature.asc
Description: PGP signature
_______________________________________________ grass-dev mailing list [email protected] https://lists.osgeo.org/mailman/listinfo/grass-dev
