Nikos wrote:
The following fragment of code ``` fraction_categories = grass.parse_command('r.category', map=fractions, delimiter='\t') fractions_sum = sum([float(x) if not math.isnan(float(x)) else 0 for x in fraction_categories.values()]) msg = "Fractions: {f}".format(f=fraction_categories) g.message(_(msg)) g.message(_("Sum: {s}".format(s=fractions_sum))) ``` gives ``` Fractions: {u'24': u'nan', u'25': u'0.0159235666814', u'26': u'0.0891719747445', u'27': u'nan', u'20': u'0.0238853509712', u'21': u'0.0286624211654', u'23': u'0.143312100132', u'29': u'nan', u'40':u'nan',u'41': u'0.0318471333627', u'1': u'nan', u'3': u'nan', u'2': u'0.0159235669187', u'5': u'nan', u'4': u'nan', u'7': u'nan', u'6':u'nan',u'9': u'nan', u'8': u'nan', u'11': u'0.00159235669186', u'10': u'nan', u'12': u'0.162420386604', u'15': u'0.366242033672', u'16': u'nan', u'18': u'0.0573248423308', u'31': u'nan', u'36': u'nan', u'35': u'0.0636942667255', u'32': u'nan'} Sum: 1.0 ``` Yet the following assertion does not work as expected ``` assert fractions_sum < 1.000000000000001, "Sum of fractions is > 1" ``` and returns ``` Traceback (most recent call last): File "/osgeo/grasstrunk/dist.x86_64-pc-linux-gnu/scripts/r.estimap",line 3979, in <module>sys.exit(main()) File "/osgeo/grasstrunk/dist.x86_64-pc-linux-gnu/scripts/r.estimap",line 3964, in main**supply_parameters) File "/osgeo/grasstrunk/dist.x86_64-pc-linux-gnu/scripts/r.estimap",line 2864, in compute_supplyassert fractions_sum < 1.000000000000001, "Sum of fractions is > 1" AssertionError: Sum of fractions is > 1 ```
Markus Metz:
What is the sum when printed out with %.17g?
Bingo! Following is from a script run, not from the fractions posted in this thread. ``` Sum: 1.0000000000003 Traceback (most recent call last): File "/osgeo/grasstrunk/dist.x86_64-pc-linux-gnu/scripts/r.estimap", line 4207, in <module> sys.exit(main()) File "/osgeo/grasstrunk/dist.x86_64-pc-linux-gnu/scripts/r.estimap", line 4192, in main **supply_parameters) File "/osgeo/grasstrunk/dist.x86_64-pc-linux-gnu/scripts/r.estimap", line 3019, in compute_supply assert fractions_sum < 1.000000000000001, "Sum of fractions is > 1" AssertionError: Sum of fractions is > 1 ```
Why do you use 1.000000000000001 and not e.g. 1.000001 as threshold?
Because of this (wrong I understand now) quick trial and error:
In [5]: 1.00000000000001 > 1 Out[5]: True In [6]: 1.000000000000001 > 1 Out[6]: True In [7]: 1.0000000000000001 > 1 Out[7]: False
1.000000000000001 is most probably a too small difference to 1 considering floating point precision limits.
It is. But then again, how do I decide how "loose" it can be? 1.01? 1.001? 1.0001? N
signature.asc
Description: PGP signature
_______________________________________________ grass-dev mailing list grass-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-dev