Hi Ari and all, why not use machine epsilon <https://en.wikipedia.org/wiki/Machine_epsilon>? somewhat over 1e-16 for double precision, or an even more tolerant tolerance? Most cases of meter coordinates 0.001 is realistic. Given projection errors, a bit more is also defensible.
Am I to simplistic? On Sat, Nov 11, 2017 at 11:05 AM, Ari Jolma <[email protected]> wrote: > I have been bitten twice, once with ArcGIS and now with Rasdaman WCS, with > numeric accuracy. > > I'm making a data request on the corner of the bounding box, let's say it > has minimum X of 75042.7273594. I'm setting my minX to that value and I'm > enforcing it to that value with MAX (this was introduced because of a case > with ArcGIS). The I print that to the request with "%.18g" (it was "%.15g" > earlier but I changes it to that because of ArcGIS) and the result is > 75042.7273593999998, which is not good for Rasdaman, since it is formally > less than 75042.7273594. Although, in gdb > > (gdb) p 75042.7273594 > 75042.727359399999 > $7 = false > > Any ideas how to detect/prevent these kinds of situations? Keep the checks > and go back to "%.15g"? Not ok with ArcGIS. > > Ari > > > _______________________________________________ > gdal-dev mailing list > [email protected] > https://lists.osgeo.org/mailman/listinfo/gdal-dev
_______________________________________________ gdal-dev mailing list [email protected] https://lists.osgeo.org/mailman/listinfo/gdal-dev
