#335: export floats and doubles with correct precision
-----------------------+----------------------------------------------------
Reporter: hamish | Owner: grass-dev@…
Type: task | Status: new
Priority: critical | Milestone: 6.4.4
Component: Default | Version: svn-develbranch6
Keywords: precision | Platform: All
Cpu: All |
-----------------------+----------------------------------------------------
Comment(by hamish):
Replying to [comment:21 mmetz]:
> Glynn: Binary to text conversions must use %.9g (float) and %.17g
> (double) in order to preserve fp values in a binary-decimal-binary
> round-trip, e.g. r.out.ascii followed by r.in.ascii.
take care that preserving b-d-b round trip ,on a single platform, is not
the only task or consideration. For the r.out.ascii + r.in.ascii round
trip it may well be appropriate, but while conceding that point I'd argue
that r.*.bin or r.*.mat would be a better choice in those cases.
%.15,7g was chosen because it's perfectly reproducible and doesn't
introduce useless .0000000000000001 crap into the data files which
G_trim_decimal() can't handle. For things like r.univar, r.info, and
v.db.select the output is at least in part for human consumption; there's
no practical need to expose the FP noise. The main thing for us to focus
on I think is all the remaining lossy %f and meaningless %.56f type stuff
in the code, not splitting hairs over preserving quanta finer than
GRASS_EPSILON.
wrt lib/gis/color_write.c, look closely & you'll see there is a +/-
GRASS_EPSILON adjustment on the range to ensure that the rounding exceeds
the range, and so you shouldn't ever get white spots at the peaks and
pits.
best,
Hamish
(this is something I hope python cleans up with 3.0)
--
Ticket URL: <http://trac.osgeo.org/grass/ticket/335#comment:23>
GRASS GIS <http://grass.osgeo.org>
_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
