#335: export floats and doubles with correct precision
-----------------------+----------------------------------------------------
Reporter: hamish | Owner: grass-dev@…
Type: task | Status: new
Priority: critical | Milestone: 6.4.4
Component: Default | Version: svn-develbranch6
Keywords: precision | Platform: All
Cpu: All |
-----------------------+----------------------------------------------------
Comment(by mmetz):
Replying to [comment:23 hamish]:
> Replying to [comment:21 mmetz]:
> > Glynn: Binary to text conversions must use %.9g (float) and %.17g
> > (double) in order to preserve fp values in a binary-decimal-binary
> > round-trip, e.g. r.out.ascii followed by r.in.ascii.
>
> take care that preserving b-d-b round trip ,on a single platform, is not
the only task or consideration. For the r.out.ascii + r.in.ascii round
trip it may well be appropriate, but while conceding that point I'd argue
that r.*.bin or r.*.mat would be a better choice in those cases.
Please add this information to the manuals if you think this is crucial.
>
> %.15,7g was chosen because it's perfectly reproducible
No, it is not. %.17,9g is reproducible. See IEEE 754 standard.
> and doesn't introduce useless .0000000000000001 crap
On what platform do you get this crap? %.17,9g does not produce this crap.
> into the data files which G_trim_decimal() can't handle.
G_trim_decimal() is not needed for %.17,9g.
> The main thing for us to focus on I think is all the remaining lossy %f
and meaningless %.56f type stuff in the code
Sure, therefore %.17,9g.
> not splitting hairs over preserving quanta finer than GRASS_EPSILON.
Huh? This thread is not about GRASS_EPSILON. GRASS_EPSILON is 1.0e-15. A
perfectly valid range of double fp data is e.g. 1.0e-200 to 1.0e-300, way
smaller than GRASS_EPSILON. You do not want to corrupt these data.
BTW, I found that fp calculation errors are rather in the range of 1.0e-4
to 1.0e-8.
Markus M
--
Ticket URL: <http://trac.osgeo.org/grass/ticket/335#comment:25>
GRASS GIS <http://grass.osgeo.org>
_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev
