Re: [GRASS-dev] r.mapcal rand() strangeness

[Maciej Sieczka]

Glynn Clements pisze:

Maciej Sieczka wrote:

The following silently forces 2147483647 on anything bigger:

$ r.mapcalc 'map=9999999999999999999'
$ r.info -rt map
min=2147483647
max=2147483647
datatype=CELL

That is down to the atoi() function used to parse integers.

Sorry if I got it wrong - is r.mapcalc limited to 32 bit integers? If so, does it have to?

I could change it to use strtol(), which sets errno to ERANGE on
overflow.

Would that make r.mapcalc accept integers bigger than int32?

Trying the same by adding a decimal place separator, results in something bizzare to me:

$ r.mapcalc 'map=99999999999999999999999.0'
$ r.info -rt map
min=99999999999999991611392.000000
max=99999999999999991611392.000000
datatype=DCELL

The presence of a dot causes the value to be parsed as a
floating-point value using atof().

A double has a precision of ~16 decimal digits, which matches what you
see above.

So processing any number with more than 16 decimal diggits in r.mapcalc must yield such "strange" values? And how many diggits after the decimal separator are safe?

At the edges of Float32 range other strange things happen:



$ r.mapcalc 'map=3.40282e38'
$ r.info -rt map
min=340282000000000014192072600942972764160.000000
max=340282000000000014192072600942972764160.000000
datatype=DCELL



$ r.mapcalc 'map=-3.40282e38'
$ r.info -rt map
min=-340282000000000014192072600942972764160.000000
max=-340282000000000014192072600942972764160.000000
datatype=DCEL

Shouldn't the below be rather min,max=340282000000000000000000000000000000000.0?: The the one below should be -340282000000000000000000000000000000000.0 I guess:

Again, you're running into the limits of the precision of the double
type.

Also, bear in mind that the conversion from a binary FP value to a
string of decimal digits is being performed by r.info; r.mapcalc has
no control over that part of the process.

Does that mean that actually in the raster the value is something else than what r.info reports?

Essentially, the %f format behaves badly for large numbers. If you use
%e or %g, it will use exponential notation, with a specific relative
precision rather than a specific absolute precision.

At bigger Float64 values r.mapcalc crashes:

$r.mapcalc 'map=3.40282e110'

*** stack smashing detected ***: r.mapcalc terminated
Aborted (core dumped)

Right. It's sprintf()ing the value into a 64-byte buffer using %f.

I'll change it to use %g:

[1]http://trac.osgeo.org/grass/attachment/ticket/67/r-out-gdal-no-data.diff

Maciek

That fixes it. Thanks. I have noticed that r.null has the same problem though. More modules could? To reproduce, please run the attached script (having set region big enough) and proceed utnil the last, Float64. step. r.null with crash with "*** stack smashing detected ***". I wrote the script to validate Martin's patch for r.out.gdal [1] in extreme conditions but I'm running into issues, thus this thread.

datatypes_grass_gdal.sh
Description: application/shellscript

_______________________________________________
grass-dev mailing list
grass-dev@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-dev