>>>>> Glynn Clements <[EMAIL PROTECTED]> writes:
>>> I tried running r.univar on the same map for comparison, but it
>>> crashed while trying to compute the percentile (the other
>>> statistics were computed okay).
>> Is it due to the memory allocation issues, or?
> I don't know. 30m cells * 8 bytes per cell = ~240MiB. The system has
> 1GiB of swap, so malloc() shouldn't fail.
$ nl -ba stats.c
...
135 /* TODO: mode, skewness, kurtosis */
136 if (param.extended->answer) {
137 qpos_perc = (int *) G_calloc(stats->n_perc, sizeof(int));
138 quartile_perc = (double *) G_calloc(stats->n_perc,
sizeof(double));
139 for (i = 0; i < stats->n_perc; i++) {
140 qpos_perc[i] = (int) (stats->n * stats->perc[i] / 100. -
0.5);
141 }
...
$
Here, it should rather be, e. g.:
qpos_perc[i] = (int) (stats->n * 1e-2 * stats->perc[i] - 0.5);
(BTW, would it benefit from an explicit round ()?)
Otherwise the result of the multiplication overflows the `int'
type:
(30 \cdot 10 ^6) \cdot 90
\approx (2 ^5 \cdot 2 ^{20}) \cdot (2 ^6 + 2 ^5)
> 2 ^{25} \cdot 2 ^6
= 2 ^{31}
[...]
_______________________________________________
grass-dev mailing list
[email protected]
http://lists.osgeo.org/mailman/listinfo/grass-dev
