Re: [GRASS-user] r.stats: negative cell counts and percentages

2009-10-02 Thread Glynn Clements
Hamish wrote: Hamish wrote: I notice r.info uses unsigned long long + printf %llu Shall we standardize on that? Glynn wrote: The main downside is that you can end up needing a lot of conditional code. I'm not seeing any alternative though. For any module which does math with

[GRASS-user] r.stats: negative cell counts and percentages

2009-10-01 Thread Hermann Peifer
Hi, I have the following raster map: Type of Map: raster Number of Categories: 255 Data Type:CELL Rows: 58000 Columns: 67000 Total Cells: 388600 r.stats (-c and -p) tells me: 1 6082321 -0.15% 2 13691259 2 -3.35%

Re: [GRASS-user] r.stats: negative cell counts and percentages

2009-10-01 Thread Hamish
which versions of grass? the more detailed the answer the better, I am wondering if it is before or after a recent off-by-one fix. tx, Hamish --- On Thu, 1/10/09, Hermann Peifer pei...@gmx.eu wrote: From: Hermann Peifer pei...@gmx.eu Subject: [GRASS-user] r.stats: negative cell counts

Re: [GRASS-user] r.stats: negative cell counts and percentages

2009-10-01 Thread Hermann Peifer
/09, Hermann Peifer pei...@gmx.eu wrote: From: Hermann Peifer pei...@gmx.eu Subject: [GRASS-user] r.stats: negative cell counts and percentages To: grass-user@lists.osgeo.org Received: Thursday, 1 October, 2009, 10:54 PM Hi, I have the following raster map: Type of Map: raster

Re: [GRASS-user] r.stats: negative cell counts and percentages

2009-10-01 Thread Hamish
Hermann wrote: svn up https://svn.osgeo.org/grass/grass/branches/releasebranch_6_4 grass64_release not that it matters here, but do you use 32 bit or 64 bit CPU+OS? (on 8 Sptember 2009) ... I have the following raster map: Type of Map:  raster              Number of Categories: 255

Re: [GRASS-user] r.stats: negative cell counts and percentages

2009-10-01 Thread Glynn Clements
Hamish wrote: I guess there is an integer overflow somewhere(?) Yeah, I think you are right. There are many int++'s in raster/r.stats/stats.c Perhaps those counters++ should be changed to long long? long long is C99, but provided by some C89 compilers as an extension. Any use must be

Re: [GRASS-user] r.stats: negative cell counts and percentages

2009-10-01 Thread Hamish
Hamish wrote: I notice r.info uses unsigned long long + printf %llu Shall we standardize on that? Glynn wrote: The main downside is that you can end up needing a lot of conditional code. I'm not seeing any alternative though. For any module which does math with it (eg casting prior to