I did not have time to respond to this line of discussion as I am
trying to get my class material ready,
but lib/rst also needs to be updated with offsets set to off_t and
fseek to G_fseek
in interp2d.c and write2d.c - see below. This should be done only in
grass7.
I think GRASS64 / 65 rst version should stay as it is - with the
recent fixes
it already should support raster output up to around 30,000x30,000
computed from
tens of millions of points on a higher end PC so if anybody wants to
do anything bigger
I would suggest to use grass7. Also the recent bug fixes in
v.surf.rst done in grass65
need to be ported to grass7 - it is not as straightforward as I
thought - there are too many differences
between grass6 and grass7 already (perhaps time to finish GRASS64 and
release it -
RC1 has been released in December 2008),
Helena
here is my exchange with Glynn on offset topic :
For 38300x30200 raster when writing to temp file, the program says
Cannot fseek elev offset2=-438164932
due to this in write2d.c:
if (fseek(params->Tmp_fd_z, (long)offset2, 0) == -1) {
fprintf(stderr, "Cannot fseek elev offset2=%d\n", (int)offset2);
offset2 is computed in interp2d.c as follows:
offset1 is number of columns
offset = offset1 * (k - 1); is rows offset, k=1,nrows so this can be
38300x30200
offset2 = (offset + ngstc - 1) * sizeof(FCELL) where ngstc is
starting column
for example:
here it does not give error but offset2 is apparently wrong
offset=1077052800, offset1=30200, k=35665
offset=1077052800, offset2=13245556, ngstc=414, fcell=4
here offset2 is negative so it gives an error:
offset=1070318200, offset1=30200, k=35442
offset=1070318200, offset2=-13694496, ngstc=1, fcell=4
Cannot fseek elev offset2=-13694496
I am on 64bit machine compiled with large file support
and I put this into Makefile (according the the advice on the devlist)
ifneq ($(USE_LARGEFILES),)
EXTRA_CFLAGS = -D_FILE_OFFSET_BITS=64 $(VECT_CFLAGS)
endif
hoping that would take care of it, but apparently that is not enough.
all offsets above are defined as int.
So how to fix this?
Should I just replace offset2 with off_t as suggested in the dev list?
Do I need to do anything with fseek?
or am I completely misunderstanding this issue?
LFS doesn't work with fseek(); but LFS does nothing on a 64-bit system
anyhow.
The main problem is that offset1 and offset2 are "int"s, and they need
to be either "long" or "off_t". Also, when you calculate an offset by
multiplying a row number by the row width, one of the operands must be
cast to "long" (or "off_t") *before* the multiply, e.g.:
off_t offset = (off_t) row * row_size;
Casting the result of a multiplication won't work; the value will
already have been truncated.
For LFS (files >2GiB on 32-bit systems), the same issues apply,
except:
1. offsets need to be "off_t" rather than "long".
2. fseek() and ftell() won't work, as they use "long" rather than
"off_t". Use G_fseek() and G_ftell() instead; these use the fseeko()
and ftello() functions on platforms where they are available.
Helena Mitasova
Associate Professor
Department of Marine, Earth
and Atmospheric Sciences
1125 Jordan Hall, Campus Box 8208
North Carolina State University
Raleigh NC 27695-8208
http://skagit.meas.ncsu.edu/~helena/
On Aug 8, 2009, at 2:34 PM, Markus Neteler wrote:
On Sat, Aug 8, 2009 at 7:03 PM, Glynn
Clements<[email protected]> wrote:
Markus Metz wrote:
Yes, if you want to work with vectors not in the current mapset
using
grass7, you have to rebuild topology for all vectors in the
corresponding mapset first.
So it's intentional that GRASS7 is no longer compatible with
previous
vector maps?
Yes, for two reasons. GRASS7 expects now a sidx file while GRASS6
doesn't, and the last GRASS version writing a sidx file (I think
GRASS57) wrote it differently to the new format. I have now
changed the
format numbers and set forward/backward compatibility info.
GRASS6 can
open GRASS7 vectors, but GRASS7 can't open GRASS6 or GRASS5 vectors,
rebuilding topology is required. IOW, I broke backwards
compatibility
which is against GRASS development policy I guess.
Breaking backwards compatibility is allowed in 7.0. We'll probably be
breaking raster compatibility at some point (but in a far more
fundamental way; old rasters won't be recognised at all).
Something this fundamental should have been announced (maybe it was
and I missed it). Also, the error message is misleading.
My argument is that
enabling backwards compatibility would be about the same effort like
rebuilding topology. I could write a module that converts any
existing
sidx file to the new format or builds the spatial index from topo
if no
sidx file is found, then writes a new sidx file in the new
format. That
would avoid rebuilding the core topology and category index.
Maybe the sidx file should have been renamed?
That isn't necessarily a problem, but I must have missed the
announcement.
Was hidden here:
http://lists.osgeo.org/pipermail/grass-dev/2009-July/045098.html
Right; I didn't get the significance of that at the time.
PERMANENT/vector/fields contains:
-rw-r--r-- 1 root root 4117 May 25 2004 cidx
-rw-r--r-- 1 root root 27905 May 25 2004 coor
-rw-r--r-- 1 root root 55 May 25 2004 dbln
-rw-r--r-- 1 root root 278 May 25 2004 head
-rw-r--r-- 1 root root 186 May 25 2004 hist
-rw-r--r-- 1 root root 24538 May 25 2004 sidx
There is no sidx file for vectors in
http://grass.osgeo.org/sampledata/spearfish_grass60data-0.3.tar.gz
IOW: sidx was removed in 6.0 (existing sidx files were silently
ignored), then an entirely different sidx file was added recently,
and
the code is confused by the existence of a 5.7 sidx file, right?
A different name might have been a better choice. Although the 5.7
datasets are old, they worked until a couple of weeks ago, so people
may still be using them.
Error messages about LFS could cause people to waste effort
rebuilding
GRASS to no effect (we've seen something similar with people being
misdirected by the "helpful" information added to the "incompatible
library version" errors).
Ah, ok, you use the grass57 dataset. I have now changed the version
number and compatibility info, you should now get reasonable
warnings/errors with the spearfish57 dataset.
Okay, I'll check it.
Also, this needs to be checked for anything which uses VECT_CFLAGS,
which isn't limited to v.* modules (a few g.* and r.* modules use
vector functions).
I fixed r.drain, v.in.dxf, v.vol.rst. AFAICT all other g.*, r.*
and v.*
modules are ok. Please let me know if I missed something.
I don't know which modules need fixing and which don't. I was just
pointing out that e.g. g.{copy,remove,rename,list} use the vector
libraries (but don't use VECT_CFLAGS; is that a bug?), r.to.vect,
d.vect, etc use the libraries and VECT_CFLAGS, so it isn't just v.*
modules which need to be checked. It might just be v.* modules which
need to be fixed; I don't know.
I got carried away (replacing all fseek/ftell occurrences I could
find
with G_fseek/G_ftell, adjusting off_t as you showed above) and
also made
r3.in|out.v5d LFS-safe, but did not submit. Should I?
Yes; ideally, we should use G_{fseek,ftell} everywhere.
Patch attached.
Note that it now fails to compile here:
make[1]: Entering directory `/home/neteler/grass70/lib/rst/
interp_float'
gcc -I/home/neteler/grass70/dist.x86_64-unknown-linux-gnu/include
-I/home/neteler/grass70/dist.x86_64-unknown-linux-gnu/include -g
-Wall -Werror-implicit-function-declaration -fno-common -mtune=nocona
-m64 -minline-all-stringops -fPIC -I/usr/local/include
-D_FILE_OFFSET_BITS=64 -DPACKAGE=\""grasslibs"\"
-I/home/neteler/grass70/dist.x86_64-unknown-linux-gnu/include
-I/home/neteler/grass70/dist.x86_64-unknown-linux-gnu/include -o
OBJ.x86_64-unknown-linux-gnu/output2d.o -c output2d.c
output2d.c: In function ‘IL_output_2d’:
output2d.c:153: error: void value not ignored as it ought to be
output2d.c:174: error: void value not ignored as it ought to be
output2d.c:190: error: void value not ignored as it ought to be
output2d.c:206: error: void value not ignored as it ought to be
output2d.c:222: error: void value not ignored as it ought to be
output2d.c:238: error: void value not ignored as it ought to be
make[1]: *** [OBJ.x86_64-unknown-linux-gnu/output2d.o] Error 1
No clue how what could be wrong in the patch. Same problem in
raster/r.in.xyz
Feel free to recycle the patch.
Markus
<G_fseek_ftell.diff.gz>
_______________________________________________
grass-dev mailing list
[email protected]
http://lists.osgeo.org/mailman/listinfo/grass-dev
_______________________________________________
grass-dev mailing list
[email protected]
http://lists.osgeo.org/mailman/listinfo/grass-dev
