Great work!
In keeping with good GRASS traditions, I would suggest to have this
user-configurable via an env var. Perhaps:
GRASS_SPATIAL_INDEX =
AUTO (default: keep in mem if enough RAM available)
FILE
MEMORY
Cheers,
Ben
Moritz Lennert wrote:
On 23/06/09 20:28, Markus GRASS wrote:
Paolo Cavallini wrote:
Markus GRASS ha scritto:
What to do now? Leave it all in memory as in grass6, build in memory
then write out (risk of running out of memory on massive datasets), or
keep it always in file? I'll not commit any time soon (also waiting for
the lib/raster commotion to settle down), I need feedback on how to
proceed or if I should forget about it.
I think advice from Radim would be very useful here.
All the best.
OK, let me rephrase. I think I have two alternatives to the current
implementation of the vector spatial index and would like to know if
grass7 should get 1) faster vector display and lower memory consumption
at the cost of (sometimes) slower vector processing [1], 2) faster
vector display, a similar speed in vector processing but keep the risk
of running out of memory when processing large datasets, or 3) no
changes to the spatial index. IMHO this should be a general decision of
the GRASS community, not of one or two developers.
What size of vector data are we talking about concerning the risk of
running out of memory ? Would it be possible to implement both 1) and 2)
with 2) being the default and a flag to switch to 1) for very large
vectors ?
You wrote:
Considering that a file based spatial index is only useful for massive
vectors where memory can become a limiting factor, I hesitate to commit
to trunk.
Well, I don't know what you call massive, but one of the main problems
with the memory based index is that currently the spatial index is
rebuilt for each run of v.what as it is stored no where. This makes
querying large (e.g. 20-30.000 polygons) _very_ slow when using the GUI
(which will be the only option in grass7 as xmons have disappeared).
I'm not sure I understand everything correctly here, but I have the
feeling that there are two questions here:
1) Should we have a file-based storage of the spatial index ? This can
then be read into memory when necessary, which still should be faster
than rebuilding it each time.
To this I clearly say +1.
The question here is: how to make sure the index is up to date ?
2) Should the entire treatment of the index be file-based, i.e. the
index is never read into memory, but always accessed via file, with the
speed penalties you spoke about.
To this I would say, if we can have a file-based, permanent storage of
the index, but read into memory for treatment, unless a flag says not to
load it into memory, than this would probably be the ideal, within my
limited understanding of the issue.
Moritz
_______________________________________________
grass-dev mailing list
[email protected]
http://lists.osgeo.org/mailman/listinfo/grass-dev
--
Benjamin Ducke
Senior Geospatial Consultant
Oxford Archaeology
Janus House
Osney Mead
OX2 0ES
Oxford, U.K.
Tel: +44 (0)1865 263 800 (switchboard)
Tel: +44 (0)1865 980 758 (direct)
Fax :+44 (0)1865 793 496
[email protected]
------
Files attached to this email may be in ISO 26300 format (OASIS Open Document
Format). If you have difficulty opening them, please visit http://iso26300.info
for more information.
_______________________________________________
grass-dev mailing list
[email protected]
http://lists.osgeo.org/mailman/listinfo/grass-dev
