On Thu, 25 Jun 2009, Markus GRASS wrote:
[...]
very little about in this case. E.g. for completely random access
there might not be a lot of gain.
It is completely random, the next chunk to be read/written can be
anywhere in the file.
[...]
But if there was random access only within a certain section of a
file, that section could be mapped into memory and access would then
be quite fast.
Does it make sense to always map a different section of the file? This
mapping would then need to replace each call to fread/fwrite because of
completely random access. I'm currently using the same method like for
the coor file which seems to work fine so far.
No, I can't imagine that would give much improvement. The scenario I had
hoped for was that when processing a particular sub-region of a vector
map, all the needed data from the index would be near each other in memory
(and thus in the file), thus that area of the file (I was thinking up to
several hundred megabytes max. for performance reasons, but this could
obviously scale with increased datasizes) could be mapped into memory and
intensive I/O (or just reading) done there, with the memory-mapping
mechanism automatically taking care of the reads and writes from/to the
underlying file.
From what you have said it sounds like that would need a complete redesign
of the algorithm used for indexing, or might not be possible. The other
option would be to map the whole of the index file into memory; it might
be worth trying, but as you have already tried to optimise reading/writing
performance based on knowledge of the algorithm, it is unlikely the OS
could do any better. Still might be worth trying, but maybe too much work.
So I think I should shut up now since (as is obvious) I don't know how the
indexing algorithm works...
Paul
_______________________________________________
grass-dev mailing list
[email protected]
http://lists.osgeo.org/mailman/listinfo/grass-dev
