Re: [GRASS-user] How GRASS loads raster images into memory?
Andranik Hayrapetyan wrote: I would like to understand how GRASS loads raster images into memory to perform calculations on them. Does it load the entire image into memory and only then do the calculation on them, or it loads image into memory by portions sequentially? The lowest-level read operation reads a specific row into memory. The lowest-level write operation writes the next row from memory. IOW, rows can be read in any order but must be written sequentially. The native GRASS raster format includes a row index, so that rows can be read out of order, and rows which aren't read can be skipped over. Modules which need more than one row at a time typically maintain a sliding window of consecutive rows. This may be done using the rowio library or within the application code. Modules which have non-sequential access patterns (e.g. r.proj) typically create a temporary copy in a format which is more suited to random access. This may be done using the segment library or within the application code. A few modules (e.g. i.fft) read the entire map into contiguous memory. This is typically restricted to modules where use of a tile cache isn't practical and/or the algorithmic complexity is such that the processing time would become a problem before the memory consumption does. Or may be this depends on specific module? In particular I am interested in 2 modules: *r.mapcalc* and *r.patch*. r.patch just uses row-by-row access. r.mapcalc uses the rowio library if the expression uses a neighbourhood modifier with a vertical component, otherwise it just uses row-by-row access. -- Glynn Clements gl...@gclements.plus.com ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] How GRASS loads raster images into memory?
Hi,
Basically, GRASS reads rasters row-by-row.
But it is up to the individual module to
keep more than one row in memory to speed
up calculations. That is why some GRASS modules
have options for the user to adjust the amount
of raster data to be cached in memory.
The GRASS raster engine is very well documented
in the GRASS developers manual:
http://grass.osgeo.org/programming6/gisrasterlib.html#gisrastintro
(for GRASS 6)
Best,
Ben
On 08/16/2013 03:08 PM, Andranik Hayrapetyan wrote:
Good day,
I would like to understand how GRASS loads raster images into memory to
perform calculations on them.
Does it load the entire image into memory and only then do the
calculation on them, or it loads image into memory by portions sequentially?
Or may be this depends on specific module?
In particular I am interested in 2 modules: *r.mapcalc* and *r.patch*.
In my experiments I had strong feeling that it loads image into memory
by portions, because the usage of memory during the calculation was not
big but the HDD I/O was continuous and quite aggressive...
Any answer or links to documentation about this issue can help me a lot.
Thanks in advance!
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user
--
Dr. Benjamin Ducke, M.A.
{*} Geospatial Consultant
{*} GIS Developer
bendu...@fastmail.fm
___
grass-user mailing list
grass-user@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/grass-user
Re: [GRASS-user] How GRASS loads raster images into memory?
On Friday 16 Aug 2013 18:08:53 Andranik Hayrapetyan wrote: Good day, I would like to understand how GRASS loads raster images into memory to perform calculations on them. Does it load the entire image into memory and only then do the calculation on them, or it loads image into memory by portions sequentially? As far as I know, there are no functions in the C-API of GRASS to load the whole raster map into the memory... And in general is not a good idea... You can read/write a raster row by row with: http://grass.osgeo.org/programming7/raster_2get__row_8c.html http://grass.osgeo.org/programming7/raster_2put__row_8c.html Then exist the Raster RowIO that provide a row cache to avoid to read several time always the same row from the hard disk. http://grass.osgeo.org/programming7/rowio_2setup_8c.html And last but not least the Raster Segment, that split the map not by row but in several tiles and provide a tile cache, allowing to open a map in read-write mode. http://grass.osgeo.org/programming7/structSEGMENT.html http://grass.osgeo.org/programming7/segment_2local__proto_8h.html http://grass.osgeo.org/programming7/segment_2get_8c.html http://grass.osgeo.org/programming7/segment_2put_8c.html Regards Pietro ___ grass-user mailing list grass-user@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-user
