Hi Hamish, interesting idea. But lets put this idea into the library of grass. :)
I would like to suggest to extent the grass lib to read the entire map into the memory if needed. G_get_rast_row() will still work with no modification, but will read the rows from memory and not from the disk. And while implementing this, a tile caching mechanism would be good to. IMHO we have to add new special --flags to allow every module to use this feature. eg: --cache-all will read all tiles (in this case a tile is a row) into the memory. --cache-size-y=<int> will set the number of rows which should be in the cache. And in case we will have "real" tile's (x,y size), we need a flag like --tile-size-x=<int> and --tile-size-y=<int> to set the size of each tile which should be loaded into the cache. But we need to rewrite the modules so they can benefit from this kind of tiles, because G_get_rast_row() needs extra overhead to create a single row from different tiles. IIRC that is approximately what Glynn suggested about a new implementation of the raster map storage. Just my 2 cent. Sören btw.: A 3d tile cache mechanism is already present in the g3d lib. -------- Original-Nachricht -------- Datum: Thu, 19 Jul 2007 19:14:29 +1200 Von: Hamish <[EMAIL PROTECTED]> An: grass5 <[email protected]> Betreff: [GRASS-dev] ramdisk as mapset? > Hi, > > * I have some scripts which are very heavy on raster map disk i/o. > r.cost chugs heavily on the hard drive & the script can take days > to loop through. I don't want to wear a hole in it if I don't have to. > * I have many GB of RAM to play with (enough to hold the region as DCELL) > * The raster modules typically don't use much ram at all. (low overheads > to compete with for RAM) > > I am trying to think of a way to get the raster ops to happen all in RAM > to save time & wear on the hard drive. (script spans a number of r.* > modules) > > ideas so far: > > 1) [Linux] create a 2GB ramdisk using ramfs. use g.mapset to swich into > it, do the heavy i/o. switch back to the original mapset, g.copy the > results map back to the "real" mapset, then destroy the ramdisk. > advantages: easy to do. > disadvantages: it's more of a local hack than a general solution. > > mkdir /mnt/ramdrive > > # default max_size is 1/2 physical ram, auto-resizes 'til then > mount -t ramfs none /mnt/ramdrive > mkdir -p /mnt/ramdrive/tmp_mapset > TMP_MAPSET="/mnt/ramdrive/tmp_mapset" > ln -s "$TMP_MAPSET" $USER/grassdata/$LOCATION/tmp_mapset > cp $USER/grassdata/$LOCATION/$MAPSET/WIND "$TMP_MAPSET" > g.mapset mapset=tmp_mapset > ... > g.module [EMAIL PROTECTED] out=result > ... > g.mapset mapset=$MAPSET > g.copy [EMAIL PROTECTED],result > umount /mnt/ramdrive > > > problem: how to set group ID and mode/umask for ramdrive without > having to do chown+chmod as root? > > > 2) Some backgrounded "grass_mapd" process to dynamically allocate and > hold a single map in memory. It's a child of the main GRASS process so > exiting GRASS tears it down. It could be a "virtual" map sort of like > how a reclass map is just a wrapper for something else. This is just a > very rough idea, probably not so easy to do; but if possible I reckon it > would be a cool tool to have. > > > anyone have ideas? > > thanks, > Hamish > > _______________________________________________ > grass-dev mailing list > [email protected] > http://grass.itc.it/mailman/listinfo/grass-dev -- Der GMX SmartSurfer hilft bis zu 70% Ihrer Onlinekosten zu sparen! Ideal für Modem und ISDN: http://www.gmx.net/de/go/smartsurfer _______________________________________________ grass-dev mailing list [email protected] http://grass.itc.it/mailman/listinfo/grass-dev
