What about running GRASS in something like Xgrid or Grid Engine?
I am about to purchase a cluster of Mac Pros for filtering and rendering sonar data and I have been curious what has been done to parallelize GRASS buy enterprising people. Grass isn't threaded so it might work well on a cluster if you can keep the nodes fed with data. Maybe some Franken-script could divide the raster data into segments and launch a node on each segment. I read a paper recently that documented using GRASS on GeoWalls via a series of shell scripts that controlled the batch processing, but I haven't looked into it in detail and can't remember the authors off the top of my head. Our data sets are toping 100 million points a piece and I am looking for ways to divide and conquer the work load. Currently, I use GMT across 8 nodes and it works well (if a bit clunky as far as job management goes). David On 7/19/07, Glynn Clements <[EMAIL PROTECTED]> wrote:
"Sören Gebbert" wrote: > interesting idea. But lets put this idea into the library of grass. :) > > I would like to suggest to extent the grass lib to > read the entire map into the memory if needed. There's no advantage if you're doing sequential I/O, which is most of GRASS. Programs which do random I/O already have mechanisms to deal with this; e.g. r.proj reads the entire useful region into memory, r.proj.seg has its own tile cache, other programs (e.g. r.cost) use the segment library. If you are concerned about efficiency, build GRASS with profiling support, and look at where the actual inefficiency lies. It would help to do the same for 4.x, as that is reported to be an order of magnitude faster than 5.x/6.x for some tasks. -- Glynn Clements <[EMAIL PROTECTED]> _______________________________________________ grass-dev mailing list [email protected] http://grass.itc.it/mailman/listinfo/grass-dev
-- David Finlayson, Ph.D. Operational Geologist U.S. Geological Survey Pacific Science Center 400 Natural Bridges Drive Santa Cruz, CA 95060, USA Tel: 831-427-4757, Fax: 831-427-4748, E-mail: [EMAIL PROTECTED] _______________________________________________ grass-dev mailing list [email protected] http://grass.itc.it/mailman/listinfo/grass-dev
