Hi Seth, more inline: 2012/2/28 Seth Price <[email protected]>: > I'm running a few instances of GRASS, and I've hit an extreme bottleneck. The > problem is that I can't figure out what it is. > > I have a module that I wrote in the GRASS 7.0 environment that should be I/O > bound. It normally takes about 10 minutes to finish running. I have seven > instances running from one disk, and each one is taking on the order of 10 > hours to finish. Why? > > NOTE: > - The processes aren't I/O bound. 'iostat' lists a constant 30 MB/s transfer > rate. It should be able to handle twice that. The transfers per second are < > 100, and I've seen it handle triple that (at 60 MB/s). > - The load average is currently below 1. I have 16 processors, so CPU > shouldn't be a problem. > - I have 16 GB of RAM, and it's full, but not thrashing.
Since you have plenty of RAM, the Linux system might buffer a lot of the module input/output in the main memory. So the bottleneck may appear when Linux tries to write the buffered output to the disc. Does the load value not change in the 10h of processing? Is the transfer rate constant over the 10h? My guess is that when your module reads and writes permanently to the file system it is IO bound and your hard disk is the limiting factor not the GRASS (raster, vector or voxel) IO implementation. I assume the seven instances writing to different maps? For comparison: I have a 24 Core + 64Gig Ram number cruncher, running 24 parallel processes of v.surf.idw for interpolation purpose. HD's are 4x256GB SSD's in a fast raid configuration. Max write speed i have measured is about 400MByte/s with large files. This scales very well without any performance issues with 24 processes which are not IO bound. Best regards Soeren > > If it's not I/O, CPU, or RAM, then what is it slowing GRASS? > Thanks, > Seth_______________________________________________ > grass-user mailing list > [email protected] > http://lists.osgeo.org/mailman/listinfo/grass-user _______________________________________________ grass-user mailing list [email protected] http://lists.osgeo.org/mailman/listinfo/grass-user
