Re: [GRASS-dev] Fwd: [GRASS-user] Re: grass-user Digest, Vol 30, Issue 22
Markus Metz wrote: > assuming that GRASS runs today on systems where at least > 500MB RAM are available 500MB total, 500MB per user, or 500MB per process? It's safe to assume 500MB for the system (although much of GRASS can run on a PDA, it's reasonable to assume that people won't be performing complex analysis on such systems), but that doesn't mean that a single process can use all of it. > Still, the seg mode is slow and testing would require a lot of patience. GRASS' segement library (which r.watershed.seg uses) is quite inefficient. For the segmented r.proj (r.proj.seg in 6.3/6.4, r.proj in 7.0), I wrote my own tile cache. If it can fit the entire map within the specified amount of RAM, then it will do so (reading the map directly into RAM without creating the segment file), without any noticeable performance impact caused by the extra level of indirection. If you can't fit the working set into RAM, it's going to be slow whichever approach you take. Reading into "memory" which is actually swap isn't going to be any quicker. Also, using a tile cache allows you to handle maps which exceed the size of the address space (i.e. maps larger than 4GiB on a 32-bit system). OTOH, r.proj does have reasonable locality of reference, so the working set tends to be small relative to the total amount of data. I don't know whether the same is true of r.watershed. -- Glynn Clements <[EMAIL PROTECTED]> ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Fwd: [GRASS-user] Re: grass-user Digest, Vol 30, Issue 22
Markus Neteler wrote: > On Thu, Oct 9, 2008 at 11:13 PM, Michael Barton <[EMAIL PROTECTED]> wrote: > >> Below is a recent exchange between Markus (Neteler) and me about the new >> r.watershed.fast. >> The gist is the question: is it ready to go into the develbranch_6 and trunk >> (7) svn or does it need more testing. I thought it was already in the main >> svn, but Markus pointed out that it is still in Addons. >> > > Indeed it is not even in Addons - I tried to point out that Addons would be > a good place to facilitate testing. > > @Markus: if you are interested, please check here: > http://trac.osgeo.org/grass/wiki/HowToContribute#WriteaccesstotheGRASS-Addons-SVNrepository > -> Write access to the GRASS-Addons-SVN repository > Yes, I am interested! And this new version might need some testing, in particular for the seg mode. The original version uses very little memory, so assuming that GRASS runs today on systems where at least 500MB RAM are available I changed the parameters for the seg mode, more data are kept in memory, speeding up the seg mode. Looking at other modules using the segment library (e.g. v.surf.contour, r.cost), it seems that there is not one universally used setting, instead the segment parameters are tuned to each module. The new settings work for me, but not necessarily for others, and maybe using 500MB is a bit much. Still, the seg mode is slow and testing would require a lot of patience. I only tested it for smaller regions, not yet for regions that would require several GB of RAM. The aim is to get close to 2,147,483,647 cells in a region... BTW, has anybody recently used the seg mode of the original version successfully and done so because the non-segmented r.watershed would run out of memory? I'm rather confident about the ram version, but it can do only good if developers review the new code. Will try to get write access to the GRASS-Addons-SVN repository, and add an entry in the GRASS-Addons wiki. Thanks for your feedback! Markus ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Fwd: [GRASS-user] Re: grass-user Digest, Vol 30, Issue 22
On Thu, Oct 9, 2008 at 11:13 PM, Michael Barton <[EMAIL PROTECTED]> wrote: > Below is a recent exchange between Markus (Neteler) and me about the new > r.watershed.fast. > The gist is the question: is it ready to go into the develbranch_6 and trunk > (7) svn or does it need more testing. I thought it was already in the main > svn, but Markus pointed out that it is still in Addons. Indeed it is not even in Addons - I tried to point out that Addons would be a good place to facilitate testing. @Markus: if you are interested, please check here: http://trac.osgeo.org/grass/wiki/HowToContribute#WriteaccesstotheGRASS-Addons-SVNrepository -> Write access to the GRASS-Addons-SVN repository Markus ___ grass-dev mailing list grass-dev@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev