Hi Laura,
Laura Toma wrote: > Hi Markus, > > How much memory was available on the machine? 8 GB > If the machine had more than 512MB RAM, it is not fair to run > terracost running with mem=400MB, and compare it with an algorithm > that can use more memory. I don't understand, both modules can use more than 400MB of memory. I set r.terracost to use 400MB max and r.cost to use 135MB max. If anything, this is not fair for r.cost and gives an advantage to r.terracost. I did not mention that I gave r.terracost another advantage by assigning the temporary directories to a folder on a separate, very fast hard drive that had nothing else to do but manage the temp files of r.terracost. The temp files of r.cost are in the standard grass .tmp/$HOST directory, in my case that (slower) hard drive also had other things to do than just manage r.cost's temp files. I really tried to give r.terracost a head start ;-) > > However, I am surprised that withnumtiles=1, it was slower than > r.cost. ??? cut'n paste from below r.terracost numtiles=1 real 0m17.969s user 0m17.276s sys 0m0.500s r.cost in grass7, percent_memory=20% # that's not percent of available memory, that's percent of the region to keep in memory, for the test region this was max 135 MB real 0m55.349s user 0m53.360s sys 0m1.797s -> r.terracost with numtiles=1 is much faster than r.cost > That's something I'd like to look into. Would you mind sharing the > raster with me, and sending me the exact commands that you ran? Is in the message you replied to, see below. cut'n paste from below North Carolina sample dataset g.region rast=elev_state_5...@permanent res=100 # gives about 28 million cells v.to.rast nc_st...@permanent use=val val=500.0 out=cost --o v.to.rast urbana...@permanent use=val val=1 out=urbanarea --o r.cost in=cost start_rast=urbanarea out=dist_urban percent_memory=20 --o r.terracost in=cost start_rast=urbanarea out=dist_urban_terracost -i gives [...] TILESIZE: nc_spm_08 N=28064550 elements, M=419430400 bytes, optimal numtiles=1870 # test command for r.terracost, same as above, now with optimal numtiles=1870 r.terracost in=cost start_rast=urbanarea out=dist_urban_terracost numtiles=1870 > > A grid with 29M points is pretty small, for today's machines. I > suggest running on something 10 times larger. Will do. > And use a lot of sources, that makes the data access pattern less > local. Yes, I thought about that too, best would be a lot of evenly distributed start points, that should force r.cost to do highly random and scattered disk access. My theory however predicts that the current r.cost will always be faster than the current r.terracost if r.terracost is run with numtiles>1. We will see. BTW, I took the liberty to fix r.terracost, it works now with numtiles>1. See changelog for r39684 https://trac.osgeo.org/grass/changeset/39684 Markus M > > -Laura > > > >> ---------------------------------------------------------------------- >> >> Message: 1 >> Date: Fri, 06 Nov 2009 09:50:02 +0100 >> From: Markus Metz <[email protected]> >> Subject: [GRASS-dev] comparing r.cost and r.terracost >> To: GRASS developers list <[email protected]> >> Message-ID: <[email protected]> >> Content-Type: text/plain; charset=ISO-8859-1; format=flowed >> >> Same test region as before >> >> North Carolina sample dataset >> >> g.region rast=elev_state_5...@permanent res=100 >> # gives about 28 million cells >> v.to.rast nc_st...@permanent use=val val=500.0 out=cost --o >> v.to.rast urbana...@permanent use=val val=1 out=urbanarea --o >> r.cost in=cost start_rast=urbanarea out=dist_urban percent_memory=20 --o >> >> grass7 r.cost >> real 0m55.349s >> user 0m53.360s >> sys 0m1.797s >> >> grass65 r.cost >> real 26m35.166s >> user 2m55.612s >> sys 23m37.921s >> >> r.terracost: check optimal tile size for 400MB memory (default setting; >> r.cost in grass7 used 135MB with 20% of maps in memory) >> r.terracost in=cost start_rast=urbanarea out=dist_urban_terracost -i >> [...] >> TILESIZE: nc_spm_08 N=28064550 elements, M=419430400 bytes, optimal >> numtiles=1870 >> >> r.terracost numtiles=1870, intermediate data are stored on disk as >> r.cost does >> real 25m13.593s >> user 22m46.978s >> sys 1m8.059s >> >> r.terracost numtiles=1, all in memory (just fits into 400MB) >> real 0m17.969s >> user 0m17.276s >> sys 0m0.500s >> >> According to Laura Toma, when comparing r.cost with r.terracost, >> numtiles must be >1 for r.terracost in order to compare disk I/O >> algorithms [1] >> >> With these test settings that intentionally reduced memory consumption >> in order to test disk I/O performance, r.terracost is not really faster >> than r.cost in grass65 and much slower than r.cost in grass7. >> >> Markus M >> >> [1] http://lists.osgeo.org/pipermail/grass-dev/2009-July/045157.html >> >> >> >> >> ------------------------------ >> >> _______________________________________________ >> grass-dev mailing list >> [email protected] >> http://lists.osgeo.org/mailman/listinfo/grass-dev >> >> End of grass-dev Digest, Vol 43, Issue 8 >> **************************************** > > _______________________________________________ > grass-dev mailing list > [email protected] > http://lists.osgeo.org/mailman/listinfo/grass-dev _______________________________________________ grass-dev mailing list [email protected] http://lists.osgeo.org/mailman/listinfo/grass-dev
