On Mon, Jan 26, 2015 at 9:30 AM, Markus Metz <[email protected]> wrote: > On Sun, Jan 25, 2015 at 6:11 PM, Fábio Dias <[email protected]> wrote: >> Hi, >> >> Running r64249, with a couple of stuff in parallel using &. It seems >> to be considerably slower. > > Very strange. Are you using trunk or GRASS 7.0?
Here, v.generalize on a TerraClass tile is down from 25 minutes to 13 seconds. > >> More than 100h, no 1% printed. To be fair, >> I'm not entirely sure I'll see it when it prints, 10 v.generalize >> running (5 for each year) + 1 v.in.ogr for 2012. That v.in.ogr is >> running for almost 100h too. I'm loading the shps directly, as advised >> way, way back in this thread. > > What exactly do you mean with "loading shps directly"? For > v.generalize, you should import them with v.in.ogr. > > What about memory consumption on your system? With 10 v.generalize + 1 > v.in.ogr on such a big dataset, quite a lot of memory would be used. > > Markus M > >> >> AFAIK, no disk is been used, the whole thing is cached (after more >> than 24h processing, cumulative iotop shows only a few mb >> written/read). I'm no longer using a ramdisk for the grassdata dir. >> >> However, it appears to be considerably slower, probably because of the >> parallel running jobs. >> >> My question then would be, considering the thread I saw about sqlite, >> should I be using something else as backend? When it starts to make >> sense to change it? >> >> F >> >> -=--=-=- >> Fábio Augusto Salve Dias >> ICMC - USP >> http://sites.google.com/site/fabiodias/ >> >> >> On Wed, Jan 14, 2015 at 1:06 PM, Markus Neteler <[email protected]> wrote: >>> On Wed, Jan 14, 2015 at 3:54 PM, Fábio Dias <[email protected]> wrote: >>> ... >>>> What would be the best way to do that in parallel? One mapset for each >>>> year? Can I run multiple v.generalizes on the same input with >>>> different outputs? >>> >>> Yes sure. >>> >>>> My first thought was to run completely separated grass processes for >>>> each simplification, but I didn't find a way to make it search >>>> something different than .grass / .grass70 for the configuration >>>> stuff.... >>> >>> Maybe take a look at this approach >>> http://grasswiki.osgeo.org/wiki/Parallel_GRASS_jobs#Grid_Engine >>> >>> but even sending different v.generalize jobs to background (&) should >>> work if you have enough RAM. >>> >>> markusN _______________________________________________ grass-user mailing list [email protected] http://lists.osgeo.org/mailman/listinfo/grass-user
