On Thu, May 25, 2017 at 5:24 PM, Giuseppe Amatulli < [email protected]> wrote: > > Hi Markus, > in reality the computation beyond the mask problem is very complex, I will try to summarize. > I have the UNIT3753 mask sit on the PERMANENT and I create 1000 location in a multicore environment (HPC). > Each location use the UNIT3753-mask from the PERMANENT. This action produce that the cell_misc/UNIT3753/reclassed_to is written simultaneously from different cores.
This is the problem. If several processes try to edit the same file at once, this can cause data corruption. The reclassed_to file holds the names of the reclassed raster maps based on that raster map. Instead of r.mask, you can use r.mapcalc to create specific MASKs for each process on each compute node. This should avoid these problems, as long as each process on each compute node uses its own unique mapset. Markus M (running GRASS on an HPC system every day) > I think that this action creates a reclassed_to file very large and probably also with different permission which was not be able to be removed. > > I solved by manually remove the reclassed_to file, so now the process is running smoothly. > > A potential solution would be: > 1) for each location g.copy raster=PERMANENT,UNIT3753 , and than apply the r.mask using the UNIT3753 in the location and not in the PERMANENT. > This would work but data duplication will be there. > 2) a second solution would be if the reclassed_to can have a unique name (e.g. reclassed_to$$) associate to the location. > > Any thought? > Thank you > Best Regards > Giuseppe > p.s. in few weeks I will be in Berlin so we may chat in person. > > > > > > > On 24 May 2017 at 18:29, Markus Neteler <[email protected]> wrote: >> >> On Wed, May 24, 2017 at 11:40 PM, Giuseppe Amatulli >> <[email protected]> wrote: >> > Hi Nikos >> > I just sort out. >> > the file /PERMANENT/cell_misc/UNIT3753/reclassed_to was very big (2.5g) and >> > it was not be able to be removed with the r.remove. >> >> That size is not an issue at all (the EU DEM is 25GB or more for >> example)... the problem must be elsewhere. Happy to help if possible. >> BTW: I have compiled GRASS 7.2 also for RHEL via EPEL, see the download page. >> >> Markus >> >> >> -- >> Markus Neteler, PhD >> http://www.mundialis.de - free data with free software >> http://grass.osgeo.org >> http://courses.neteler.org/blog > > > > > -- > Giuseppe Amatulli, Ph.D. > > Research scientist at > Yale School of Forestry & Environmental Studies > Yale Center for Research Computing > Center for Science and Social Science Information > New Haven, 06511 > Teaching: http://spatial-ecology.org > Work: https://environment.yale.edu/profile/giuseppe-amatulli/ > > _______________________________________________ > grass-user mailing list > [email protected] > https://lists.osgeo.org/mailman/listinfo/grass-user
_______________________________________________ grass-user mailing list [email protected] https://lists.osgeo.org/mailman/listinfo/grass-user
