Selon Giuseppe Amatulli <[email protected]>: > Hi all, > I solved the down reported issues. I change -ot Float32 to -ot > Float64 and it works. > Anyway Float32 should be enough to store the max value of > 560459513003129 < 3.4E38 > but probably during the multi-core several values are stored in the > same array and it needs Float64.
I'm extremly skeptical that what you have seen initially is linked to multi-processing and I would have recommanded you to double check your findings that the results with -P 1 or -P 10 were really different. From what I've seen, the -P option of xargs forks separate processes, so I can't imagine how they could interact badly (this is different from having several threads inside a single process). But what happens likely with Float32 is that the precision isn't sufficient to keep all the digits of the integral value. So 5000000000001 and 5000000000002 might be represented as the same Float32 binary value (let's say 5000000000000), hence the impression that they are burnt with the same value. Float64 will give you more precision (and more range, but that's not what you need). > > > ciao ciao > Giuseppe > > On 24 September 2012 17:53, Giuseppe Amatulli > <[email protected]> wrote: > > Hi, > > i'm running gdal_rasterize for several *.shp using xargs, sending > > the process to 10 cpu. > > > > 1) ls /weldgfs/p51/gius_urban/pop_urban/2010/shp_albers/*.shp | > > xargs -n 1 -P 10 bash > > /weldgfs/p51/gius_urban/pop_urban/scripts/sc2a_shp_rasterizeP30_2010.sh > > > > the same process can be run using one processor. > > > > 2) ls /weldgfs/p51/gius_urban/pop_urban/2010/shp_albers/*.shp | > > xargs -n 1 -P 1 bash > > /weldgfs/p51/gius_urban/pop_urban/scripts/sc2a_shp_rasterizeP30_2010.sh > > > > Inputs and outputs are independent in each process so no issues in > > reading/writing the same file. > > > > The obtained tifs produced by the gdal_rasterize > > (gdal_rasterize -a_nodata 0 -ot Float32 -co 'COMPRESS=LZW' -te > > $geo_string_tif -tr 30 30 -l $filename -a BLOCKID10 > > $INDIR/$filename.shp $OUTDIR/$filename"_"$tile".tif" ; > > where BLOCKID10 is an integer number if e.g 550 019 505 012 101 ) > > > > should be same but is not the case. > > Line 2) produce the tif perfect similar with the attribute of the shp; > > so no issues. > > Line 1) burn the same BLOCKID10 value for several polygons, resulting > > in less final polygons. > > Seems to me that this phenomena manly effect BLOCKID10 > > values having several "0" (e.g. 10979900000033 ; 10030101002035) > > > > May question is. Can be possible that the gdal driver used inside C > > code not support multi-core processing? > > If yes it is possible to find a solution to run gdal_rasterize in parallel. > > Any idea? > > > > Thanks in advance > > Giuseppe > > > > -- > > Web: www.spatial-ecology.net > > > > -- > Giuseppe Amatulli > Web: www.spatial-ecology.net > _______________________________________________ > gdal-dev mailing list > [email protected] > http://lists.osgeo.org/mailman/listinfo/gdal-dev > _______________________________________________ gdal-dev mailing list [email protected] http://lists.osgeo.org/mailman/listinfo/gdal-dev
