Jose, I'm sorry, but I don't think this will work trivially.
If you write a small part of an image, GDAL is likely to read a larger area, update that and write it back. If other programs happen to try and update the same larger area at the same time some data is likely to be lost. If you are careful to write on tile/block boundaries, and use an output file that preinitializes everything then it might work but it is likely to be somewhat fragile. Best regards, Frank On Mon, May 20, 2013 at 3:46 PM, Jose Gomez-Dans <[email protected]>wrote: > Hi, > I have a rather demanding processing scheme (read data from files, and > process pixel per pixel). It's trivial to parallelise (just do pixels in > different machines/cores), and I intend to run this on several machines on > our cluster. > > I'm unsure whether I can use GDAL (python bindings) like this. My idea > would be to first create the output files. After this, the scene is split > into several windows, and different works start processing the different > windows. Once a worker is done, it gets a handle to the created file and > writes a chunk to it, closes the handle an exits. > > Questions are: does this "create and fill" make sense? Do I need to lock > the output files, or can I just write to the file handles (the windows > don't overlap)? Is this format dependent? Am I missing something? I'd > rather avoid creating lots of intermediate files to then stitch up. > > Thanks! > Jose > > _______________________________________________ > gdal-dev mailing list > [email protected] > http://lists.osgeo.org/mailman/listinfo/gdal-dev > -- ---------------------------------------+-------------------------------------- I set the clouds in motion - turn up | Frank Warmerdam, [email protected] light and sound - activate the windows | http://pobox.com/~warmerdam and watch the world go round - Rush | Geospatial Software Developer
_______________________________________________ gdal-dev mailing list [email protected] http://lists.osgeo.org/mailman/listinfo/gdal-dev
