huh - processing using Make - clever. We are researching several different approaches for this also but I imagine we will use Conductor, created by (UofA). Another of our commercial systems uses Condor (open source - http://www.cs.wisc.edu/condor/ ) .
Currently the UofA Conductor HiRISE pipeline uses our ISIS image processing software which is not yet multi-threaded. Thus this system should also work well for GDAL processing. They also have another tool that monitors Conductor called Maestro. Don't jump into this lightly as I imagine it will be tricky to setup (it also requires a database - MySQL). http://pirlwww.lpl.arizona.edu/software/Conductor.shtml BTW, they are a completely *nix house (Solaris, Mac OS-X, FreeBSD and Linux). here is an abstract (by the author B. Castalia) http://www.lsstmail.org/meet_coll/ongoing/Software/ Regards, Trent From: Fabrice LALLAURET - PSV <[email protected]> To: [email protected] Date: 06/16/2009 08:32 AM Subject: Re: [gdal-dev] Distributed processing Sent by: [email protected] John Donovan a écrit : > Hi, > We currently have a stand-alone app that converts, mosaicks and scales > GDAL-supported images to a proprietary format. It works well, but we > handle tens of thousands of source files at a time, which can slow the > process down to a crawl. > > So we're investigating parallelising this process over several machines, > and I was wondering if anyone has any experience of this that they'd be > willing to share? It's still early days yet, so we're open to all > suggestions. > > Regards, > John Donovan - Programmer, Virtalis Ltd. > > ______________________________________________________________________ > This email has been scanned for Virtalis Ltd by the MessageLabs Email Security System. ______________________________________________________________________ > _______________________________________________ > gdal-dev mailing list > [email protected] > http://lists.osgeo.org/mailman/listinfo/gdal-dev Personally, I create Makefile and use distmake for the distribution purpose. Regards _______________________________________________ gdal-dev mailing list [email protected] http://lists.osgeo.org/mailman/listinfo/gdal-dev
_______________________________________________ gdal-dev mailing list [email protected] http://lists.osgeo.org/mailman/listinfo/gdal-dev
