Hi,
J.P. Delport wrote:
Hi Luc,
Luc Claustres wrote:
J.P. Delport wrote:
"The vpbmaster command created 473 tasks and after 50 hours of
processing created 1.5 million files with a total size of 487GB."
The input data is around 800GB if I remember OK.
So the 60 days does sound a bit extreme.
In our case, VPB created about 130 000 tasks...
How many does it create if you do not specify the split (like you had on
the command line) explicitly?
just another thought. Since the tasks are independent I think this many
tasks would reread many times from the same file (unless gdal does some
magic on its own), especially if your source files cover a large area.
You can monitor this during the build with e.g. iostat on Linux. Iow you
might have 500GB input data, but the build might read 5TB from the disk.
jp
--
This message is subject to the CSIR's copyright terms and conditions, e-mail legal notice, and implemented Open Document Format (ODF) standard.
The full disclaimer details can be found at http://www.csir.co.za/disclaimer.html.
This message has been scanned for viruses and dangerous content by MailScanner,
and is believed to be clean. MailScanner thanks Transtec Computers for their support.
_______________________________________________
osg-users mailing list
[email protected]
http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
