Hi, Over the weekend I discovered some data preparation steps that have resulted in a dramatic improvement in the performance of my VPB generation runs and thought I should share. While I haven't done a proper benchmark, it appears that the changes I made to the source data have cut the processing time for one of my datasets from around 18 days to just 1.5 days.
The original source data was several thousand DTED tiles, 12 large GeoTiffs (about 1 gig each) and hundreds of small GeoTiffs (10s-100s of megs each). Before sending the data to VPB, I used Global Mapper (not free but it is cheap for what you get) to convert all the color data into one ecw file per layer (3 total). I also converted the DTED tiles into a single compressed .img (edras) file and generated overviews in this file using GDAL. I tried creating a JPEG2000 elevation file but this crashed on each attempt. After this, the source data went from 1000s of files to just 4 and the size on disk was reduced by roughly one order of magnitude. All this processing took (a vague guess of) 24 hours. Given that the VPB process has been reduced by many times this it appears to be a relatively small overhead and makes the data a lot easier to deal with. I think the main improvements in speed come from ECWs brilliant performance in generating arbitrary overviews of the color data as well as a clear savings in disk overhead. The only downside with using the ecw format is the slightly painful liscing of the ECW sdk which meant that I had to rebuild GDAL with ECW support rather than using the gdal binaries available from the 3rdParty libs on the OSG wiki. It is however possible to download pre-built gdal binaries (which you can link OSG against) with ecw support, though I am not convinced they meet the license requirements of ECW. If anyone else gives this a shot I would be very interested to see if the gains I have found can be repeated. Cheers, Brad DISCLAIMER:--------------------------------------------------------------------------- This e-mail transmission and any documents, files and previous e-mail messages attached to it are private and confidential. They may contain proprietary or copyright material or information that is subject to legal professional privilege. They are for the use of the intended recipient only. Any unauthorised viewing, use, disclosure, copying, alteration, storage or distribution of, or reliance on, this message is strictly prohibited. No part may be reproduced, adapted or transmitted without the written permission of the owner. If you have received this transmission in error, or are not an authorised recipient, please immediately notify the sender by return email, delete this message and all copies from your e-mail system, and destroy any printed copies. Receipt by anyone other than the intended recipient should not be deemed a waiver of any privilege or protection. Thales Australia does not warrant or represent that this e-mail or any documents, files and previous e-mail messages attached are error or virus free. -------------------------------------------------------------------------------------- _______________________________________________ osg-users mailing list osg-users@lists.openscenegraph.org http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
