I support Frank's suggestion, and note that it is my understanding
that when the user does not specify a value for --config
GDAL_CACHEMAX, then the default of (only) 40 MB is used.
In January 2009, working with GDAL version 1.6.0, I did some testing
and documenting of the effect of GDAL_CACHEMAX. Using as input an
uncompressed GeoTif 15,000 x 15,000 x 3 = 675 MB, organized into 1024
x 1024 tiles, I found that using gdal_translate to convert this
GeoTif to a GeoJP2 with the default --config GDAL_CACHEMAX 40 took
10473 sec (= 175 minutes), but after increasing GDAL_CACHEMAX by only
12.5% to --config GDAL_CACHEMAX 45 the processing took only 166 sec
(= 2.7 minutes), which is 63 times faster!
Greg
GDAL 1.6.0, released 2008/12/04
Input GeoTif 15,000 x 15,000 x 3 = 675 MB
gdal_translate -of JP2KAK -co QUALITY=10 --config GDAL_CACHEMAX
0040 10473.14 sec
0041 10326.75 sec
0042 10492.85 sec
0043 7081.57 sec
0044 3503.33 sec
0045 166.23 sec
0050 166.31 sec
0055 165.89 sec
0060 165.78 sec
0080 165.48 sec
0100 166.53 sec
0125 165.59 sec
0150 165.72 sec
0200 165.65 sec
On Sep 16, 2009, at 9:42 AM, Frank Warmerdam wrote:
Hermann Peifer wrote:
Hi,
Are there any hints from the experts about how to improve
gdal_rasterize perfomance?
My specific case is to rasterize a 1+GB shapefile into a 26800 x
23200 GeoTiff. Are there some memory settings or GeoTiff creation
options that could speed up the rasterization?
Hermann,
This sounds like a heap of processing that needs to be done.
You can increase the chunk size on which gdal_rasterize operates by
increasing the GDAL_CACHEMAX config option. I would suggest sizing
it to be roughly 1/3 of your machine RAM.
eg.
gdal_rasterize --config GDAL_CACHEMAX 600 ...
would be appropriate on a machine with 2GB of RAM.
Beyond that I can't think of much that might help. The above
basically
reduces the number of passes needed through the vector data. It is
all
processed once for each chunk of the raster that is processed.
Best regards,
_______________________________________________
gdal-dev mailing list
[email protected]
http://lists.osgeo.org/mailman/listinfo/gdal-dev
