On Sat, 2008-10-25 at 21:24 +0100, Jon Burgess wrote:
> On Fri, 2008-10-24 at 12:25 -0700, Dane Springmeyer wrote:
> > So, my attempts continue to get mapnik rendering the benchmarking
> > datasets used in the Geoserver vs. Mapserver tests
> > (http://blog.geoserver.org/2008/10/22/geoserver-benchmarks-at-foss4g-2008/
> > ).
> >
> > In trying to render a 200MB gdal layer in Mr Sid format I get the
> > following error with mapnik build in DEBUG:
> >
> > size = 2
> > file=S-34-30_2000/S-34-30_2000.sid
> > type=gdal
> > datasource=0x36d130 type=1
> > name=gdal encoding=utf-8
> > layer1 - > Envelope(160576.125,-3875914.625,839403.375,-3317784.875)
> > layer2 - > Envelope(160576.125,-3875914.625,839403.375,-3317784.875)
> > scale=1395.32
> > start map processing
> > bbox=Envelope(81392.4375,-3875914.625,918587.0625,-3317784.875)
> > scale denominator = 5.54739e+11
> > start layer processing : S-34-30_2000
> > datasource = 0x36d130
> > Block=1024x128 Type=Byte Color=Red
> > Block=1024x128 Type=Byte Color=Green
> > Block=1024x128 Type=Byte Color=Blue
> > Python(36546) malloc: *** mmap(size=3168227328) failed (error code=12)
> > *** error: can't allocate region
> > *** set a breakpoint in malloc_error_break to debug
> > 0.01 s
> > MemoryError
> > close connection 0x895200
> >
> > I think other users have run into memory errors like this when using a
> > gdal layer, but I've not seen any workarounds.
> >
> > Short of doing malloc debugging like the error hints, I may trying
> > converting to tiff.
> >
> > Anyone else have suggestions?
>
> I don't see any crashes, but I don't get anything being rendered either.
> What are you using as your test script & style?
It seems I have been getting a memory error but it was just being
reported as 'failed to render image' by the nik2img.py script. It became
visible after I disabled the try/except blocks.
I've added some extra debug statements and it isn't surprising that
there is an error. It fails when trying to allocate a 32bit buffer for
the whole 47637 x 39167 pixels of the SID file:
scale=1860.43
start map processing
bbox=Envelope(127903.25,-3875914.625,872076.25,-3317784.875)
scale denominator = 7.39651e+11
start layer processing : mrsid
datasource = 0x120e540
Block=1024x128 Type=Byte Color=Red
Block=1024x128 Type=Byte Color=Green
Block=1024x128 Type=Byte Color=Blue
GDAL gdal_featureset::next(): Raster Image(47637,39167)
GDAL gdal_featureset::next(): Raster
Envelope(160576.125,-3875914.625,839403.375,-3317784.875)
GDAL gdal_featureset::next(): Query
Envelope(160576.125,-3875914.625,839403.375,-3317784.875)
GDAL gdal_featureset::next(): Raster Extent(678827,558130)
GDAL gdal_featureset::next(): Query Extent(678827,558130)
GDAL gdal_featureset::next(): Creating image(47637,39167)
0.03 s
Traceback (most recent call last):
File "./nik2img.py", line 571, in <module>
render(mapnik_map,o,FORMAT)
File "./nik2img.py", line 188, in render
mapnik.render_to_file(*args)
MemoryError
I'm not certain at this point whether this is a bug in the test setup or
a fundamental flaw in the Mapnik GDAL handling. I guess the approach of
turning the source file into a bitmap works fine for normal sized images
but falls down when faced with such an enormous raw bitmap.
Mapnik would be much better off it could extract lower resolution image
from the multi-resolution file. I don't know if GDAL exports this aspect
of the file through the API. If not, it might need a special MrSID input
plugin in Mapnik.
Jon
_______________________________________________
Mapnik-users mailing list
[email protected]
https://lists.berlios.de/mailman/listinfo/mapnik-users
