Re: [mapserver-users] Raster performance hints
I’ve had excellent results using jpeg YCBR in geotiff compression. It make small files, supports overviews and is fast. Building a custom overview layer is a good idea and save a lot in response time over opening a large number if files. -Steve W Sent from my iPhone > On Nov 6, 2018, at 1:24 PM, Ziegler Stefan wrote: > > Hi > > We are using MapServer for vector data stored in a PostgreSQL/PostGIS > database. For a new project we are planning to provide a WMS based on raster > layers and therefore in need of some performance hints or your experiences. > > The raw data will be a lot of single GeoTIFF (aerials, hillshading, slopes, > etc.) files, like 5'000 - 10'000 from different organizations. Since the > originate from different organizations it's possible that they can have > different resolutions. Conceptionally for aerials it would look like this: > > Aerial group > |_ aerial from organization one (1000 single tiffs, resolution 10cm, approx > 10k x 10k pixels) > |_ aerial from organization two (600 single tiffs, resolution, 12.5 cm, > approx 10k x 10k pixels) > |_ aerial from organization three (3000 single tiffs, resolution, 10 cm, > approx 10k x 10k pixels) > > For some hundred single GeoTIFFs we have good experience with the Tileindex > approach and an handmade external overview that kicks in at smaller scale. > > Will this approach still work with a massive amount of single tiffs? Or are > the better ones? What about the compression (quality vs. decompression speed > vs. disk space?) Or would you expect a much better performance with - let's > say - a BigTIFF for each organisation? > > Thanks for any hints. > > Best regards > Stefan > > Freundliche Grüsse > > Stefan Ziegler > Kantonsgeometer / Leiter Amt für Geoinformation > > Amt für Geoinformation > Rötistrasse 4 > 4500 Solothurn > > Telefon +41 32 627 75 96 > stefan.zieg...@bd.so.ch > http://www.so.ch > > > ___ > mapserver-users mailing list > mapserver-users@lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/mapserver-users ___ mapserver-users mailing list mapserver-users@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/mapserver-users
Re: [mapserver-users] Raster performance hints
What you describe is pretty much the approach we take. If you are using geotiffs ensure that they are internally tiles (-co TILED=YES) and the you are creating a shptree index on the tile index as noted in the docs. At some point you may want to consider creating a lower resolution mosaics of coverages and using GROUPS and MIN/MAXSCALEDENOM to load the layers at different scales. Make sure to declare the layer's extent as well Something like: LAYER NAME "full_res_10cm_tiles_withovr" EXTENT 607500.00 4824000.00 654000.00 4859500.00 TYPE RASTER PROCESSING "RESAMPLE=AVERAGE" TILEINDEX /maps/tindex.shp TILEITEM location GROUP "my_photo" MAXSCALEDENOM 1 OFFSITE 0 0 0 METADATA "wms_extent" "607500.00 4824000.00 654000.00 4859500.00" END PROJECTION "init=epsg:2958" END END LAYER NAME "resampled_2m_tiled_bigtif_withovr" EXTENT 607500.00 4824000.00 654000.00 4859500.00 TYPE RASTER PROCESSING "RESAMPLE=AVERAGE" DATA /maps/2m_overall.tif GROUP "my_photo" MINSCALEDENOM 1 MAXSCALEDENOM 65000 OFFSITE 0 0 0 METADATA "wms_extent" "607500.00 4824000.00 654000.00 4859500.00" END PROJECTION "init=epsg:2958" END END On Tue, 6 Nov 2018 at 13:39, Ziegler Stefan wrote: > Hi > > We are using MapServer for vector data stored in a PostgreSQL/PostGIS > database. For a new project we are planning to provide a WMS based on > raster layers and therefore in need of some performance hints or your > experiences. > > The raw data will be a lot of single GeoTIFF (aerials, hillshading, > slopes, etc.) files, like 5'000 - 10'000 from different organizations. > Since the originate from different organizations it's possible that they > can have different resolutions. Conceptionally for aerials it would look > like this: > > Aerial group > |_ aerial from organization one (1000 single tiffs, resolution 10cm, > approx 10k x 10k pixels) > |_ aerial from organization two (600 single tiffs, resolution, 12.5 cm, > approx 10k x 10k pixels) > |_ aerial from organization three (3000 single tiffs, resolution, 10 cm, > approx 10k x 10k pixels) > > For some hundred single GeoTIFFs we have good experience with the > Tileindex approach and an handmade external overview that kicks in at > smaller scale. > > Will this approach still work with a massive amount of single tiffs? Or > are the better ones? What about the compression (quality vs. decompression > speed vs. disk space?) Or would you expect a much better performance with - > let's say - a BigTIFF for each organisation? > > Thanks for any hints. > > Best regards > Stefan > > Freundliche Grüsse > > Stefan Ziegler > Kantonsgeometer / Leiter Amt für Geoinformation > > Amt für Geoinformation > Rötistrasse 4 > 4500 Solothurn > > Telefon +41 32 627 75 96 > stefan.zieg...@bd.so.ch > http://www.so.ch > > > ___ > mapserver-users mailing list > mapserver-users@lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/mapserver-users ___ mapserver-users mailing list mapserver-users@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/mapserver-users
[mapserver-users] Raster performance hints
Hi We are using MapServer for vector data stored in a PostgreSQL/PostGIS database. For a new project we are planning to provide a WMS based on raster layers and therefore in need of some performance hints or your experiences. The raw data will be a lot of single GeoTIFF (aerials, hillshading, slopes, etc.) files, like 5'000 - 10'000 from different organizations. Since the originate from different organizations it's possible that they can have different resolutions. Conceptionally for aerials it would look like this: Aerial group |_ aerial from organization one (1000 single tiffs, resolution 10cm, approx 10k x 10k pixels) |_ aerial from organization two (600 single tiffs, resolution, 12.5 cm, approx 10k x 10k pixels) |_ aerial from organization three (3000 single tiffs, resolution, 10 cm, approx 10k x 10k pixels) For some hundred single GeoTIFFs we have good experience with the Tileindex approach and an handmade external overview that kicks in at smaller scale. Will this approach still work with a massive amount of single tiffs? Or are the better ones? What about the compression (quality vs. decompression speed vs. disk space?) Or would you expect a much better performance with - let's say - a BigTIFF for each organisation? Thanks for any hints. Best regards Stefan Freundliche Grüsse Stefan Ziegler Kantonsgeometer / Leiter Amt für Geoinformation Amt für Geoinformation Rötistrasse 4 4500 Solothurn Telefon +41 32 627 75 96 stefan.zieg...@bd.so.ch http://www.so.ch ___ mapserver-users mailing list mapserver-users@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/mapserver-users
