We had the exact same type of problem. 1:1 GeoTIFF to layer. We switched to using raster mosaics with a PostGIS mosaic index. We now have 1 store, 1 layer for each layer type.
We also had a ridiculous number of SLD files that were in the GeoServer directory and were therefore being managed by GeoServer. We moved this out of the folder and that also greatly improved performance. Regards, Richard -----Original Message----- From: Bourgault , Pascal via Geoserver-users <geoserver-users@lists.sourceforge.net> Sent: Thursday, April 11, 2024 12:03 PM To: geoserver-users@lists.sourceforge.net Subject: [Geoserver-users] Seeking advice for large amount of layers Hi all, My team is using Geoserver to power a web app that shows maps of projected climate change. We are currently running into issues that I believe are related to the large amount of layers. I was wondering if other users had ideas on how to make it more efficient and stable. Our data is originally in Zarr datasets. It could be converted to netCDF, but we had too many issues with the Rotated Pole projection. As we added the reprojection step, we decided to save everything in GeoTIFFs it seemed easier to handle. The data has many facets : dataset (3), indicator (22), scenario (3), season (5), stat (4), ensemble percentile (5), time period (15). This list is somewhat simplified for the question. All in all, we currently have 50 000 such geotiffs and as many layers in Geoserver. That number is expected to grow a lot in the near future. I see the multi-dimensional support in Geoserver, but in our case that only covers the time period. Is there a format where we could combine some layers together to have fewer stores on Geoserver ? I tried using a GeoPackage with multiple "subdatasets", but Geoserver failed to parse it. We currently provision the server using the REST API. We mount the data in a folder local to the server and make API calls to create each layer/store. On the production server, this currently takes around 2h. It's very slow. Is it possible to pre-generate the catalog offline ? We haven't had performance issue for the WMS service for now, we don't have that many users. Most difficulties arise in the data provisionning step. Thanks! -- Pascal Bourgault Ouranos, Montréal, Qc _______________________________________________ Geoserver-users mailing list Please make sure you read the following two resources before posting to this list: - Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/ - The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer Geoserver-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-users _______________________________________________ Geoserver-users mailing list Please make sure you read the following two resources before posting to this list: - Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/ - The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer Geoserver-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-users
