Hi I use mapserver which is using gdal for raster from local files. mapserv is running as processes under fcgi module from apache2 My use case is raster netcdf data in a custom reprojection stored locally with a week of data every 5 minutes, with new data every 5 minute. Mapserver, using the raster gdal, reads the data, resample and return the data. This works fine.
If I test with requesting the same dataset again and again for hundreds of times, I see little or no memory usage increase of the mapserv process. However, my setup is datasets with many timestamps. One timestamp per netcdf file. If I request each dataset once for each timestamp, I see a steady increase in memory usage of the mapserv process ( still running under fcgi module and apache2). I'm guessing this as gdal is caching data. So where does all this memory usage come from? I have run valgrind on the mapserv process as a standalone processi. No leaks. I have run valgrind on apache2 running the fcgi module running the mapserv as a long living process. No leaks as far as I can see. I suspect the gdal block cache mechanism. There is an environmental variable GDAL_CACHEMAX, which should control the GDAL CACHE usage? However I have tried to set GDAL_CACHEMAX to various values, even to 0 ( and verified the variable is read with logs output like: [Thu Feb 26 07:05:34.677597 2026] [fcgid:warn] [pid 421231:tid 421234] [client ::1:60610] mod_fcgid: stderr: msDrawRasterLayerGDAL(): [Thu Feb 26 07:05:34 2026].6776, 0.0302: GDAL: GDALDefaultOverviews::OverviewScan() [Thu Feb 26 07:05:34 2026].6777, 0.0303: LOCK: Lock contention : max = 510, avg = 510 [Thu Feb 26 07:05:34 2026].6778, 0.0303: GDAL: GDAL_CACHEMAX = 0 MB [Thu Feb 26 07:05:34 2026].6778, 0.0303: LOCK: Lock contention : max = 6882, avg = 3696 But still, the memory usage just increases, in what I see, in an uncontrolled manner. This leads to eventually OOM kills of the mapserv process. I can of course set the fcgi environmental variable FcgidMaxRequestsPerProcess to a low value, but that will require the fcgi module to respawn new mapserv processes all the time. The data is stored locally, so no vsi is involved as far as I can see. It must be a way to control the memory usage in gdal in a better way ( if this really is gdal causing this) Any ideas? This test has been set up using mapserver 8.6, gdal 3.12.2 on a ubuntu noble system. -- Trygve Aspenes - Senioringeniør/Senior Engineer Telefon/Phone: +47 48223736 Observasjonskvalitet og databehandling (OKD) Observasjonsdivisjonen/Department of observation Meteorologisk Institutt /MET Norway Postboks 43, Blindern 0313 OSLO, NORWAY
_______________________________________________ gdal-dev mailing list [email protected] https://lists.osgeo.org/mailman/listinfo/gdal-dev
