Angus,
there has been a recent extra fix that landed in GDAL 3.6.2 that might
possibly help: https://github.com/OSGeo/gdal/pull/6926
Even
Le 25/01/2023 à 01:36, Angus Dickey a écrit :
Hi all,
I am running into an issue where GDAL is overestimating the amount of
physical memory it has leading to it locking up the OS by taking 100%
of the memory. Here is an example program that illustrates the issue:
#include <stdio.h>
#include "gdal.h"
int main(void) {
printf("GDAL version is %s\n", GDALVersionInfo("RELEASE_NAME"));
printf("GDAL thinks is has %lld bytes of physical memory\n",
CPLGetPhysicalRAM());
printf("GDAL thinks it has %lld bytes of usable physical memory\n",
CPLGetUsablePhysicalRAM());
return 0;
}
When this is compiled with GDAL 3.5.1 on Ubuntu 22.04 we get:
$ ./get_gdal_memory
GDAL version is 3.5.1
GDAL thinks is has 811526475776 bytes of physical memory
GDAL thinks it has 811526475776 bytes of usable physical memory
Which is not consistent with the actual available memory:
$ free -h
total used free shared buff/cache
available
Mem: 2.0Gi 148Mi 1.2Gi 0.0Ki 639Mi 1.8Gi
Swap: 256Mi 0B 256Mi
So GDAL thinks it has 755GB of memory but it only has 2GB, this causes
issues with the raster read cache and maybe elsewhere. I suspect this
is happening because it is running in a Linux container
<https://linuxcontainers.org/> and GDAL is getting the total physical
memory of the host, not the container. The strange thing is Linux
containers use cgroups for memory restrictions and the API docs
mention it was fixed in GDAL 2.4.0
<https://gdal.org/api/cpl.html#_CPPv417CPLGetPhysicalRAMv> but I am
still seeing the issue in 3.5.1.
Any help or insight would be appreciated; I am happy to provide any
additional information or testing.
Thanks,
Angus
_______________________________________________
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
--
http://www.spatialys.com
My software is free, but my time generally not.
_______________________________________________
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
