Re: [gdal-dev] [EXTERNAL] [BULK] Re: Experience with slowness of free() on Windows with lots of allocations?

2024-03-29 Thread Javier Jimenez Shaw via gdal-dev
for tcmalloc do you need master? this recent release seems to have CMake https://github.com/gperftools/gperftools/releases/tag/gperftools-2.15 Of course, I do not mean to force the usage of it. But could be a suggestion in case we do not find anything better and a user has problems. Or a way to

Re: [gdal-dev] [EXTERNAL] [BULK] Re: Experience with slowness of free() on Windows with lots of allocations?

2024-03-21 Thread Meyer, Jesse R. (GSFC-618.0)[SCIENCE SYSTEMS AND APPLICATIONS INC] via gdal-dev
I’ve used mimalloc successfully in the past, worth a look if a drop in replacement for new / delete / malloc / free is desirable. Do note that its performance is usually uniformly superior to glibc / msvc but there are unintuitive performance cliffs. Given the block nature of most gdal raster

Re: [gdal-dev] [EXTERNAL] [BULK] Re: Experience with slowness of free() on Windows with lots of allocations?

2024-03-21 Thread Even Rouault via gdal-dev
I've played with VirtualAlloc(NULL, SINGLE_ALLOC_SIZE, MEM_COMMIT | MEM_RESERVE, PAGE_READWRITE), and it does avoid the performance issue. However I see that VitualAlloc() allocates by chunks of 64 kB, so depending on the size of a block, it might cause significant waste of RAM, so that can't

Re: [gdal-dev] [EXTERNAL] [BULK] Re: Experience with slowness of free() on Windows with lots of allocations?

2024-03-21 Thread Meyer, Jesse R. (GSFC-618.0)[SCIENCE SYSTEMS AND APPLICATIONS INC] via gdal-dev
+1. We use a variety of hand-rolled VirtualAlloc based (for basic tasks, a simple pointer bump, and for more elaborate needs, a ‘buddy’) allocators, some of which try to be smart about memory usage via de-committing regions. In our work, we tend to disable the GDAL cache entirely and rely on