I am measuring memory usage using psutil:
Process: `psutil.Process().memory_info().rss` and
`psutil.Process().memory_percent()`
General: `psutil.virtual_memory().available`
I do tests both on MacOS and linux.
Any idea about how to investigate the file cache or how to use psutil to check
the difference between this file-system cache and other memory usage?
By the way, instead of using a real tempfile but removing it and using `input +
’.tmp’` as output file yields the same behavior.
When calling the gdal calls in a Process, the issues do not appear (3 different
approaches)
```
74 109.5 MiB 0.0 MiB ctx =
multiprocessing.get_context('spawn')
75 109.5 MiB 0.0 MiB p =
ctx.Process(target=gdalwarp_wrapper, args=(output_path, input_path),
kwargs=kwargs)
76 109.7 MiB 0.1 MiB p.start()
77 105.5 MiB 0.0 MiB p.join()
78 105.5 MiB 0.0 MiB ctx =
multiprocessing.get_context('fork')
79 105.5 MiB 0.0 MiB p =
ctx.Process(target=gdalwarp_wrapper, args=(output_path, input_path),
kwargs=kwargs)
80 105.5 MiB 0.0 MiB p.start()
81 105.5 MiB 0.0 MiB p.join()
82 3750.6 MiB 3645.0 MiB gdalwarp_wrapper(output_path,
input_path, **kwargs)
```
Thanks!
On 26 Dec 2019, at 20:21, Andrew C Aitchison
<[email protected]<mailto:[email protected]>> wrote:
How are you measuring free RAM at this point ?
The GDAL cache will be included in the the memory use reported by the
python interpreter, but at least on Linux the system command "free"
also reports the memory used by the file-system cache, which will contain
(the read parts of) the file, until the system decides to use that RAM for
something else.
Your tempfile is likely being stored in the filesystem cache.
--
Andrew C. Aitchison Kendal, UK
[email protected]<mailto:[email protected]>
_______________________________________________
gdal-dev mailing list
[email protected]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
