New submission from 李超然 <seraphliv...@gmail.com>:
We find an issue while using shared memory. When opening another process to
overwrite the shared memory, the memory of this process will increase to about
the size of this shared memory. So when several processes try to read or write
the shared memory, there will be N times memory usage.
```
import os, psutil
import numpy as np
from multiprocessing import Process, shared_memory
SHM_SIZE = 100000 * 30 * 20
def f(name):
print('[Sub] (Before) Used Memory of {}: {} MiB'.format(
os.getpid(),
psutil.Process(os.getpid()).memory_info().rss / 1024 / 1024,
))
shm = shared_memory.SharedMemory(name=name)
b = np.ndarray(shape=(SHM_SIZE, 1), dtype=np.float64, buffer=shm.buf)
for i in range(SHM_SIZE):
b[i, 0] = 1
print('[Sub] (After) Used Memory of {}: {} MiB'.format(
os.getpid(),
psutil.Process(os.getpid()).memory_info().rss / 1024 / 1024,
))
def main():
print('[Main] Used Memory of {}: {} MiB'.format(
os.getpid(),
psutil.Process(os.getpid()).memory_info().rss / 1024 / 1024,
))
shm = shared_memory.SharedMemory(create=True, size=8*SHM_SIZE, name='shm')
p = Process(target=f, args=('shm',))
p.start()
p.join()
print('[Main] Used Memory of {}: {} MiB'.format(
os.getpid(),
psutil.Process(os.getpid()).memory_info().rss / 1024 / 1024,
))
input()
shm.close()
shm.unlink()
if __name__ == '__main__':
main()
```
----------
components: Library (Lib)
messages: 375642
nosy: seraphlivery
priority: normal
severity: normal
status: open
title: Potential memory leak while using shared memory
type: resource usage
versions: Python 3.8
_______________________________________
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue41587>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
