Hello,
I'm filing this as a bug report here as the issue tracker is closed to
anonymous users.
## Summary
There seems to be a memory leak issue when using the buffer and memoryview
support of the cpython.array Cython interface to the inbuilt Python
array.array module.
## OS / Environment
Ubuntu 14.04
Cython 0.22
Python 2.7.6/2.7.9
## Description
When allocating `cpython.array` objects using the `clone` method provided
in the Cython interface to a buffer or memoryview type the memory does not
seem to be freed correctly when all references to the object have been
removed when running Cython code with Python 2.7 (tested with both Ubuntu
provided 2.7.6 version and 2.7.9 downloaded from python.org this evening).
This issue does not seem to occur when running the exact same Cython code
with Python 3.4.0.
This appears like it could potentially be a long standing bug as the same
issue was mentioned on the cython-user mailing list in November 2012
([here][1]).
Using `cpython.array` with memory views seems to be quite a typical use
case: it is discussed in the documentation [here][2] and [here][3]
(specifically being stated to be memory safe) and is recommended in a
popular StackOverflow answer [here][4] on the best way for allocating
memory for a typed memory view in Cython (based on benchmarks performed in
Python 3.x).
[1]: https://groups.google.com/d/msg/cython-users/CwtU_jYADgM/660O2JJuO54J
[2]:
http://docs.cython.org/src/userguide/memoryviews.html#cpython-array-module
[3]: http://docs.cython.org/src/tutorial/array.html
[4]: http://stackoverflow.com/a/21054369/4798943
## Steps to reproduce
Cython code is attached (also copied below as not sure whether attachment
will be preserved in posting) to reproduce this error.
Running this code with Python 2.7.9 gives output
> Memory usage: 64.16015625 MiB
> Starting cpython array buffer initialisation loop
> Finished cpython array buffer initialisation loop
> Memory usage: 459.71484375 MiB
> Starting cpython array memview initialisation loop
> Finished cpython array memview initialisation loop
> Memory usage: 849.55078125 MiB
and with Python 3.4.0
> Memory usage: 52.79296875 MiB
> Starting cpython array buffer initialisation loop
> Finished cpython array buffer initialisation loop
> Memory usage: 52.8125 MiB
> Starting cpython array memview initialisation loop
> Finished cpython array memview initialisation loop
> Memory usage: 52.81640625 MiB
Let me know if any more details are needed.
Thanks,
Matt
```
# cython: boundscheck=False
# cython: wraparound=False
import resource
import gc
from cpython.array cimport array, clone
def init_cpython_array_buffer(int length, int loops):
cdef int i
cdef array[double] arr, template = array('d')
for i in range(loops):
arr = clone(template, length, False)
def init_cpython_array_memview(int length, int loops):
cdef int i
cdef double[::1] arr
cdef array template = array('d')
for i in range(loops):
arr = clone(template, length, False)
mem = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss / 1024.
print('Memory usage: {0} MiB'.format(mem))
print('Starting cpython array buffer initialisation loop')
init_cpython_array_buffer(100000, 100000)
# Force a garbage collection just to be safe
gc.collect()
print('Finished cpython array buffer initialisation loop')
mem = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss / 1024.
print('Memory usage: {0} MiB'.format(mem))
print('Starting cpython array memview initialisation loop')
init_cpython_array_memview(100000, 100000)
# Force a garbage collection just to be safe
gc.collect()
print('Finished cpython array memview initialisation loop')
mem = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss / 1024.
print('Memory usage: {0} MiB'.format(mem))
```
import resource
import gc
from cpython.array cimport array, clone
def init_cpython_array_buffer(int length, int loops):
cdef int i
cdef array[double] arr, template = array('d')
for i in range(loops):
arr = clone(template, length, False)
def init_cpython_array_memview(int length, int loops):
cdef int i
cdef double[::1] arr
cdef array template = array('d')
for i in range(loops):
arr = clone(template, length, False)
mem = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss / 1024.
print('Memory usage: {0} MiB'.format(mem))
print('Starting cpython array buffer initialisation loop')
init_cpython_array_buffer(100000, 100000)
# Force a garbage collection just to be safe
gc.collect()
print('Finished cpython array buffer initialisation loop')
mem = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss / 1024.
print('Memory usage: {0} MiB'.format(mem))
print('Starting cpython array memview initialisation loop')
init_cpython_array_memview(100000, 100000)
# Force a garbage collection just to be safe
gc.collect()
print('Finished cpython array memview initialisation loop')
mem = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss / 1024.
print('Memory usage: {0} MiB'.format(mem))_______________________________________________
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel
