[issue22534] Possible Memory Leak with 'shelve'

[TJ]

TJ added the comment:

Thanks for the help. For my linux box, I have no issue. This is Ubuntu 13.10:

Linux localhost 3.11.0-26-generic #45-Ubuntu SMP Tue Jul 15 04:02:06 UTC 
2014 x86_64 x86_64 x86_64 GNU/Linux

2.7.5+ (default, Feb 27 2014, 19:37:08) 
[GCC 4.8.1]

However, I'm still getting issues with my other box, which is Mac OS. I think I 
have the relevant info here (let me know if we need additional info):

[10:55:00] ~$ sw_vers -productVersion
10.9.2

[10:55:10] ~$ python -c "import sys; print sys.version"
2.7.8 (default, Aug 21 2014, 20:13:48) 
[GCC 4.2.1 Compatible Apple LLVM 5.1 (clang-503.0.40)]

[10:55:57] ~$ python -c "import anydbm;d = anydbm.open('bla', 
'c');print(type(d))"


[10:56:05] ~$ cat leaktest.py
import anydbm
while True:
d = anydbm.open('bla', 'c')
d.close()

[10:56:10] ~$ python leaktest.py &
[1] 99030

[10:56:18] ~$ ps -o pid,rss,vsz,command -p 99030
  PIDRSS  VSZ COMMAND
99030 481928  3010600 python leaktest.py

[10:56:56] ~$ ps -o pid,rss,vsz,command -p 99030
  PIDRSS  VSZ COMMAND
99030 3985920  6516788 python leaktest.py

[11:01:10] ~$ # After 5 minutes, "top" shows MEM as 13G

[11:01:23] ~$ ps -o pid,rss,vsz,command -p 99030
  PIDRSS  VSZ COMMAND
99030 13891152 16429156 python leaktest.py

Those numbers are stable on my Linux box, and ever growing on Mac.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22534] Possible Memory Leak with 'shelve'

[STINNER Victor]

STINNER Victor added the comment:

I modified the example a little bit to display the RSS memory 10 times. The RSS 
increases by +176 kB at the beginning and then it is stable.

I tested on Fedora 20 (Linux): anydbm.open('bla', 'c') creates a 
'bsddb._DBWithCursor' object. Can you please give us the DBM type returned by:

$ ./python
Python 2.7.8+ (2.7:9b4673d7b046, Oct  1 2014, 00:20:22) 
>>> import anydbm
>>> d=anydbm.open('bla', 'c')
>>> type(d)

>>> d.close()

The loop of your example can maybe by simplified to:

d=anydbm.open('bla', 'c')
d.close()

instead of

sh = shelve.open('blah')
sh.close()

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22534] Possible Memory Leak with 'shelve'

[Ned Deily]

Ned Deily added the comment:

Sorry, I'm not able to reproduce any major memory leak, using a couple of 
different dbm implementations, and there is not enough information to go on.

To pursue further, you should identify:
- what platform (operating system distribution and version)
- exactly what version of Python you are using and its origins (if not provided 
by the operating system distribution):
python2.7 -c 'import sys;print(sys.version)'
- a test case that shows exactly what you run and how you determine that memory 
is growing
- after running the test (and possibly aborting it) and the database file 
"blah" has been created:
python2.7 -c 'import whichdb;print(whichdb.whichdb("blah"))'
That should show which dbm is being used.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22534] Possible Memory Leak with 'shelve'

[TJ]

TJ added the comment:

Put it in the while loop. Same result. Memory usage grows about 1 GiB per 
minute.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22534] Possible Memory Leak with 'shelve'

[Ned Deily]

Ned Deily added the comment:

What happens if you use xrange instead of range?

>>> range(100).__sizeof__()
856
>>> xrange(100).__sizeof__()
56

--
nosy: +ned.deily

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22534] Possible Memory Leak with 'shelve'

[TJ]

New submission from TJ:

The following code causes memory usage to grow excessively.

'''
import shelve

def func():
for i in range(100):
sh = shelve.open('blah')
sh.close()

func()
'''

--
components: Library (Lib)
messages: 228127
nosy: tjhnson
priority: normal
severity: normal
status: open
title: Possible Memory Leak with 'shelve'
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com