[issue13133] FD leaks in ZipFile.read(), ZipFile.extract() and also using explicit arc_member.close()

[Valery Khamenya]

Valery Khamenya khame...@gmail.com added the comment:

Amaury, I followed your advice. All relevant changes of 3.2 are backported via 
the patch attached.

P.S. now I can install Twisted using pypy too. pypy setup.py install works 
fine for me.

--
keywords: +patch
status: pending - open
Added file: http://bugs.python.org/file23354/zipfile.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13133
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13131] FD leak in urllib2

[Valery Khamenya]

New submission from Valery Khamenya khame...@gmail.com:

Explanation from dablitz's comment at https://bugs.pypy.org/issue867 :

quote
urllib2 in the stdlib leaks fd's if an exception is raised while opening a 
connection. The issue occurs due to a socket being opened then an exception 
being raised before an object with the socket is returned, leaving no way to 
explicitly close the object. On cpython this would not be an issue as the 
object would lose all references almost immediately however it lingers around 
with a proper GC causing FD's to build up if the same condition happens 
repeatedly (eg a loop/web crawling)

The file enclosed is a script to generate the leakage, to run invok it as 
follows python leak.py

pypy should start leaking FD's and can be see in /proc/pid of leak.py/fd

/quote

Related issues:
http://bugs.python.org/issue3066
http://bugs.python.org/issue1208304
http://bugs.python.org/issue1601399

--
components: IO, Library (Lib)
files: leak.py
messages: 145166
nosy: Valery.Khamenya
priority: normal
severity: normal
status: open
title: FD leak in urllib2
type: resource usage
versions: Python 2.7
Added file: http://bugs.python.org/file23344/leak.py

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13131
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13131] FD leak in urllib2

[Valery Khamenya]

Valery Khamenya khame...@gmail.com added the comment:

by the way, timeout parameter should be set to 0.2 as for my 13Mbit ADSL line. 
With 0.002 it is not reproducible for my environment

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13131
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13133] FD leaks in ZipFile.read(), ZipFile.extract() and also using explicit arc_member.close()

[Valery Khamenya]

New submission from Valery Khamenya khame...@gmail.com:

The attached file reproduces 3 types of FD leaks and leads to the error like:

IOError: [Errno 24] Too many open files: '/tmp/1019'

For example if executed with pypy.

--
components: IO, Library (Lib)
files: zipfiletest.py
messages: 145182
nosy: Valery.Khamenya
priority: normal
severity: normal
status: open
title: FD leaks in ZipFile.read(), ZipFile.extract() and also using explicit 
arc_member.close()
type: resource usage
versions: Python 2.7
Added file: http://bugs.python.org/file23347/zipfiletest.py

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13133
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13131] FD leak in urllib2

[Valery Khamenya]

Valery Khamenya khame...@gmail.com added the comment:

the problem seems to be fixed with the patch attached. 
Thanks go to fijal@freenode

--
keywords: +patch
Added file: http://bugs.python.org/file23348/urllib2.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13131
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13133] FD leaks in ZipFile.read(), ZipFile.extract() and also using explicit arc_member.close()

[Valery Khamenya]

Valery Khamenya khame...@gmail.com added the comment:

I can't reproduce this problem in either 2.7.2 or 3.3.0a0.

You probably mean CPython implementation of Python. No, I didn't mean this 
implementation.

Do you mean that this problem is only reproducible when the attached
script is run with pypy?

I can't say only. I just could say yes, it is reproducible with pypy. 
Perhaps, there are others Python implementations that will suffer from the bug 
in current implementation of ZipFile

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13133
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12955] urllib2.build_opener().open() is not friendly to with ... as:

[Valery Khamenya]

Valery Khamenya khame...@gmail.com added the comment:

Terry, Senthil, thanks, for replying to this ticket. OK, to the question:

1. @Terry, here is the full example as for CPython 2.7 I am talking about and 
the output:

#

from urllib2 import Request, build_opener

request = Request('http://example.com')

with build_opener().open(request) as f:
txt = f.read()
print '%d characters fetched' % len(txt)


Traceback (most recent call last):
  File stdin, line 1, in module
AttributeError: addinfourl instance has no attribute '__exit__'


2. @Senthil, regarding the statement being a feature. I assume, that to open a 
connection, to read from a connection and to close it -- are the fundamental 
functions for what urllib2 was in particular created for. I was looking in docs 
for some hints of canonical way of doing this using urllib2.opener. After I 
have failed to find any guidance in docs, I've created this ticket. That is, I 
assume that no new feature is needed, but just a documented 5-lines example 
about a typical way of doing the above operations, especially *closing* the 
connection.

regards,
Valery

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12955
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12955] urllib.request example should use with ... as:

[Valery Khamenya]

Valery Khamenya khame...@gmail.com added the comment:

Guys, in my item 2 the simplistic goal was stated clearly: open, read and close.

Do you confirm that this basic sequence is not supported by urllib2 under 2.7 ?

(I just requested for a tiny documentation update entry)

regards,
Valery

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12955
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12955] urllib2.build_opener().open() is not friendly to with ... as:

[Valery Khamenya]

New submission from Valery Khamenya khame...@gmail.com:

The following intuitive construction

with urllib2.build_opener().open() as:
...

leads to AttributeError: addinfourl instance has no attribute '__exit__'

http://docs.python.org/library/urllib2.html says almost nothing about concept 
of closing.

Could it be possible to add a canonical with ... as: example for 
urllib2.build_opener() ?

Thanks,
Valery

--
assignee: docs@python
components: Documentation, IO, Library (Lib)
messages: 143836
nosy: Valery.Khamenya, docs@python
priority: normal
severity: normal
status: open
title: urllib2.build_opener().open() is not friendly to with ... as:
type: resource usage
versions: Python 2.6, Python 2.7, Python 3.1, Python 3.2, Python 3.3, Python 3.4

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue12955
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10351] Add autocompletion for keys in dictionaries

[Valery Khamenya]

Valery Khamenya khame...@gmail.com added the comment:

Guys, do you expect anythig else from me in respect to this issue? Let me know 
it before my non-stopable garbage collector wipes all the details from my brain 
away :)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10351
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10351] Add autocompletion for keys in dictionaries

[Valery Khamenya]

Valery Khamenya khame...@gmail.com added the comment:

Hi Éric, thanks for guiding. 

So, attached is the concatenation of two forward unified diffs for 
rlcompleter.py and test_rlcompleter.py -- both as of py3k trunk. Tested against 
Python 3.1.2 though.

P.S. hm, py3k code appeared to be surprisingly nicer -- no unicode troubles and 
work-arounds at all...

regards
Valery

--
assignee:  - d...@python
components: +Documentation
keywords: +patch
nosy: +d...@python
Added file: http://bugs.python.org/file19555/patch.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10351
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10351] to introduce autocompletion for keys in dictionaries (patch attached)

[Valery Khamenya]

New submission from Valery Khamenya khame...@gmail.com:

1. The patch introduces autocompletion for keys in dictionaries (patch attached)

2. The patched rlcompleter as such works OK for unicode dictionary keys as 
well. All tests pass OK. HOWEVER, readline's completion mechanism seem to be 
confused with unicode strings -- see comments to Completer.dict_key_matches(). 
So, perhaps, one day some changes should be applied to readline code too.

3. rlcompleter.py has no tests in trunk -- I spawn a separate issue for it. 
Meanwhile I took test_rlcompleter.py from 2.7 and extended it.

4. The following usual lines in .pythonstartup:
import readline
import rlcompleter
readline.parse_and_bind('tab: complete')
readline.parse_and_bind('Control-Space: complete')

should be extended by this one:
readline.set_completer_delims(re.compile(r'[\'\\[]').sub('', 
readline.get_completer_delims()))

--
components: Library (Lib)
files: rlcompleter-dict-keys-autocompletion.tar.gz
messages: 120721
nosy: Valery.Khamenya
priority: normal
severity: normal
status: open
title: to introduce autocompletion for keys in dictionaries (patch attached)
type: behavior
versions: Python 2.6, Python 2.7, Python 3.1, Python 3.2, Python 3.3
Added file: 
http://bugs.python.org/file19540/rlcompleter-dict-keys-autocompletion.tar.gz

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10351
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10352] rlcompleter.py has no tests in trunk

[Valery Khamenya]

New submission from Valery Khamenya khame...@gmail.com:

rlcompleter.py has no test_rlcompleter in trunk, see 
http://svn.python.org/view/python/trunk/Lib/test/
There is one in 2.7 though.

Remark: the issue http://bugs.python.org/issue10351 introduces autocompletion 
patch and comes with new tests. So, one may want to use it as for up-to-date 
test_rlcompleter.py, given the issue #10351 is accepted ;-)

--
components: Library (Lib), Tests
messages: 120722
nosy: Valery.Khamenya
priority: normal
severity: normal
status: open
title: rlcompleter.py has no tests in trunk
versions: Python 2.6

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10352
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com