Roundup Robot devnull@devnull added the comment:
New changeset 0dee36595699 by Charles-François Natali in branch '2.7':
Issue #1441530: In imaplib, use makefile() to wrap the SSL socket to avoid
http://hg.python.org/cpython/rev/0dee36595699
--
nosy: +python-dev
Roundup Robot devnull@devnull added the comment:
New changeset 14bb95a8d7ee by Charles-François Natali in branch 'default':
Issue #1441530: In imaplib, read the data in one chunk to speed up large
http://hg.python.org/cpython/rev/14bb95a8d7ee
--
___
Charles-François Natali neolo...@free.fr added the comment:
I've committed the patch to 2.7, and also to default (and only to default since
for py3k it's more of an optimization than a bug fix).
Closing.
--
resolution: - fixed
stage: patch review - committed/rejected
status: open -
Charles-François Natali neolo...@free.fr added the comment:
Digging a little deeper, here's the conclusion:
- with py3k, fragmentation is less likely: the buffered reader returned by
makefile() ensures that we can allocate only one result buffer for the total
number of bytes read() (thanks to
Changes by Charles-François Natali neolo...@free.fr:
Added file: http://bugs.python.org/file22063/imaplib_recv_27.diff
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1441530
___
Changes by Charles-François Natali neolo...@free.fr:
Removed file: http://bugs.python.org/file22044/imaplib_read.diff
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1441530
___
Changes by Charles-François Natali neolo...@free.fr:
Removed file: http://bugs.python.org/file22051/imaplib_ssl_makefile.diff
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1441530
___
Antoine Pitrou pit...@free.fr added the comment:
Both patches look ok to me.
--
versions: +Python 3.3
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1441530
___
Charles-François Natali neolo...@free.fr added the comment:
It's actually an obvious case of heap fragmentation due to long-lived chunks
being realloc()ed to a smaller size. Some malloc implementations can choke on
this (e.g. OS-X's malloc is known to not shrink blocks when realloc() is called
Changes by Charles-François Natali neolo...@free.fr:
Removed file: http://bugs.python.org/file16747/imaplib_read.diff
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1441530
___
Antoine Pitrou pit...@free.fr added the comment:
Patch looks ok. Is 3.x also affected? The I/O stack changed quite a bit in 3.x.
--
nosy: +pitrou
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1441530
Charles-François Natali neolo...@free.fr added the comment:
Patch looks ok. Is 3.x also affected? The I/O stack changed quite a bit in
3.x.
I think it's not affected, but I can't reproduce this behaviour with
glibc/eglibc, so don't just take my word for it.
The reason is that in py3k,
Charles-François Natali neolo...@free.fr added the comment:
In the buffered reader case, the result buffer is actually pre-allocated with
the total size, making fragmentation even less likely.
--
___
Python tracker rep...@bugs.python.org
Mark Lawrence breamore...@yahoo.co.uk added the comment:
At least three solutions to this have been suggested. The simplest is that
provided by Márcio but is this acceptable?
--
nosy: +BreamoreBoy
stage: - patch review
type: performance - behavior
versions: +Python 3.1 -Python 2.6
Márcio Faustino marciombfaust...@gmail.com added the comment:
I got that error on Windows 7 Professional 64 bits, using Python 2.6.4 64 bits.
I just changed the imaplib.IMAP4_SSL class, by making the read and
readline functions use cStringIO.StringIO() instead of an array of strings
for
Amaury Forgeot d'Arc amaur...@gmail.com added the comment:
I fail to see *why* the patch fixes the issue. You still have to allocate the
big string when joining the parts.
--
nosy: +amaury.forgeotdarc
___
Python tracker rep...@bugs.python.org
Márcio Faustino marciombfaust...@gmail.com added the comment:
Speaking for myself, I'm not using the attached patch, I'm using the simple fix
I included in my previous reply which works perfectly to avoid getting a
MemoryError exception thrown.
--
Amaury Forgeot d'Arc amaur...@gmail.com added the comment:
Which module did you change, and which precise version of python are you using?
I wonder if this was fixed with issue2632.
--
___
Python tracker rep...@bugs.python.org
Charles-Francois Natali neolo...@free.fr added the comment:
$ cat /tmp/test.py
import socket
SIZE = 10L
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
try:
s.recv(SIZE)
finally:
s.close()
$ python /tmp/test.py
Traceback (most recent call last):
File /tmp/test.py, line
Anthony Lenton [EMAIL PROTECTED] added the comment:
It's probably just a typo from copying from an editor, but there is a
bug in the workaround. It should be:
maxRead = 100
class MySSL (imaplib.IMAP4_SSL):
def read (self, n):
#print ..Attempting to read %d bytes % n
if n =
Anthony Lenton [EMAIL PROTECTED] added the comment:
I confirm that the bug occurs with Python 2.5.1 on Windows XP. Also,
anglocelt's fix worked fine for me.
--
nosy: +elachuni
___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue1441530
Iain MacKay [EMAIL PROTECTED] added the comment:
Knowing that the large read provokes the problem enables one to write
this simple workaround by subclassing IMAP4 without patching the library:
maxRead = 100
class MySSL (imaplib.IMAP4_SSL):
def read (self, n):
#print ..Attempting to
Changes by vila:
--
nosy: +vila
_
Tracker [EMAIL PROTECTED]
http://bugs.python.org/issue1441530
_
___
Python-bugs-list mailing list
Unsubscribe:
23 matches
Mail list logo
