[issue1441530] socket read() can cause MemoryError in Windows

2011-05-24 Thread Charles-François Natali
Charles-François Natali added the comment: I've committed the patch to 2.7, and also to default (and only to default since for py3k it's more of an optimization than a bug fix). Closing. -- resolution: -> fixed stage: patch review -> committed/rejected status: open -> closed

[issue1441530] socket read() can cause MemoryError in Windows

2011-05-24 Thread Roundup Robot
Roundup Robot added the comment: New changeset 14bb95a8d7ee by Charles-François Natali in branch 'default': Issue #1441530: In imaplib, read the data in one chunk to speed up large http://hg.python.org/cpython/rev/14bb95a8d7ee -- ___ Python tracker

[issue1441530] socket read() can cause MemoryError in Windows

2011-05-24 Thread Roundup Robot
Roundup Robot added the comment: New changeset 0dee36595699 by Charles-François Natali in branch '2.7': Issue #1441530: In imaplib, use makefile() to wrap the SSL socket to avoid http://hg.python.org/cpython/rev/0dee36595699 -- nosy: +python-dev ___

[issue1441530] socket read() can cause MemoryError in Windows

2011-05-22 Thread Antoine Pitrou
Antoine Pitrou added the comment: Both patches look ok to me. -- versions: +Python 3.3 ___ Python tracker ___ ___ Python-bugs-list

[issue1441530] socket read() can cause MemoryError in Windows

2011-05-22 Thread Charles-François Natali
Changes by Charles-François Natali : Removed file: http://bugs.python.org/file22051/imaplib_ssl_makefile.diff ___ Python tracker ___ ___ Pyt

[issue1441530] socket read() can cause MemoryError in Windows

2011-05-22 Thread Charles-François Natali
Changes by Charles-François Natali : Removed file: http://bugs.python.org/file22044/imaplib_read.diff ___ Python tracker ___ ___ Python-bugs

[issue1441530] socket read() can cause MemoryError in Windows

2011-05-22 Thread Charles-François Natali
Changes by Charles-François Natali : Added file: http://bugs.python.org/file22063/imaplib_recv_27.diff ___ Python tracker ___ ___ Python-bug

[issue1441530] socket read() can cause MemoryError in Windows

2011-05-22 Thread Charles-François Natali
Charles-François Natali added the comment: Digging a little deeper, here's the conclusion: - with py3k, fragmentation is less likely: the buffered reader returned by makefile() ensures that we can allocate only one result buffer for the total number of bytes read() (thanks to socket's readinto

[issue1441530] socket read() can cause MemoryError in Windows

2011-05-21 Thread Charles-François Natali
Charles-François Natali added the comment: In the buffered reader case, the result buffer is actually pre-allocated with the total size, making fragmentation even less likely. -- ___ Python tracker

[issue1441530] socket read() can cause MemoryError in Windows

2011-05-21 Thread Charles-François Natali
Charles-François Natali added the comment: > Patch looks ok. Is 3.x also affected? The I/O stack changed quite a bit in > 3.x. I think it's not affected, but I can't reproduce this behaviour with glibc/eglibc, so don't just take my word for it. The reason is that in py3k, imaplib uses a buffer

[issue1441530] socket read() can cause MemoryError in Windows

2011-05-21 Thread Antoine Pitrou
Antoine Pitrou added the comment: Patch looks ok. Is 3.x also affected? The I/O stack changed quite a bit in 3.x. -- nosy: +pitrou ___ Python tracker ___ _

[issue1441530] socket read() can cause MemoryError in Windows

2011-05-21 Thread Charles-François Natali
Changes by Charles-François Natali : Removed file: http://bugs.python.org/file16747/imaplib_read.diff ___ Python tracker ___ ___ Python-bugs

[issue1441530] socket read() can cause MemoryError in Windows

2011-05-21 Thread Charles-François Natali
Charles-François Natali added the comment: It's actually an obvious case of heap fragmentation due to long-lived chunks being realloc()ed to a smaller size. Some malloc implementations can choke on this (e.g. OS-X's malloc is known to not shrink blocks when realloc() is called with a smaller

[issue1441530] socket read() can cause MemoryError in Windows

2010-08-21 Thread Mark Lawrence
Mark Lawrence added the comment: At least three solutions to this have been suggested. The simplest is that provided by Márcio but is this acceptable? -- nosy: +BreamoreBoy stage: -> patch review type: performance -> behavior versions: +Python 3.1 -Python 2.6 ___

[issue1441530] socket read() can cause MemoryError in Windows

2010-07-23 Thread Márcio Faustino
Márcio Faustino added the comment: I got that error on Windows 7 Professional 64 bits, using Python 2.6.4 64 bits. I just changed the "imaplib.IMAP4_SSL" class, by making the "read" and "readline" functions use "cStringIO.StringIO()" instead of an array of strings for "chunks = []".

[issue1441530] socket read() can cause MemoryError in Windows

2010-07-21 Thread Amaury Forgeot d'Arc
Amaury Forgeot d'Arc added the comment: Which module did you change, and which precise version of python are you using? I wonder if this was fixed with issue2632. -- ___ Python tracker _

[issue1441530] socket read() can cause MemoryError in Windows

2010-07-21 Thread Márcio Faustino
Márcio Faustino added the comment: Speaking for myself, I'm not using the attached patch, I'm using the simple fix I included in my previous reply which works perfectly to avoid getting a MemoryError exception thrown. -- ___ Python tracker

[issue1441530] socket read() can cause MemoryError in Windows

2010-07-21 Thread Amaury Forgeot d'Arc
Amaury Forgeot d'Arc added the comment: I fail to see *why* the patch fixes the issue. You still have to allocate the big string when joining the parts. -- nosy: +amaury.forgeotdarc ___ Python tracker _

[issue1441530] socket read() can cause MemoryError in Windows

2010-04-03 Thread Charles-Francois Natali
Charles-Francois Natali added the comment: $ cat /tmp/test.py import socket SIZE = 10L s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) try: s.recv(SIZE) finally: s.close() $ python /tmp/test.py Traceback (most recent call last): File "/tmp/test.py", line 8, in s.r

[issue1441530] socket read() can cause MemoryError in Windows

2010-02-24 Thread Márcio Faustino
Márcio Faustino added the comment: I've been also receiving the same error while using "imaplib.IMAP4_SSL" to download large e-mails. A simple change in the "read" and "readline" functions to use instead a "cStringIO.StringIO" buffer object works for me. -- nosy: +marcio ___

[issue1441530] socket read() can cause MemoryError in Windows

2008-09-11 Thread Anthony Lenton
Anthony Lenton <[EMAIL PROTECTED]> added the comment: It's probably just a typo from copying from an editor, but there is a bug in the workaround. It should be: maxRead = 100 class MySSL (imaplib.IMAP4_SSL): def read (self, n): #print "..Attempting to read %d bytes" % n if n <=

[issue1441530] socket read() can cause MemoryError in Windows

2008-09-09 Thread Anthony Lenton
Anthony Lenton <[EMAIL PROTECTED]> added the comment: I confirm that the bug occurs with Python 2.5.1 on Windows XP. Also, anglocelt's fix worked fine for me. -- nosy: +elachuni ___ Python tracker <[EMAIL PROTECTED]>

[issue1441530] socket read() can cause MemoryError in Windows

2008-09-02 Thread Iain MacKay
Iain MacKay <[EMAIL PROTECTED]> added the comment: Knowing that the large read provokes the problem enables one to write this simple workaround by subclassing IMAP4 without patching the library: maxRead = 100 class MySSL (imaplib.IMAP4_SSL): def read (self, n): #print "..Attempting to

[issue1441530] socket read() can cause MemoryError in Windows

2008-01-05 Thread vila
Changes by vila: -- nosy: +vila _ Tracker <[EMAIL PROTECTED]> _ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.o