STINNER Victor victor.stin...@haypocalc.com added the comment:
Something is very wrong with our code too. I have dumped the text
that's cousing the freeze and run it using the test case scripts.
It worked slow, but worked.
I retried test_MIMEText.tar.bz2 on FreeBSD 8.0 with 640 MB of memory:
Changes by STINNER Victor victor.stin...@haypocalc.com:
--
nosy: -haypo
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue4177
___
___
Martin v. Löwis [EMAIL PROTECTED] added the comment:
IIUC, no patch has been proposed. So retargetting it to later versions.
--
nosy: +loewis
versions: +Python 2.7 -Python 2.5, Python 2.5.3
___
Python tracker [EMAIL PROTECTED]
Sérgio Surkamp [EMAIL PROTECTED] added the comment:
Ok. Something is very wrong with our code too. I have dumped the text
that's cousing the freeze and run it using the test case scripts. It
worked slow, but worked. It seems that our application is eating too
many memory from server (about
Roumen Petrov [EMAIL PROTECTED] added the comment:
what about data segment and stack size limits ?
___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue4177
___
___
STINNER Victor [EMAIL PROTECTED] added the comment:
Your example work here on:
- Linux, i386, 2 Go of memory, Python 2.5
- FreeBSD in Qemu, i386, 512 MB of memory, Python 2.5
The program just eat all the CPU and with high memory usage or raise
a MemoryError
Yes, it takes one minute or more
Sérgio Surkamp [EMAIL PROTECTED] added the comment:
Your text file is ~40 MB. Python may allocate mutiple objects bigger
than 40 MB to create the email content. The algorithm should be
changed to work on a stream (process small chunks, eg. 4 KB) instead
of manipule the full text in memory
STINNER Victor [EMAIL PROTECTED] added the comment:
The original text block is about 5 to 9 Mbytes (...), it freezes
building the MIMEText object. Actually no MemoryError isn't raised,
just a sudden freeze of the running thread.
Can you give more details about the freeze?
- FreeBSD
Sérgio Surkamp [EMAIL PROTECTED] added the comment:
- FreeBSD version?
FreeBSD 7.0-RELEASE
- CPU, memory?
CPU: 2 x Pentium III 1.133 GHz
Memory: 512 Mbytes
- Full Python version?
Python 2.5.2 (r252:60911, Oct 2 2008, 10:03:50)
[GCC 4.2.1 20070719 [FreeBSD]] on freebsd7
On freeze, the
STINNER Victor [EMAIL PROTECTED] added the comment:
Usually 100%. But saw it with more (using both CPU's), I think that mean
more then one thread freezed.
Does the program finish its job after 10 minutes or 1 hour? Using all the CPU
doesn't mean that Python is frozen, it's the opposite:
Sérgio Surkamp [EMAIL PROTECTED] added the comment:
When I first saw the problem, the email system queue was stopped about 2
days (weekend) :-(
The email system control the number of open threads, so I wasn't opening
new threads too and issuing many warnings about it on logs
Anyway, already
Sérgio Surkamp [EMAIL PROTECTED] added the comment:
Testing on Linux:
$ ulimit -m 128000
$ ulimit -v 196000
$ python test_MIMEText.py
[...]
Traceback (most recent call last):
File test_MIMEText.py, line 23, in module
txt = MIMEText(buffer, _subtype=plain, _charset=iso-8859-1)
File
New submission from Sérgio Surkamp [EMAIL PROTECTED]:
If you try to create a MIMEText object from a very large string (test
case include a 40Mbytes string), the program just eat all the CPU and
with high memory usage or raise a MemoryError. Sometimes it just
deadlocks when using _charset =
Roumen Petrov [EMAIL PROTECTED] added the comment:
I don't think that test work on linux without MemoryError. What about if
you set user limits on linux ?
If you enable core file on linux did the test really crash and dump core
or just raise exception and exit without coredump ?
--
14 matches
Mail list logo
