Amaury Forgeot d'Arc [EMAIL PROTECTED] added the comment:
I found that on my Debian64, running test_multiprocessing under gdb
hangs even before the first test is started - somewhere in the
installation of the Manager.
And it appears that the problem is described in issue874900: threading
module
Jesse Noller [EMAIL PROTECTED] added the comment:
Thanks Amaury - I've been working through the tests and identifying
the problem children - I'll finish that up and then try re-running
them with the 874900 patch.
___
Python tracker [EMAIL PROTECTED]
Nick Coghlan [EMAIL PROTECTED] added the comment:
I'm still seeing intermittent lockups on Ubuntu 7.10 - traceback on
ctrl-C is similar to that posted by Ismail above.
Since Jesse seems to be on top of this, I'll stick to using -x
test_multiprocessing for the moment.
--
nosy: +ncoghlan
Nick Coghlan [EMAIL PROTECTED] added the comment:
Bumping back to release blocker for beta 2 (Barry may choose to defer it
again, but it should at least be on his radar).
--
priority: critical - release blocker
resolution: accepted -
___
Python
Ismail Donmez [EMAIL PROTECTED] added the comment:
I managed to hang on Ubuntu, here is the backtrace that I got with CTRL-C:
Process PoolWorker-5:1:
Traceback (most recent call last):
File /home/cartman/Sources/py3k/Lib/multiprocessing/process.py, line
232, in _bootstrap
test_bsddb
Amaury Forgeot d'Arc [EMAIL PROTECTED] added the comment:
The two tracebacks provided by Mark seem to correspond to the following
python stack (innermost last):
Lib/test/test_multiprocessing.py, line 1005, in _test_map_unordered
self.assertEqual(sorted(it), map(sqr, range(1000)))
Ismail Donmez [EMAIL PROTECTED] added the comment:
The test hanged for me at first try but worked fine on the second test,
weird.
___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3088
___
Paul Melis [EMAIL PROTECTED] added the comment:
On a Linux system (FC4) with r64686 of the Py3k branch I also still get
occassional hangs (with ./python -E -bb ./Lib/test/regrtest.py -v
test_multiprocessing). Mostly this seems to occur with the very first
test executed, i.e. before any of the
Mark Dickinson [EMAIL PROTECTED] added the comment:
Are you sure that's right? That traceback has no mention of
PyObject_Malloc or obmalloc.c.
So now I think that the traceback was right. There was no mention of
PyObject_Malloc or obmalloc.c because the traceback only showed 1 of the 9
Amaury Forgeot d'Arc [EMAIL PROTECTED] added the comment:
Not sure that all problems reported here are resolved.
For example, test_get (__main__.WithProcessesTestQueue) does not seem
to use threading.local at all.
___
Python tracker [EMAIL PROTECTED]
Miki Tebeka [EMAIL PROTECTED] added the comment:
Still hangs for me on revision 64665
___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3088
___
___
Jesse Noller [EMAIL PROTECTED] added the comment:
Miki where does it hang? I've run it in a tight loop for over 500
iterations without a single hang.
___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3088
Mark Dickinson [EMAIL PROTECTED] added the comment:
The g4 buildbot still isn't happy, either. See:
http://www.python.org/dev/buildbot/trunk.stable/g4%20osx.4%20trunk/builds/
3645/step-test/0
(This was a run of revision 64663, I think.)
___
Python tracker
Jesse Noller [EMAIL PROTECTED] added the comment:
The G4 buildbot is green right now.
___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3088
___
___
Jesse Noller [EMAIL PROTECTED] added the comment:
Looking through the buildbot's history (OSX G4) I can see it hung *prior*
and after the change to turn the manager tests back on, so it's unrelated
to those tests. I'll reopen this bug.
--
resolution: fixed - accepted
status: closed -
Mark Dickinson [EMAIL PROTECTED] added the comment:
The G4 buildbot is green right now.
Sure. But it looks like test_multiprocessing hung on build 3645, which
as far as I can tell included all the recent fixes.
My machine's behaving the same way: when doing 'make test', sometimes
Jesse Noller [EMAIL PROTECTED] added the comment:
What platform mark?
___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3088
___
___
Python-bugs-list mailing list
Mark Dickinson [EMAIL PROTECTED] added the comment:
Here's an interesting snippet from the my most recent attempt to run
'make test'. I got a failed assertion:
[...]
test_multibytecodec_support
test_multifile
test_multiprocessing
Assertion failed: (bp != NULL), function PyObject_Malloc, file
Mark Dickinson [EMAIL PROTECTED] added the comment:
What platform mark?
OS X 10.5.4/Intel Core 2 Duo. (It's a 2007 Macbook Pro.)
___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3088
___
Jesse Noller [EMAIL PROTECTED] added the comment:
Thanks. I'm a been mystified why it hanging on you - I'm on a 2008 Macbook
Pro running latest 10.5.4. I'm still trying to get it to hang though.
___
Python tracker [EMAIL PROTECTED]
Jesse Noller [EMAIL PROTECTED] added the comment:
Mark, can you try commenting out _TestCondition and seeing if you can
still get it to hang?;
___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3088
___
Mark Dickinson [EMAIL PROTECTED] added the comment:
Mark, can you try commenting out _TestCondition and seeing if you can
still get it to hang?;
I removed the _TestCondition class entirely from test_multiprocessing,
and did make test again. It didn't hang! :-) It crashed instead. :-(
Jesse Noller [EMAIL PROTECTED] added the comment:
That output exceeds my knowledge of python internals by several light
years. :(
___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3088
___
Adam Olsen [EMAIL PROTECTED] added the comment:
On Wed, Jul 2, 2008 at 3:44 PM, Mark Dickinson [EMAIL PROTECTED] wrote:
Mark Dickinson [EMAIL PROTECTED] added the comment:
Mark, can you try commenting out _TestCondition and seeing if you can
still get it to hang?;
I removed the
Amaury Forgeot d'Arc [EMAIL PROTECTED] added the comment:
And please add the -v option at the end, so we can see which function
fails
___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3088
___
Mark Dickinson [EMAIL PROTECTED] added the comment:
Okay. I just got about 5 perfect runs of the test suite, followed by:
Macintosh-3:trunk dickinsm$ ./python.exe -m test.regrtest
[...]
test_multiprocessing
Assertion failed: (bp != NULL), function PyObject_Malloc, file
Objects/obmalloc.c,
Adam Olsen [EMAIL PROTECTED] added the comment:
On Wed, Jul 2, 2008 at 5:08 PM, Mark Dickinson [EMAIL PROTECTED] wrote:
Mark Dickinson [EMAIL PROTECTED] added the comment:
Okay. I just got about 5 perfect runs of the test suite, followed by:
Macintosh-3:trunk dickinsm$ ./python.exe -m
Miki Tebeka [EMAIL PROTECTED] added the comment:
I just run make test and it never moves past test_multiprocessing.
Maybe it's my machine which is dual cpu quad core (total of 8 cores)?
___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3088
Jesse Noller [EMAIL PROTECTED] added the comment:
Doubtful Miki - I do the work on the module on an 8 Core Gentoo, 8 Core
Mac Pro and Dual Core Macbook Pro - it's not a # of cores issue, unless
it's simply a 1 issue.
___
Python tracker [EMAIL PROTECTED]
Mark Dickinson [EMAIL PROTECTED] added the comment:
Are you sure that's right?
Not at all. :-)
That traceback has no mention of
PyObject_Malloc or obmalloc.c. Try checking the date. Also, if you
use gdb ./python.exe corefile to start gdb it should print a
warning if the program
Mark Dickinson [EMAIL PROTECTED] added the comment:
Here's a new traceback (a different error again, this time: a negative
refcount in Objects/tupleobject.c.)
Added file: http://bugs.python.org/file10796/traceback2.txt
___
Python tracker [EMAIL PROTECTED]
Adam Olsen [EMAIL PROTECTED] added the comment:
That looks better. It crashed while deleting an exception, who's args
tuple has a bogus refcount. Could be a refcount issue of the
exception or the args, or of something that that references them, or a
dangling pointer, or a buffer overrun, etc.
Adam Olsen [EMAIL PROTECTED] added the comment:
Also, make sure you do a make clean since you last updated the tree or
touched any file or ran configure. The automatic dependency checking
isn't 100% reliable.
___
Python tracker [EMAIL PROTECTED]
Jesse Noller [EMAIL PROTECTED] added the comment:
Barring the segfaults Mark is seeing, I went through and removed all of
the tests, and I have been incrementally adding them back one by one.
_TestQueue seems to be the one (at least, the first) which is hanging
intermittently in a racquire().
Jesse Noller [EMAIL PROTECTED] added the comment:
Alright - I'll do that asap
On Jun 30, 2008, at 7:06 PM, Amaury Forgeot d'Arc [EMAIL PROTECTED]
wrote:
Amaury Forgeot d'Arc [EMAIL PROTECTED] added the comment:
Committed my changes to threading.local as r64601.
I think we could enable
Amaury Forgeot d'Arc [EMAIL PROTECTED] added the comment:
I think I narrowed the problem to a race condition in *subclasses* of
threading.local:
In threadmodule.c::local_getattro, there is a chance that self-dict is
changed before PyObject_GenericGetAttr is called.
--
nosy:
Amaury Forgeot d'Arc [EMAIL PROTECTED] added the comment:
After 4 hours spent with the debugger I think I have an explanation and
a correction of one of the problems.
Note that this only affects the Manager tests.
- The threading.local type overrides getattr and setattr: on every call,
it
Miki Tebeka [EMAIL PROTECTED] added the comment:
Still hangs for me on the 2.6 trunk on Ubuntu 8.04
--
nosy: +tebeka
___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3088
___
Jesse Noller [EMAIL PROTECTED] added the comment:
Where exactly does it hang Miki?
___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3088
___
___
Python-bugs-list
Miki Tebeka [EMAIL PROTECTED] added the comment:
Jesse,
I just run make test, it runs until test_multiprocessing and then
hangs there
___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3088
___
Changes by Jesse Noller [EMAIL PROTECTED]:
--
assignee: - jnoller
___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3088
___
___
Python-bugs-list mailing
Barry A. Warsaw [EMAIL PROTECTED] added the comment:
I'm going to knock this one down to critical since it's working for me
now on OS X and buildbot looks green. We can address any additional
patches after the beta release.
--
priority: release blocker - critical
Jesse Noller [EMAIL PROTECTED] added the comment:
I don't have commit rights, so I can't apply the
test_multiprocessing_reduced.diff myself. Anyone willing? I think this
should help the buildbots.
___
Python tracker [EMAIL PROTECTED]
Jesse Noller [EMAIL PROTECTED] added the comment:
After talking with Richard, I think the best way to attack this issue
(and the other ones around suite unreliability) is to remove the
unreliable test cases for the first beta, and then refactor the suite
post beta with an eye towards
Jesse Noller [EMAIL PROTECTED] added the comment:
Here is the loop I ran the tests with:
#!/bin/sh
for (( i=1;i=100;i+=1 )); do
make test TESTOPTS=-v test_multiprocessing
done
___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3088
Ismail Donmez [EMAIL PROTECTED] added the comment:
Seems to work fine for me now with latest py3k branch.
___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3088
___
roudkerk [EMAIL PROTECTED] added the comment:
I suspect the problems with WithManagerTestCondition.notify_all() may
have to do with the thread safety of the proxies. If you replace
Thread(...) by self.Process(...) in that test then the problem may go away.
Changes by Adam Olsen [EMAIL PROTECTED]:
--
nosy: +Rhamphoryncus
___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3088
___
___
Python-bugs-list mailing
Paul Melis [EMAIL PROTECTED] added the comment:
I made a copy of test_multiprocessing.py (to test_mp.py) and basically
removed all test classes, except _TestCondition. In it, I commented all
test methods except test_notify_all. When run with make test
TESTOPTS=-v test_mp there's the lockups
Jesse Noller [EMAIL PROTECTED] added the comment:
FWIW: In order to boost the logging level within the test(s) do the
following:
Search for LOG_LEVEL, set it to:
LOG_LEVEL=util.SUBDEBUG
And then in the main() replace:
multiprocessing.get_logger().setLevel(LOG_LEVEL)
With:
Jesse Noller [EMAIL PROTECTED] added the comment:
I also isolated the test(s) like Paul did, and it looks like a semi-
consistent lock up in:
File /root/py/python-3000/Lib/multiprocessing/queues.py, line 337, in
get
racquire()
This is running only the test_event test. The racquire traces
Benjamin Peterson [EMAIL PROTECTED] added the comment:
It passes for me on Leopard. Can you post the test running in verbose
mode so we can see where it hangs?
--
nosy: +benjamin.peterson
___
Python tracker [EMAIL PROTECTED]
Jesse Noller [EMAIL PROTECTED] added the comment:
On python-3000 trunk, _multiprocessing doesn't even compile:
/Users/jesse/open_source/subversion/python-
3000/Modules/_multiprocessing/semaphore.c: In function ‘semlock_iszero’:
/Users/jesse/open_source/subversion/python-
Barry A. Warsaw [EMAIL PROTECTED] added the comment:
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On Jun 12, 2008, at 9:02 AM, Benjamin Peterson wrote:
It passes for me on Leopard. Can you post the test running in verbose
mode so we can see where it hangs?
It never hangs when run
Jesse Noller [EMAIL PROTECTED] added the comment:
I did a make clean ./configure make and it started compiling for me
again. Sorry for the noise.
___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3088
___
Guido van Rossum [EMAIL PROTECTED] added the comment:
If it's only failing during the second run of make test, typically
there's some implicit dependency on something that is disturbed by
running a test that's later in the suite of tests. This could be either
the fault of that other test (not
Guido van Rossum [EMAIL PROTECTED] added the comment:
I should add this was in the trunk (2.6).
___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3088
___
___
Paul Melis [EMAIL PROTECTED] added the comment:
I think I'm having a similar lockup on fedora core 4 (smp machine). This
is with the py3k branch, freshly svn updated. When running make test
TESTOPTS=test_multiprocessing the first of the two test runs always
succeeds in something like 10-15
Paul Melis [EMAIL PROTECTED] added the comment:
After a few more runs with -v and redirecting output to a file it seems
the lockup I get is in
test_notify_all (test.test_multiprocessing.WithManagerTestCondition)
___
Python tracker [EMAIL PROTECTED]
Jesse Noller [EMAIL PROTECTED] added the comment:
It's taking me longer to get to this than I planned, any help is
appreciated.
___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3088
___
Jesse Noller [EMAIL PROTECTED] added the comment:
I can get an intermittent (1 every 15 or so runs) lock in:
test_get (__main__.WithProcessesTestQueue) ...
Executed like this:
./python Lib/test/test_multiprocessing.py
When I control-c it the stack looks like this:
...snip
File
Changes by Jesse Noller [EMAIL PROTECTED]:
--
nosy: +roudkerk
___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3088
___
___
Python-bugs-list mailing list
New submission from Barry A. Warsaw [EMAIL PROTECTED]:
For me, test_multiprocessing hangs consistently on OS X 10.5.3. It
passes just fine on Ubuntu 8.04.
--
components: Library (Lib)
messages: 68053
nosy: barry
priority: release blocker
severity: normal
status: open
title:
Ismail Donmez [EMAIL PROTECTED] added the comment:
I can confirm this on Leopard too.
--
nosy: +cartman
___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue3088
___
64 matches
Mail list logo
