[issue11727] Add a --timeout option to regrtest.py using the faulthandler module

2011-04-04 Thread STINNER Victor 

STINNER Victor victor.stin...@haypocalc.com added the comment:

regrtest default timeout is now 30 minutes.

I opened specific issues for each failure:

 * test_sendall_interrupted() of test_socket: issue #11753
 * test_itimer_real() of test_signal: issue #11755
 * test_threadsignals: issue #11768
 * test_notify() of test_threading: issue #11769

If you get new timeout issue, it's now better to open a new specific issue.

--
resolution:  - fixed
status: open - closed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11727
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11727] Add a --timeout option to regrtest.py using the faulthandler module

2011-04-03 Thread Roundup Robot 

Roundup Robot devnull@devnull added the comment:

New changeset 394f0ea0d29e by Victor Stinner in branch 'default':
Issue #11727, issue #11753, issue #11755: disable regrtest timeout
http://hg.python.org/cpython/rev/394f0ea0d29e

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11727
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11727] Add a --timeout option to regrtest.py using the faulthandler module

2011-04-01 Thread Roundup Robot 

Roundup Robot devnull@devnull added the comment:

New changeset 15f6fe139181 by Victor Stinner in branch 'default':
Issue #11727: set regrtest default timeout to 15 minutes
http://hg.python.org/cpython/rev/15f6fe139181

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11727
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11727] Add a --timeout option to regrtest.py using the faulthandler module

2011-04-01 Thread STINNER Victor 

STINNER Victor victor.stin...@haypocalc.com added the comment:

Great! The timeout works:
-
...
[ 25/354] test_threadsignals
Thread 0xa000ed88:
  File 
/Users/buildbot/buildarea/3.x.parc-tiger-1/build/Lib/test/test_threadsignals.py,
 line 46 in test_signals
  File /Users/buildbot/buildarea/3.x.parc-tiger-1/build/Lib/unittest/case.py, 
line 387 in _executeTestPart
  File /Users/buildbot/buildarea/3.x.parc-tiger-1/build/Lib/unittest/case.py, 
line 442 in run
  File /Users/buildbot/buildarea/3.x.parc-tiger-1/build/Lib/unittest/case.py, 
line 494 in __call__
  File 
/Users/buildbot/buildarea/3.x.parc-tiger-1/build/Lib/unittest/suite.py, line 
105 in run
  File 
/Users/buildbot/buildarea/3.x.parc-tiger-1/build/Lib/unittest/suite.py, line 
67 in __call__
  File 
/Users/buildbot/buildarea/3.x.parc-tiger-1/build/Lib/unittest/suite.py, line 
105 in run
  File 
/Users/buildbot/buildarea/3.x.parc-tiger-1/build/Lib/unittest/suite.py, line 
67 in __call__
  File /Users/buildbot/buildarea/3.x.parc-tiger-1/build/Lib/test/support.py, 
line 1078 in run
  File /Users/buildbot/buildarea/3.x.parc-tiger-1/build/Lib/test/support.py, 
line 1166 in _run_suite
  File /Users/buildbot/buildarea/3.x.parc-tiger-1/build/Lib/test/support.py, 
line 1192 in run_unittest
  File 
/Users/buildbot/buildarea/3.x.parc-tiger-1/build/Lib/test/test_threadsignals.py,
 line 210 in test_main
  File ./Lib/test/regrtest.py, line 1032 in runtest_inner
  File ./Lib/test/regrtest.py, line 826 in runtest
  File ./Lib/test/regrtest.py, line 650 in main
  File ./Lib/test/regrtest.py, line 1607 in module
make: *** [buildbottest] Error 1
program finished with exit code 2
elapsedTime=1103.489396
-
http://www.python.org/dev/buildbot/all/builders/PPC%20Tiger%203.x/builds/1685/steps/test/logs/stdio

Compare it without the same failure without the timeout:
-
...
[106/354] test_threadsignals

command timed out: 3600 seconds without output, killing pid 1745
process killed by signal 9
program finished with exit code -1
elapsedTime=4785.840329
-
http://www.python.org/dev/buildbot/all/builders/PPC%20Tiger%203.x/builds/1684/steps/test/logs/stdio

Thanks to this timeout, the build stops after 18 minutes instead of 80 
minutes 18 minutes including the timeout of 15 minutes :-)

test_threadsignals hangs at:

class ThreadSignals(unittest.TestCase):
def test_signals(self):
signalled_all.acquire()
self.spawnSignallingThread()
signalled_all.acquire()  here
...


self.spawnSignallingThread() calls:

def send_signals():
os.kill(process_pid, signal.SIGUSR1)
os.kill(process_pid, signal.SIGUSR2)
signalled_all.release()


--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11727
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11727] Add a --timeout option to regrtest.py using the faulthandler module

2011-04-01 Thread Roundup Robot 

Roundup Robot devnull@devnull added the comment:

New changeset 053bc5ca199b by Victor Stinner in branch 'default':
Issue #11727: set regrtest default timeout to 30 minutes
http://hg.python.org/cpython/rev/053bc5ca199b

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11727
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11727] Add a --timeout option to regrtest.py using the faulthandler module

2011-04-01 Thread STINNER Victor 

STINNER Victor victor.stin...@haypocalc.com added the comment:

Failures on the default branch with a timeout of 15 minutes:

 - test_io on x86 FreeBSD 3.x
 - test_signal on x86 FreeBSD 7.2 3.x
 - test_stdin_none on sparc solaris10 gcc 3.x

I think that the tests failed because the buildbot is slow, not because of a 
hang. These tests did not fail before. So I changed the timeout from 15 minutes 
to 30 minutes.

Trace of the x86 FreeBSD 3.x failure:
--
...
[  2/354] test_io
Thread 0x081f3000:
  File /usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Lib/unittest/case.py, 
line 574 in assertRaises
  File /usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Lib/test/test_io.py, 
line 2647 in check_interrupted_write
  File /usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Lib/test/test_io.py, 
line 2669 in test_interrupted_write_buffered
  File /usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Lib/unittest/case.py, 
line 387 in _executeTestPart
  File /usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Lib/unittest/case.py, 
line 442 in run
  File /usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Lib/unittest/case.py, 
line 494 in __call__
  File 
/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Lib/unittest/suite.py, line 
105 in run
  File 
/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Lib/unittest/suite.py, line 
67 in __call__
  File 
/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Lib/unittest/suite.py, line 
105 in run
  File 
/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Lib/unittest/suite.py, line 
67 in __call__
  File /usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Lib/test/support.py, 
line 1078 in run
  File /usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Lib/test/support.py, 
line 1166 in _run_suite
  File /usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Lib/test/support.py, 
line 1192 in run_unittest
  File /usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Lib/test/test_io.py, 
line 2845 in test_main
  File ./Lib/test/regrtest.py, line 1032 in runtest_inner
  File ./Lib/test/regrtest.py, line 826 in runtest
  File ./Lib/test/regrtest.py, line 650 in main
  File ./Lib/test/regrtest.py, line 1607 in module
--

Trace of the x86 FreeBSD 7.2 3.x failure:
---
...
[ 19/354] test_signal
Thread 0x28401040:
  File 
/usr/home/db3l/buildarea/3.x.bolen-freebsd7/build/Lib/test/test_signal.py, 
line 436 in test_itimer_real
  File 
/usr/home/db3l/buildarea/3.x.bolen-freebsd7/build/Lib/unittest/case.py, line 
387 in _executeTestPart
  File 
/usr/home/db3l/buildarea/3.x.bolen-freebsd7/build/Lib/unittest/case.py, line 
442 in run
  File 
/usr/home/db3l/buildarea/3.x.bolen-freebsd7/build/Lib/unittest/case.py, line 
494 in __call__
  File 
/usr/home/db3l/buildarea/3.x.bolen-freebsd7/build/Lib/unittest/suite.py, line 
105 in run
  File 
/usr/home/db3l/buildarea/3.x.bolen-freebsd7/build/Lib/unittest/suite.py, line 
67 in __call__
  File 
/usr/home/db3l/buildarea/3.x.bolen-freebsd7/build/Lib/unittest/suite.py, line 
105 in run
  File 
/usr/home/db3l/buildarea/3.x.bolen-freebsd7/build/Lib/unittest/suite.py, line 
67 in __call__
  File /usr/home/db3l/buildarea/3.x.bolen-freebsd7/build/Lib/test/support.py, 
line 1078 in run
  File /usr/home/db3l/buildarea/3.x.bolen-freebsd7/build/Lib/test/support.py, 
line 1166 in _run_suite
  File /usr/home/db3l/buildarea/3.x.bolen-freebsd7/build/Lib/test/support.py, 
line 1192 in run_unittest
  File 
/usr/home/db3l/buildarea/3.x.bolen-freebsd7/build/Lib/test/test_signal.py, 
line 490 in test_main
  File ./Lib/test/regrtest.py, line 1032 in runtest_inner
  File ./Lib/test/regrtest.py, line 826 in runtest
  File ./Lib/test/regrtest.py, line 650 in main
  File ./Lib/test/regrtest.py, line 1607 in module
---
http://www.python.org/dev/buildbot/all/builders/x86%20FreeBSD%207.2%203.x/builds/1643/steps/test/logs/stdio

Trace of the Solaris failure:

...
[277/354] test_subprocess
[37361 refs]
[37361 refs]
[37366 refs]
[37363 refs]
[37363 refs]
[37363 refs]
[37363 refs]
[37363 refs]
[37361 refs]
[37361 refs]
[37361 refs]
[37363 refs]
[37363 refs]
[37362 refs]
[37591 refs]
[37363 refs]
[37365 refs]
[37363 refs]
[37363 refs]
[44594 refs]
[37590 refs]
[37363 refs]
[37362 refs]
[37363 refs]
[37363 refs]
[37363 refs]
[37364 refs]
[37362 refs]
.
[37364 refs]
[37362 refs]
this bit of output is from a test of stdout in a different process ...
[37362 refs]
[37362 refs]
[37590 refs]
[37590 refs]
[37363 refs]
[37361 refs]
[37361 refs]
[37361 refs]
[37361 refs]
[37363 refs]
[37372 refs]
[37372 refs]
[37372 refs]
[37363 refs]
[37372 refs]
[37363 refs]
[37372 refs]
[37363 refs]
[37372 refs]
[37363 refs]
[37372 refs]
[37363 refs]
[37372 refs]
[37363 refs]
[37372 refs]
[37363 refs]
[37372 refs]
[37363 refs]
[37372 refs]
[37363 refs]
[37372 refs]
[37363 refs]
[37372 refs]
[37363 refs]
[37364 refs]
[37365 refs]
[37361 refs]
[37361 refs]
[37361 refs]
[37361 refs]
[37362 refs]
[37364 refs]
[37363 refs]
[37368 refs]
[37366 refs]
[37368 refs]
[37366 refs]
[37363

[issue11727] Add a --timeout option to regrtest.py using the faulthandler module

2011-04-01 Thread STINNER Victor 

STINNER Victor victor.stin...@haypocalc.com added the comment:

(oh, sparc solaris10 gcc 3.x failed on test_subprocess, test_stdin_none is 
the function, not the file)

With a timeout of 30 minutes:

 - sparc solaris10 gcc 3.x doesn't fail anymore.
 - x86 FreeBSD 3.x still fails on the same test: test_io 
(test_interrupted_write_buffered)
 - x86 FreeBSD 7.2 3.x fails on another test: test_socket 
(test_sendall_interrupted)

x86 FreeBSD 7.2 3.x trace:
--
...
[201/354] test_socket
Thread 0x28401040:
  File 
/usr/home/db3l/buildarea/3.x.bolen-freebsd7/build/Lib/test/test_socket.py, 
line 729 in check_sendall_interrupted
  File 
/usr/home/db3l/buildarea/3.x.bolen-freebsd7/build/Lib/test/test_socket.py, 
line 740 in test_sendall_interrupted
  File 
/usr/home/db3l/buildarea/3.x.bolen-freebsd7/build/Lib/unittest/case.py, line 
387 in _executeTestPart
  File 
/usr/home/db3l/buildarea/3.x.bolen-freebsd7/build/Lib/unittest/case.py, line 
442 in run
  File 
/usr/home/db3l/buildarea/3.x.bolen-freebsd7/build/Lib/unittest/case.py, line 
494 in __call__
  File 
/usr/home/db3l/buildarea/3.x.bolen-freebsd7/build/Lib/unittest/suite.py, line 
105 in run
  File 
/usr/home/db3l/buildarea/3.x.bolen-freebsd7/build/Lib/unittest/suite.py, line 
67 in __call__
  File 
/usr/home/db3l/buildarea/3.x.bolen-freebsd7/build/Lib/unittest/suite.py, line 
105 in run
  File 
/usr/home/db3l/buildarea/3.x.bolen-freebsd7/build/Lib/unittest/suite.py, line 
67 in __call__
  File /usr/home/db3l/buildarea/3.x.bolen-freebsd7/build/Lib/test/support.py, 
line 1078 in run
  File /usr/home/db3l/buildarea/3.x.bolen-freebsd7/build/Lib/test/support.py, 
line 1166 in _run_suite
  File /usr/home/db3l/buildarea/3.x.bolen-freebsd7/build/Lib/test/support.py, 
line 1192 in run_unittest
  File 
/usr/home/db3l/buildarea/3.x.bolen-freebsd7/build/Lib/test/test_socket.py, 
line 2052 in test_main
  File ./Lib/test/regrtest.py, line 1032 in runtest_inner
  File ./Lib/test/regrtest.py, line 826 in runtest
  File ./Lib/test/regrtest.py, line 650 in main
  File ./Lib/test/regrtest.py, line 1607 in module
--
http://www.python.org/dev/buildbot/all/builders/x86%20FreeBSD%207.2%203.x/builds/1644/steps/test/logs/stdio

x86 FreeBSD 3.x trace:
--
...
[ 54/354] test_io
Thread 0x081f3000:
  File /usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Lib/unittest/case.py, 
line 574 in assertRaises
  File /usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Lib/test/test_io.py, 
line 2647 in check_interrupted_write
  File /usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Lib/test/test_io.py, 
line 2669 in test_interrupted_write_buffered
  File /usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Lib/unittest/case.py, 
line 387 in _executeTestPart
  File /usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Lib/unittest/case.py, 
line 442 in run
  File /usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Lib/unittest/case.py, 
line 494 in __call__
  File 
/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Lib/unittest/suite.py, line 
105 in run
  File 
/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Lib/unittest/suite.py, line 
67 in __call__
  File 
/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Lib/unittest/suite.py, line 
105 in run
  File 
/usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Lib/unittest/suite.py, line 
67 in __call__
  File /usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Lib/test/support.py, 
line 1078 in run
  File /usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Lib/test/support.py, 
line 1166 in _run_suite
  File /usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Lib/test/support.py, 
line 1192 in run_unittest
  File /usr/home/db3l/buildarea/3.x.bolen-freebsd/build/Lib/test/test_io.py, 
line 2845 in test_main
  File ./Lib/test/regrtest.py, line 1032 in runtest_inner
  File ./Lib/test/regrtest.py, line 826 in runtest
  File ./Lib/test/regrtest.py, line 650 in main
  File ./Lib/test/regrtest.py, line 1607 in module
--
http://www.python.org/dev/buildbot/all/builders/x86%20FreeBSD%203.x/builds/1343/steps/test/logs/stdio

Reopen the issue because the story is not completly finished.

--
resolution: fixed - 
status: closed - open

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11727
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11727] Add a --timeout option to regrtest.py using the faulthandler module

2011-04-01 Thread STINNER Victor 

STINNER Victor victor.stin...@haypocalc.com added the comment:

Because the timeout was 30 minutes, we missed where test_ssl hangs on x86 
Windows7 3.x:
--
...
[259/354] test_filecmp
[260/354] test_ssl

command timed out: 1200 seconds without output, killing pid 3012
SIGKILL failed to kill process
using fake rc=-1
program finished with exit code -1

remoteFailed: [Failure instance: Traceback from remote host -- Traceback (most 
recent call last):
Failure: buildbot.slave.commands.TimeoutError: SIGKILL failed to kill process
]
--
http://www.python.org/dev/buildbot/all/builders/x86%20Windows7%203.x/builds/2877/steps/test/logs/stdio

It looks like this buildbot has some network issues:

 * build 2877 failed on test_ssl
 * build 2872 failed on test_poplib
 * build 2866 failed on test_ftplib
 * build 2865 failed on test_smtplib
 * build 2864 failed on test_socket
 * ...

Note: build 2869 failed on test_multiprocessing.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11727
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11727] Add a --timeout option to regrtest.py using the faulthandler module

2011-04-01 Thread STINNER Victor 

STINNER Victor victor.stin...@haypocalc.com added the comment:

For test_subprocess timeout, see also:
http://bugs.python.org/issue8429#msg103368
For test_io timeout, see also: issue #8431

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11727
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11727] Add a --timeout option to regrtest.py using the faulthandler module

2011-04-01 Thread STINNER Victor 

STINNER Victor victor.stin...@haypocalc.com added the comment:

Timeout of 15 minutes on x86 XP-4 3.x:

...
[334/354] test_threading
Thread 0x024c:
  File 
D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\threading.py, line 
235 in wait
  File 
D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\test\lock_tests.py, 
line 392 in f
  File 
D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\test\lock_tests.py, 
line 37 in task

Thread 0x02d8:
  File 
D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\threading.py, line 
235 in wait
  File 
D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\test\lock_tests.py, 
line 392 in f
  File 
D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\test\lock_tests.py, 
line 37 in task

Thread 0x0da4:
  File 
D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\threading.py, line 
235 in wait
  File 
D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\test\lock_tests.py, 
line 392 in f
  File 
D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\test\lock_tests.py, 
line 37 in task

Thread 0x0bb0:
  File 
D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\test\lock_tests.py, 
line 16 in _wait
  File 
D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\test\lock_tests.py, 
line 416 in _check_notify
  File 
D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\test\lock_tests.py, 
line 433 in test_notify
  File 
D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\unittest\case.py, 
line 387 in _executeTestPart
  File 
D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\unittest\case.py, 
line 442 in run
  File 
D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\unittest\case.py, 
line 494 in __call__
  File 
D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\unittest\suite.py, 
line 105 in run
  File 
D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\unittest\suite.py, 
line 67 in __call__
  File 
D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\unittest\suite.py, 
line 105 in run
  File 
D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\unittest\suite.py, 
line 67 in __call__
  File 
D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\test\support.py, 
line 1078 in run
  File 
D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\test\support.py, 
line 1166 in _run_suite
  File 
D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\test\support.py, 
line 1192 in run_unittest
  File 
D:\cygwin\home\db3l\buildarea\3.x.bolen-windows\build\lib\test\test_threading.py,
 line 728 in test_main
  File ../lib/test/regrtest.py, line 1032 in runtest_inner
  File ../lib/test/regrtest.py, line 826 in runtest
  File ../lib/test/regrtest.py, line 650 in main
  File ../lib/test/regrtest.py, line 1607 in module
s_push: parser stack overflow
program finished with exit code 1
elapsedTime=2601.059000

http://www.python.org/dev/buildbot/all/builders/x86%20XP-4%203.x/builds/4317/steps/test/logs/stdio

Hum, it looks like we have 4 threads, and all threads are waiting.

= See issue #8799 and maybe also #4188 and #5114.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11727
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11727] Add a --timeout option to regrtest.py using the faulthandler module

2011-03-31 Thread STINNER Victor 

New submission from STINNER Victor victor.stin...@haypocalc.com:

I would be nice to add a timeout option to regrtest.py to be able to dump the 
traceback after TIMEOUT seconds and also exit the process. It would help 
debugging timeout issues (e.g. test_multiprocessing ), but also protect 
buildbots against evil (long) tests.

Attached patch implements this feature.

--
components: Tests
files: regrtest_timeout.patch
keywords: buildbot, patch
messages: 132643
nosy: haypo
priority: normal
severity: normal
status: open
title: Add a --timeout option to regrtest.py using the faulthandler module
versions: Python 3.3
Added file: http://bugs.python.org/file21488/regrtest_timeout.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11727
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11727] Add a --timeout option to regrtest.py using the faulthandler module

2011-03-31 Thread STINNER Victor 

STINNER Victor victor.stin...@haypocalc.com added the comment:

I created a new Mercurial repository to test this issue: 
features/regrtest_timeout.

Let's try it with a timeout of 5 minutes on:
 - x86 Tiger custom to learn more about test_threadsignals timeout (1 hour) 
= issue #11223
 - x86 FreeBSD 7.2 3.x to learn more about test_multiprocessing failures 
(timeout of 1 hour sometimes) = issue #8428

http://www.python.org/dev/buildbot/all/builders/x86%20Tiger%20custom/builds/2
http://www.python.org/dev/buildbot/all/builders/x86%20FreeBSD%207.2%20custom/builds/3

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11727
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11727] Add a --timeout option to regrtest.py using the faulthandler module

2011-03-31 Thread STINNER Victor 

STINNER Victor victor.stin...@haypocalc.com added the comment:

Try the branch on PPC Leopard 3.x (test_subprocess timeout):

http://www.python.org/dev/buildbot/all/builders/PPC%20Leopard%20custom/builds/6

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11727
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11727] Add a --timeout option to regrtest.py using the faulthandler module

2011-03-31 Thread Antoine Pitrou 

Antoine Pitrou pit...@free.fr added the comment:

Just a nit but:
if timeout and 0  timeout

Could you use the usual coding style? (if timeout and timeout  0)

Putting constants on the left-side of comparisons is pointless in Python, and 
even in C it's quite ugly and surprising.

--
nosy: +pitrou

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11727
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11727] Add a --timeout option to regrtest.py using the faulthandler module

2011-03-31 Thread Antoine Pitrou 

Antoine Pitrou pit...@free.fr added the comment:

 x86 Tiger custom to learn more about test_threadsignals timeout 
 (1 hour)

Please read the comments. There is no timeout anymore.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11727
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11727] Add a --timeout option to regrtest.py using the faulthandler module

2011-03-31 Thread STINNER Victor 

STINNER Victor victor.stin...@haypocalc.com added the comment:

 x86 Tiger custom to learn more about test_threadsignals timeout 
 (1 hour)

 Please read the comments. There is no timeout anymore.

The last build failed with:
---
...
[213/354] test_plistlib
[214/354] test_minidom
[215/354] test_threadsignals

command timed out: 3600 seconds without output, killing pid 24165
process killed by signal 9
---
http://www.python.org/dev/buildbot/all/builders/x86%20Tiger%203.x/builds/2262/steps/test/logs/stdio

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11727
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11727] Add a --timeout option to regrtest.py using the faulthandler module

2011-03-31 Thread STINNER Victor 

STINNER Victor victor.stin...@haypocalc.com added the comment:

I ran the full test suite on x86 Tiger custom on my features/regrtest_timeout 
repository: no failure (test_threadsignals failure was not reproduced).

On PPC Leopard custom, the test suite was interrupted (timeout) on 
test_io.test_basic_io:

...
[ 53/354] test_xdrlib
[ 54/354] test_descr
[ 55/354] test_io
Current thread 0xa09e1820:
  File /Users/buildbot/buildarea/custom.parc-leopard-1/build/Lib/codecs.py, 
line 302 in decode
  File /Users/buildbot/buildarea/custom.parc-leopard-1/build/Lib/_pyio.py, 
line 1378 in decode
  File /Users/buildbot/buildarea/custom.parc-leopard-1/build/Lib/_pyio.py, 
line 1682 in _read_chunk
  File /Users/buildbot/buildarea/custom.parc-leopard-1/build/Lib/_pyio.py, 
line 2002 in readline
  File 
/Users/buildbot/buildarea/custom.parc-leopard-1/build/Lib/test/test_io.py, 
line 1964 in multi_line_test
  File 
/Users/buildbot/buildarea/custom.parc-leopard-1/build/Lib/test/test_io.py, 
line 1945 in test_basic_io
  File 
/Users/buildbot/buildarea/custom.parc-leopard-1/build/Lib/unittest/case.py, 
line 387 in _executeTestPart
  File 
/Users/buildbot/buildarea/custom.parc-leopard-1/build/Lib/unittest/case.py, 
line 442 in run
  File 
/Users/buildbot/buildarea/custom.parc-leopard-1/build/Lib/unittest/case.py, 
line 494 in __call__
  File 
/Users/buildbot/buildarea/custom.parc-leopard-1/build/Lib/unittest/suite.py, 
line 105 in run
  File 
/Users/buildbot/buildarea/custom.parc-leopard-1/build/Lib/unittest/suite.py, 
line 67 in __call__
  File 
/Users/buildbot/buildarea/custom.parc-leopard-1/build/Lib/unittest/suite.py, 
line 105 in run
  File 
/Users/buildbot/buildarea/custom.parc-leopard-1/build/Lib/unittest/suite.py, 
line 67 in __call__
  File 
/Users/buildbot/buildarea/custom.parc-leopard-1/build/Lib/test/support.py, 
line 1078 in run
  File 
/Users/buildbot/buildarea/custom.parc-leopard-1/build/Lib/test/support.py, 
line 1166 in _run_suite
  File 
/Users/buildbot/buildarea/custom.parc-leopard-1/build/Lib/test/support.py, 
line 1192 in run_unittest
  File 
/Users/buildbot/buildarea/custom.parc-leopard-1/build/Lib/test/test_io.py, 
line 2845 in test_main
  File ./Lib/test/regrtest.py, line 1029 in runtest_inner
  File ./Lib/test/regrtest.py, line 823 in runtest
  File ./Lib/test/regrtest.py, line 649 in main
  File ./Lib/test/regrtest.py, line 1604 in module
make: *** [buildbottest] Error 1
program finished with exit code 2
---
http://www.python.org/dev/buildbot/all/builders/PPC%20Leopard%20custom/builds/6

Cool! The traceback was printed correctly, the process exited correctly , and 
the build stopped also correctly. The full (interrupted) build took 27 min 41 
sec, it takes usually 56 min 44 sec.

So a timeout of 5 minutes on each test (each function of each test file) is not 
enough on PPC Leopard buildslave. Anyway, I plan to use something closer to the 
actual buildbot timeout (1200 or 3600 seconds). And it would be better if the 
buildbot can choose its own regrtest.py timeout.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11727
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11727] Add a --timeout option to regrtest.py using the faulthandler module

2011-03-31 Thread STINNER Victor 

STINNER Victor victor.stin...@haypocalc.com added the comment:

Result on x86 FreeBSD 7.2 custom:

...
[148/354] test_fileinput
[149/354] test_subprocess
[36645 refs]
[36645 refs]
...
[36647 refs]
[44034 refs]
Current thread 0x28401040:
  File 
/usr/home/db3l/buildarea/custom.bolen-freebsd7/build/Lib/subprocess.py, line 
1282 in _execute_child
  File 
/usr/home/db3l/buildarea/custom.bolen-freebsd7/build/Lib/subprocess.py, line 
766 in __init__
  File 
/usr/home/db3l/buildarea/custom.bolen-freebsd7/build/Lib/test/test_subprocess.py,
 line 687 in test_leaking_fds_on_error
  File 
/usr/home/db3l/buildarea/custom.bolen-freebsd7/build/Lib/unittest/case.py, 
line 387 in _executeTestPart
  File 
/usr/home/db3l/buildarea/custom.bolen-freebsd7/build/Lib/unittest/case.py, 
line 442 in run
  File 
/usr/home/db3l/buildarea/custom.bolen-freebsd7/build/Lib/unittest/case.py, 
line 494 in __call__
  File 
/usr/home/db3l/buildarea/custom.bolen-freebsd7/build/Lib/unittest/suite.py, 
line 105 in run
  File 
/usr/home/db3l/buildarea/custom.bolen-freebsd7/build/Lib/unittest/suite.py, 
line 67 in __call__
  File 
/usr/home/db3l/buildarea/custom.bolen-freebsd7/build/Lib/unittest/suite.py, 
line 105 in run
  File 
/usr/home/db3l/buildarea/custom.bolen-freebsd7/build/Lib/unittest/suite.py, 
line 67 in __call__
  File 
/usr/home/db3l/buildarea/custom.bolen-freebsd7/build/Lib/test/support.py, 
line 1078 in run
  File 
/usr/home/db3l/buildarea/custom.bolen-freebsd7/build/Lib/test/support.py, 
line 1166 in _run_suite
  File 
/usr/home/db3l/buildarea/custom.bolen-freebsd7/build/Lib/test/support.py, 
line 1192 in run_unittest
  File 
/usr/home/db3l/buildarea/custom.bolen-freebsd7/build/Lib/test/test_subprocess.py,
 line 1607 in test_main
  File ./Lib/test/regrtest.py, line 1029 in runtest_inner
  File ./Lib/test/regrtest.py, line 823 in runtest
  File ./Lib/test/regrtest.py, line 649 in main
  File ./Lib/test/regrtest.py, line 1604 in module
program finished with exit code 1
elapsedTime=2100.157408

http://www.python.org/dev/buildbot/all/builders/x86%20FreeBSD%207.2%20custom/builds/3

So test_subprocess does timeout (5 min) because of test_leaking_fds_on_error, 
it may be related to #8052. subprocess.MAXFD=655000 on FreeBSD: create a 
subprocess requires to call close() system calls 655,000 times and 
test_leaking_fds_on_error() creates 1024 subprocesses.

There is something interesting in this output: the test uses a subprocess and 
we only have the traceback of the parent. It may be nice to have the trace of 
the child process. It might be possible by sending a signal to the child 
process (but how can we get the list of the child processes in a C signal 
handler?).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11727
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11727] Add a --timeout option to regrtest.py using the faulthandler module

2011-03-31 Thread Arfrever Frehtes Taifersar Arahesis 

Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com:


--
nosy: +Arfrever

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11727
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11727] Add a --timeout option to regrtest.py using the faulthandler module

2011-03-31 Thread Charles-Francois Natali 

Charles-Francois Natali neolo...@free.fr added the comment:

 There is something interesting in this output: the test uses a subprocess and 
 we only have the traceback of the parent. It may be nice to have the trace of 
 the child process. It might be possible by sending a signal to the child 
 process (but how can we get the list of the child processes in a C signal 
 handler?).

I don't think you can find that, but you could send a signal to the whole 
process group:
if (getpgrp() == getpid()) {
kill(-getpgrp(), signal);
}

The getpgrp() == getpid() makes sure that you'll only do that if the current 
process is the group leader (and it's async-safe). You'll probably want to 
block signal in the parent's handler first.
Note that it won't work if your child process calls setsid(), of course 
(start_new_session argument to Popen).

--
nosy: +neologix

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11727
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11727] Add a --timeout option to regrtest.py using the faulthandler module

2011-03-31 Thread Roundup Robot 

Roundup Robot devnull@devnull added the comment:

New changeset bdc946dc512a by Victor Stinner in branch 'default':
Issue #11727: add --timeout option to regrtest (disabled by default).
http://hg.python.org/cpython/rev/bdc946dc512a

--
nosy: +python-dev

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11727
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11727] Add a --timeout option to regrtest.py using the faulthandler module

2011-03-31 Thread Roundup Robot 

Roundup Robot devnull@devnull added the comment:

New changeset 0f3dbf93ac06 by Victor Stinner in branch 'default':
Issue #11727: Antoine and Arfrever don't like 0  timeout style (regrtest.py)
http://hg.python.org/cpython/rev/0f3dbf93ac06

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11727
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11727] Add a --timeout option to regrtest.py using the faulthandler module

2011-03-31 Thread STINNER Victor 

STINNER Victor victor.stin...@haypocalc.com added the comment:

Patch commited into 3.3 (bdc946dc512a, 0f3dbf93ac06). I opened a thread on 
python-dev to decide if we turn this on by default on buildbots.

--
resolution:  - fixed
status: open - closed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11727
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com