All right, it picked my curiosity so I looked into it.
There was fundamentally nothing wrong with your plugin (it does work) however
none of the hooks were being called.
I could see the FaultHandlerPlugin class was being instanciated during the
pytest_configure hook but then None of the hooks
were called afterwards.
Somehow (and I don't know why), registering the plugin as 'timeout' instead of
'pytest_timeout' was the cause of the problem,
so for it to work on my side, I had to do:
config.pluginmanager.register(FaultHandlerPlugin(config), 'pytest_timeout')
as opposed to
config.pluginmanager.register(FaultHandlerPlugin(config), 'timeout')
I also played with the ability to use markers to override the default timeout
at the test level (here is an excerpt of
your code modified)
class FaultHandlerPlugin(object):
"""The timeout plugin"""
def __init__(self, config):
self.config = config
self._current_timer = None
self.cancel = None
self._default_timeout = self.config.getvalue('timeout') # Default time
outset during cfg - Laurent
self._timeout = self._default_timeout # Current
timeout value
@property
def timeout(self):
return self._timeout
@timeout.setter
def timeout(self, value):
""" Overrides the current timeout - Laurent """
self._timeout = value
def reset_timeout(self):
""" Resets the timeout to default config value"""
self.timeout = self._default_timeout
def pytest_runtest_setup(self, item):
# Attempt to retrieve a timeout override from the timeout marker
# Example:
# @pytest.mark.timeout(3)
# def test_something():
# ...
try:
if isinstance(item, item.Function) and hasattr(item.obj, 'timeout'):
self.timeout = getattr(item.obj, 'timeout').args[0]
except Exception:
pass
...
def pytest_runtest_teardown(self):
"""Cancel the timeout trigger"""
# When skipping is raised from a pytest_runtest_setup function
# (as is the case when using the pytest.mark.skipif marker) we
# may be called without our setup counterpart having been
# called. Hence the test for self.cancel.
if self.cancel:
self.cancel()
self.cancel = None
self.reset_timeout() # reset the timeout to default config value.
Here is a sample of tests "working" with the timeout.
import sys
import pytest
import time
def test_time_out():
""" Should timeout only with the default timeout. """
counter = 0
while(1):
print "hanging ...", counter
counter += 1
time.sleep(1)
class TestTimeout():
@pytest.mark.timeout(2)
def test_time_out_2(self):
""" Timeout should be overriden """
counter = 0
while(1):
print "hanging ...", counter
counter += 1
time.sleep(1)
assert counter <= 3
@pytest.mark.timeout(5)
def test_time_out_5(self):
""" Timeout should be overriden """
counter = 0
while(1):
print "hanging ...", counter
counter += 1
time.sleep(1)
assert counter <= 6
And the output is (I know know, I should have use funcargs but I was lazy :))
def test_time_out():
""" Should timeout only with the default timeout. """
counter = 0
while(1):
print "hanging ...", counter
counter += 1
> time.sleep(1)
E Failed: Timeout >8s
test/test_sample.py:14: Failed
_________________________________________________________________________________________
TestTimeout.test_time_out_2
_________________________________________________________________________________________
self = <test_sample.TestTimeout instance at 0x90404ac>
@pytest.mark.timeout(2)
def test_time_out_2(self):
""" Timeout should be overriden """
counter = 0
while(1):
print "hanging ...", counter
counter += 1
> time.sleep(1)
E Failed: Timeout >2s
test/test_sample.py:25: Failed
________________________________________________________________________________________
TestTimeout.test_time_out_5
_________________________________________________________________________________________
self = <test_sample.TestTimeout instance at 0x904086c>
@pytest.mark.timeout(5)
def test_time_out_5(self):
""" Timeout should be overriden """
counter = 0
while(1):
print "hanging ...", counter
counter += 1
> time.sleep(1)
E Failed: Timeout >5s
Cheers
/Laurent
-----Original Message-----
From: py-dev-boun...@codespeak.net on behalf of Brack, Laurent P.
Sent: Wed 3/7/2012 10:36 AM
To: Floris Bruynooghe
Cc: Brard, Valentin; py-dev@codespeak.net
Subject: Re: [py-dev] New plugin: pytest-timeout
Hello Floris,
Thanks for your prompt reply (and sorry for my delayed one :).
I haven't had time to play more with the plugin but my observations (and
Valentin in CC) has been that neither method was working for us.
I haven't done my homework in terms of investigating this further (but I will
next week) however, to answer your question the infinite loop looked like this:
------------------------------------------
import time
def test_time_out():
counter = 0
while(1):
print "hanging ...", counter
counter += 1
time.sleep(1)
------------------------------------------
Neither with or without sigalarm worked (I am running on Ubuntu as I mentioned
and Valentine tried it on Windows).
Based on the code above, I think that both methods should work in that case.
Don't worry about it as I will try to figure out
what is going on (unless you can think of something). It is however a very nice
addition to pytest, as in early development
stages things do have a tendency to hang.
Thanks
/Laurent
-----Original Message-----
From: floris.bruynoo...@gmail.com on behalf of Floris Bruynooghe
Sent: Wed 2/29/2012 2:31 AM
To: Brack, Laurent P.
Cc: py-dev@codespeak.net
Subject: Re: [py-dev] New plugin: pytest-timeout
Hello Laurent,
On 28 February 2012 18:08, Brack, Laurent P. <lpb...@dolby.com> wrote:
> Just wrote an infinite loop which never got preempted. I didn't have time to
> dig any deeper. Does the test need to be executed in a separate process via
> xdist?
xdist is not needed, in fact I haven't tried it but assume it will
behave fine. By default SIGALM will be used which does have it's
limitations, the main issue is that it needs a chance to deliver the
signal to python code. So depending on how your infinite loop is
written this might be an issue and you may have to consider
--nosigalrm which will use a timer thread (this is more expensive
obviously). Could you show me the test function you where trying out?
> Also, currently the time out applies to all tests (and defaults to 5
> minutes). I think it would be useful to be able to override the timeout
> at the test level using markers. If something goes wrong and we have
> thousands of tests (we use pytest to test embedded systems), this is an
> awful lot of time to realize that things are DOA.
Yes, I agree with this and it is noted in the TODO file that
individual tests should have some control on the timeout and the
mechanism used (timer vs sigalrm). But we only have about 10000 tests
running nightly with normally none blocking, this plugin was written
on the spur of the moment when we had about a dozen hanging and this
minimal functionality seemed to do the job. It does assume hanging
tests are an exception rather then the norm, certainly evident from
the behaviour of --nosigalrm where os._exit() is called on a hanging
test.
I'm happy to work with you on adding a marker which will fulfil your
needs however.
Regards,
Floris
--
Debian GNU/Linux -- The Power of Freedom
www.debian.org | www.gnu.org | www.kernel.org
_______________________________________________
py-dev mailing list
py-dev@codespeak.net
http://codespeak.net/mailman/listinfo/py-dev
