subject:"Test fails when run in whole test suite \- but not stand\-alone\?"

Re: Test fails when run in whole test suite - but not stand-alone?

2016-06-25 Thread Bill Freeman

I have no immediate clue.

I know that, using nosetest, I can add -s and -v to the command line,
making it possible to drive pdb from the test, using:

import pdb;pdb.set_trace()

inserted in the code to get into pdb at the relevant point(s).

Evan if you're not running under nosetest, there may be an equivalent to -s
-v .

Clearly something interesting is going on.  If the framework uses a new
process for the second test, then the import would actually happen twice,
but the environment should be pristine, unless you have left crud in a file
or database that is written during the test.

If the same process is used, the second import doesn't actually import, but
just gives you a reference to the already loaded module (at least that's
the way it is in python 2.x, where I still live).

I would start with a set_trace() at the top of the imported module, to see
if it is reached before the failure.  If it is, you can single step along
in hope of insight.

On Sat, Jun 25, 2016 at 5:17 AM, Derek  wrote:

> Hi Adam
>
> I have narrowed the issue right down.  As soon as I have even *one* other
> test that imports *anything* from the app's admin file, the test crashes.
> So for example, if the test sequence is group of test files - one of which
> is my one in the OP and one which contains:
>
> import unittest
> from trees.admin import AnyModelAdmin
>
> class TestDummy(unittest.TestCase):
>
> def setUp(self):
> pass
>
> def test_dummy(self):
> pass
>
> def tearDown(self):
> pass
>
> Then I get errors.  Do you know why an import statement would interfere
> with a test?
>
> Thanks,
> Derek
>
>
>
> On Thursday, 2 June 2016 21:47:00 UTC+2, Adam wrote:
>>
>> When I've had that happen before, it's because some previous test changed
>> something (like a setting value or the site domain) that influenced the
>> test that failed. To narrow it down, I would run all the tests up to and
>> including the failed one. That should fail, then I start taking out half
>> the tests before. If the test still fails, I keep cutting in half until I
>> can determine which previous test is causing issues. If, after cutting out
>> tests, the problem test passes, then I need to put back in what I took out
>> since it was one of those.
>>
>> Once I figure out which previous test it was, I can start removing the
>> individual tests to finally get to the code causing the problem. Usually,
>> it's a case that a test changed something and I just have to add in the
>> teardown function to restore the state of whatever was changed.
>>
>> On Thu, 2016-06-02 at 21:33 +0200, Derek wrote:
>>
>> I have a test that is failing when the file it is in is run as part of
>> all the other test files by the test runner.
>>
>> If I just run only the file that contains this test -  then it passes.
>>
>> (pass/fail refers to the very last assert in the code below.)
>>
>> I'd appreciate any ideas or insights from anyone who can spot an obvious
>> mistake - or suggest some options to explore.
>>
>> Thanks
>> Derek
>>
>>
>> # THIS IS AN EXTRACT OF RELEVANT CODE (not all of it...)
>> from django.contrib.messages.storage.fallback import FallbackStorage
>> from django.core.urlresolvers import reverse
>> from django.test import TestCase, Client
>> # ... various app-related imports ...
>>
>>
>> class MockRequest(object):
>> """No code needed."""
>> pass
>>
>>
>> REQUEST = MockRequest()
>> # see: https://stackoverflow.com/queI
>> stions/11938164/why-dont-my-django-\
>> #  unittests-know-that-messagemiddleware-is-installed
>> setattr(REQUEST, 'session', 'session')
>> MESSAGES = FallbackStorage(REQUEST)
>> setattr(REQUEST, '_messages', MESSAGES)
>> setup_test_environment()
>>
>>
>> class PersonAdminTest(TestCase):
>>
>> def setUp(self):
>> self.user, password = utils.user_factory(model=None)
>> self.client = Client()
>> login_status = self.client.login(username=self.user.email,
>> password=password)
>> self.assertEqual(login_status, True)
>>
>> def test_action_persons_make_unreal(self):
>> try:
>> change_url = reverse('admin:persons_realpersons_changelist')
>> except NoReverseMatch:
>> change_url = '/admin/persons/realpersons/'
>> sys.stderr.write('\n   WARNING: Unable to reverse URL! ... ')
>> response = self.client.get(change_url)
>> self.assertEqual(response.status_code, 200)
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-users...@googlegroups.com.
>> To post to this group, send email to django...@googlegroups.com.
>> Visit this group at https://groups.google.com/group/django-users.
>> To view this discussion on the web visit
>>

Re: Test fails when run in whole test suite - but not stand-alone?

2016-06-25 Thread Derek

Hi Adam

I have narrowed the issue right down.  As soon as I have even *one* other 
test that imports *anything* from the app's admin file, the test crashes. 
So for example, if the test sequence is group of test files - one of which 
is my one in the OP and one which contains:

import unittest
from trees.admin import AnyModelAdmin

class TestDummy(unittest.TestCase):

def setUp(self):
pass

def test_dummy(self):
pass

def tearDown(self):
pass

Then I get errors.  Do you know why an import statement would interfere 
with a test?

Thanks,
Derek



On Thursday, 2 June 2016 21:47:00 UTC+2, Adam wrote:
>
> When I've had that happen before, it's because some previous test changed 
> something (like a setting value or the site domain) that influenced the 
> test that failed. To narrow it down, I would run all the tests up to and 
> including the failed one. That should fail, then I start taking out half 
> the tests before. If the test still fails, I keep cutting in half until I 
> can determine which previous test is causing issues. If, after cutting out 
> tests, the problem test passes, then I need to put back in what I took out 
> since it was one of those.
>
> Once I figure out which previous test it was, I can start removing the 
> individual tests to finally get to the code causing the problem. Usually, 
> it's a case that a test changed something and I just have to add in the 
> teardown function to restore the state of whatever was changed.
>
> On Thu, 2016-06-02 at 21:33 +0200, Derek wrote:
>
> I have a test that is failing when the file it is in is run as part of all 
> the other test files by the test runner.
>
> If I just run only the file that contains this test -  then it passes.
>
> (pass/fail refers to the very last assert in the code below.)
>
> I'd appreciate any ideas or insights from anyone who can spot an obvious 
> mistake - or suggest some options to explore.
>
> Thanks
> Derek
>
>
> # THIS IS AN EXTRACT OF RELEVANT CODE (not all of it...)
> from django.contrib.messages.storage.fallback import FallbackStorage
> from django.core.urlresolvers import reverse
> from django.test import TestCase, Client
> # ... various app-related imports ...
>
>
> class MockRequest(object):
> """No code needed."""
> pass
>
>
> REQUEST = MockRequest()
> # see: https://stackoverflow.com/queI stions/11938164/why-dont-my-django-\
> #  unittests-know-that-messagemiddleware-is-installed
> setattr(REQUEST, 'session', 'session')
> MESSAGES = FallbackStorage(REQUEST)
> setattr(REQUEST, '_messages', MESSAGES)
> setup_test_environment()
>
>
> class PersonAdminTest(TestCase):
>
> def setUp(self):
> self.user, password = utils.user_factory(model=None)
> self.client = Client()
> login_status = self.client.login(username=self.user.email, 
> password=password)
> self.assertEqual(login_status, True)
>
> def test_action_persons_make_unreal(self):
> try:
> change_url = reverse('admin:persons_realpersons_changelist')
> except NoReverseMatch:
> change_url = '/admin/persons/realpersons/'
> sys.stderr.write('\n   WARNING: Unable to reverse URL! ... ')
> response = self.client.get(change_url)
> self.assertEqual(response.status_code, 200)
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-users...@googlegroups.com .
> To post to this group, send email to django...@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-users/CAF1Wu3MoGgm_dSToObDX9gH7GQJ%3DTZzGLV7RPOiZk4kvV-_Nbg%40mail.gmail.com
>  
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>
> -- 
> Adam (ad...@csh.rit.edu )
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/cbebca39-4446-4ffa-93aa-c058861bdc22%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Test fails when run in whole test suite - but not stand-alone?

2016-06-08 Thread learn django

Hi Adam,

I was hitting same issue but never got chance to debug it.
If the underneath database support database transactions then isn't that
each test will be treated as a separate transaction and transaction should 
be rolled back
after the test is over to keep the database sane ?

On Wednesday, June 8, 2016 at 1:43:40 AM UTC-7, Derek wrote:
>
> Thanks Adam, I will try that.  I think I have been lax in the use of 
> tearDown.
>
> On Thursday, 2 June 2016 21:47:00 UTC+2, Adam wrote:
>>
>> When I've had that happen before, it's because some previous test changed 
>> something (like a setting value or the site domain) that influenced the 
>> test that failed. To narrow it down, I would run all the tests up to and 
>> including the failed one. That should fail, then I start taking out half 
>> the tests before. If the test still fails, I keep cutting in half until I 
>> can determine which previous test is causing issues. If, after cutting out 
>> tests, the problem test passes, then I need to put back in what I took out 
>> since it was one of those.
>>
>> Once I figure out which previous test it was, I can start removing the 
>> individual tests to finally get to the code causing the problem. Usually, 
>> it's a case that a test changed something and I just have to add in the 
>> teardown function to restore the state of whatever was changed.
>>
>> On Thu, 2016-06-02 at 21:33 +0200, Derek wrote:
>>
>> I have a test that is failing when the file it is in is run as part of 
>> all the other test files by the test runner.
>>
>> If I just run only the file that contains this test -  then it passes.
>>
>> (pass/fail refers to the very last assert in the code below.)
>>
>> I'd appreciate any ideas or insights from anyone who can spot an obvious 
>> mistake - or suggest some options to explore.
>>
>> Thanks
>> Derek
>>
>>
>> # THIS IS AN EXTRACT OF RELEVANT CODE (not all of it...)
>> from django.contrib.messages.storage.fallback import FallbackStorage
>> from django.core.urlresolvers import reverse
>> from django.test import TestCase, Client
>> # ... various app-related imports ...
>>
>>
>> class MockRequest(object):
>> """No code needed."""
>> pass
>>
>>
>> REQUEST = MockRequest()
>> # see: https://stackoverflow.com/queI 
>> stions/11938164/why-dont-my-django-\
>> #  unittests-know-that-messagemiddleware-is-installed
>> setattr(REQUEST, 'session', 'session')
>> MESSAGES = FallbackStorage(REQUEST)
>> setattr(REQUEST, '_messages', MESSAGES)
>> setup_test_environment()
>>
>>
>> class PersonAdminTest(TestCase):
>>
>> def setUp(self):
>> self.user, password = utils.user_factory(model=None)
>> self.client = Client()
>> login_status = self.client.login(username=self.user.email, 
>> password=password)
>> self.assertEqual(login_status, True)
>>
>> def test_action_persons_make_unreal(self):
>> try:
>> change_url = reverse('admin:persons_realpersons_changelist')
>> except NoReverseMatch:
>> change_url = '/admin/persons/realpersons/'
>> sys.stderr.write('\n   WARNING: Unable to reverse URL! ... ')
>> response = self.client.get(change_url)
>> self.assertEqual(response.status_code, 200)
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-users...@googlegroups.com.
>> To post to this group, send email to django...@googlegroups.com.
>> Visit this group at https://groups.google.com/group/django-users.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-users/CAF1Wu3MoGgm_dSToObDX9gH7GQJ%3DTZzGLV7RPOiZk4kvV-_Nbg%40mail.gmail.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>> -- 
>> Adam (ad...@csh.rit.edu)
>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/ff76d73a-3baf-4053-b539-7734ba53e61d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Test fails when run in whole test suite - but not stand-alone?

2016-06-08 Thread Derek

Thanks Adam, I will try that.  I think I have been lax in the use of 
tearDown.

On Thursday, 2 June 2016 21:47:00 UTC+2, Adam wrote:
>
> When I've had that happen before, it's because some previous test changed 
> something (like a setting value or the site domain) that influenced the 
> test that failed. To narrow it down, I would run all the tests up to and 
> including the failed one. That should fail, then I start taking out half 
> the tests before. If the test still fails, I keep cutting in half until I 
> can determine which previous test is causing issues. If, after cutting out 
> tests, the problem test passes, then I need to put back in what I took out 
> since it was one of those.
>
> Once I figure out which previous test it was, I can start removing the 
> individual tests to finally get to the code causing the problem. Usually, 
> it's a case that a test changed something and I just have to add in the 
> teardown function to restore the state of whatever was changed.
>
> On Thu, 2016-06-02 at 21:33 +0200, Derek wrote:
>
> I have a test that is failing when the file it is in is run as part of all 
> the other test files by the test runner.
>
> If I just run only the file that contains this test -  then it passes.
>
> (pass/fail refers to the very last assert in the code below.)
>
> I'd appreciate any ideas or insights from anyone who can spot an obvious 
> mistake - or suggest some options to explore.
>
> Thanks
> Derek
>
>
> # THIS IS AN EXTRACT OF RELEVANT CODE (not all of it...)
> from django.contrib.messages.storage.fallback import FallbackStorage
> from django.core.urlresolvers import reverse
> from django.test import TestCase, Client
> # ... various app-related imports ...
>
>
> class MockRequest(object):
> """No code needed."""
> pass
>
>
> REQUEST = MockRequest()
> # see: https://stackoverflow.com/queI stions/11938164/why-dont-my-django-\
> #  unittests-know-that-messagemiddleware-is-installed
> setattr(REQUEST, 'session', 'session')
> MESSAGES = FallbackStorage(REQUEST)
> setattr(REQUEST, '_messages', MESSAGES)
> setup_test_environment()
>
>
> class PersonAdminTest(TestCase):
>
> def setUp(self):
> self.user, password = utils.user_factory(model=None)
> self.client = Client()
> login_status = self.client.login(username=self.user.email, 
> password=password)
> self.assertEqual(login_status, True)
>
> def test_action_persons_make_unreal(self):
> try:
> change_url = reverse('admin:persons_realpersons_changelist')
> except NoReverseMatch:
> change_url = '/admin/persons/realpersons/'
> sys.stderr.write('\n   WARNING: Unable to reverse URL! ... ')
> response = self.client.get(change_url)
> self.assertEqual(response.status_code, 200)
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-users...@googlegroups.com .
> To post to this group, send email to django...@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-users/CAF1Wu3MoGgm_dSToObDX9gH7GQJ%3DTZzGLV7RPOiZk4kvV-_Nbg%40mail.gmail.com
>  
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>
> -- 
> Adam (ad...@csh.rit.edu )
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/fee4ed8a-ff68-41ce-965a-500e5d8aa694%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Test fails when run in whole test suite - but not stand-alone?

2016-06-02 Thread Adam Stein

When I've had that happen before, it's because some previous test
changed something (like a setting value or the site domain) that
influenced the test that failed. To narrow it down, I would run all the
tests up to and including the failed one. That should fail, then I
start taking out half the tests before. If the test still fails, I keep
cutting in half until I can determine which previous test is causing
issues. If, after cutting out tests, the problem test passes, then I
need to put back in what I took out since it was one of those.
Once I figure out which previous test it was, I can start removing the
individual tests to finally get to the code causing the problem.
Usually, it's a case that a test changed something and I just have to
add in the teardown function to restore the state of whatever was
changed.
On Thu, 2016-06-02 at 21:33 +0200, Derek wrote:
> I have a test that is failing when the file it is in is run as part
> of all the other test files by the test runner.
> 
> If I just run only the file that contains this test -  then it
> passes.
> 
> (pass/fail refers to the very last assert in the code below.)
> 
> I'd appreciate any ideas or insights from anyone who can spot an
> obvious mistake - or suggest some options to explore.
> 
> Thanks
> Derek
> 
> 
> # THIS IS AN EXTRACT OF RELEVANT CODE (not all of it...)
> from django.contrib.messages.storage.fallback import FallbackStorage
> from django.core.urlresolvers import reverse
> from django.test import TestCase, Client
> # ... various app-related imports ...
> 
> 
> class MockRequest(object):
>     """No code needed."""
>     pass
> 
> 
> REQUEST = MockRequest()
> # see: https://stackoverflow.com/queI stions/11938164/why-dont-my-
> django-\
> #      unittests-know-that-messagemiddleware-is-installed
> setattr(REQUEST, 'session', 'session')
> MESSAGES = FallbackStorage(REQUEST)
> setattr(REQUEST, '_messages', MESSAGES)
> setup_test_environment()
> 
> 
> class PersonAdminTest(TestCase):
> 
>     def setUp(self):
>         self.user, password = utils.user_factory(model=None)
>         self.client = Client()
>         login_status = self.client.login(username=self.user.email,
> password=password)
>         self.assertEqual(login_status, True)
> 
>     def test_action_persons_make_unreal(self):
>         try:
>             change_url =
> reverse('admin:persons_realpersons_changelist')
>         except NoReverseMatch:
>             change_url = '/admin/persons/realpersons/'
>             sys.stderr.write('\n   WARNING: Unable to reverse URL!
> ... ')
>         response = self.client.get(change_url)
>         self.assertEqual(response.status_code, 200)
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Django users" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/django-users/CAF1Wu3MoGgm_dSToObDX9gH7GQJ%3DTZzGLV7RPOiZk4kvV-
> _Nbg%40mail.gmail.com.
> For more options, visit https://groups.google.com/d/optout.
-- 
Adam (a...@csh.rit.edu)


-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/1464896791.20313.13.camel%40csh.rit.edu.
For more options, visit https://groups.google.com/d/optout.

Test fails when run in whole test suite - but not stand-alone?

2016-06-02 Thread Derek

I have a test that is failing when the file it is in is run as part of all
the other test files by the test runner.

If I just run only the file that contains this test -  then it passes.

(pass/fail refers to the very last assert in the code below.)

I'd appreciate any ideas or insights from anyone who can spot an obvious
mistake - or suggest some options to explore.

Thanks
Derek


# THIS IS AN EXTRACT OF RELEVANT CODE (not all of it...)
from django.contrib.messages.storage.fallback import FallbackStorage
from django.core.urlresolvers import reverse
from django.test import TestCase, Client
# ... various app-related imports ...


class MockRequest(object):
"""No code needed."""
pass


REQUEST = MockRequest()
# see: https://stackoverflow.com/queI stions/11938164/why-dont-my-django-\
#  unittests-know-that-messagemiddleware-is-installed
setattr(REQUEST, 'session', 'session')
MESSAGES = FallbackStorage(REQUEST)
setattr(REQUEST, '_messages', MESSAGES)
setup_test_environment()


class PersonAdminTest(TestCase):

def setUp(self):
self.user, password = utils.user_factory(model=None)
self.client = Client()
login_status = self.client.login(username=self.user.email,
password=password)
self.assertEqual(login_status, True)

def test_action_persons_make_unreal(self):
try:
change_url = reverse('admin:persons_realpersons_changelist')
except NoReverseMatch:
change_url = '/admin/persons/realpersons/'
sys.stderr.write('\n   WARNING: Unable to reverse URL! ... ')
response = self.client.get(change_url)
self.assertEqual(response.status_code, 200)

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAF1Wu3MoGgm_dSToObDX9gH7GQJ%3DTZzGLV7RPOiZk4kvV-_Nbg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Test fails when run in whole test suite - but not stand-alone?

Re: Test fails when run in whole test suite - but not stand-alone?

Re: Test fails when run in whole test suite - but not stand-alone?

Re: Test fails when run in whole test suite - but not stand-alone?

Re: Test fails when run in whole test suite - but not stand-alone?

Test fails when run in whole test suite - but not stand-alone?

6 matches

Site Navigation

Mail list logo

Footer information