Re: failure to load fixtures during unit tests
FYI, I was finally able to resolve this. I had an assumption that TestCase's began with a freshly created database between TestCase classes. I asserted this in _fixture_setup() and found that assumption to be false. Upon further research I found a setUpClass() in another TestCase that created objects that conflicted with the fixtures loaded in other TestCase classes. Somehow Django-nose's FastFixtureTestCase hid this problem. What I'm not sure about (and need to investigate) is if Django-nose's TestRunner is the optimizing away fresh databases between TestCase classes, or if that is standard. Rich -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at https://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/f74ac850-26d4-4db2-8aef-1d595f8279b7%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: failure to load fixtures during unit tests
Thanks, Tim. Unfortunately I can't move past Django 1.7 yet -- dependencies. I've been marching my way up one revision at a time hopefully up to 1.9 as a way to keep the scope of what breaks under control as I move through each major revision and stabilize my project. Then I attack replacing dependencies. I really think I've found a bug here ... which I hope to suggest a patch for and submit, hence the post to the developers channel, but I can go back to the users group for now... My recent experience with that list doesn't bode well, however, and I don't have high hopes with anyone there able to respond at the internals level I may need to track down the issue. I've almost rewritten my tests to just load raw sql, but if there is a bug here I'd like to help find it rather than work around/ignore it. As I step through the code, it really looks like the _save_table() method in Model is trying to insert a row even though the object has already been restored/inserted. At the moment, I'm reproducing it with the auth.User Model. I'm getting closer to seeing what is happening I have a user, rich, which expects to be pk=1 per the fixture. > /opt/perfcat/virtualenv-2.7.11/lib/python2.7/site-packages/django/db/models/base.py(686)_save_table()uest_by_build_workflow_fail 685 import ipdb; ipdb.set_trace() --> 686 if not updated: 687 if meta.order_with_respect_to: ipdb> self.id 1 ipdb> self.__class__.objects.all() [] ipdb> self.__class__.objects.all()[0].id 5 ipdb> self.username u'rich' ipdb> But In this particular run I'm currently tracing, rich is already in the db (as the only entry) as pk=5 (via fixture loading process). For one, this tells me the sequence generators aren't always resetting between fixture loads/tests. So I think the code is trying to reassign it to pk=1. We did drop into the update code, ipdb> pk_set and not force_insert True But updated is False ipdb> updated False So now it tries to drop into an insert, but it is going to get an Integrity error because username has to be unique. Not sure what this means, yet, but my current step through looks like this: ipdb> IntegrityError: Integrit...sts.\n',) > /opt/perfcat/virtualenv-2.7.11/lib/python2.7/site-packages/django/db/models/base.py(700)_save_table() 699 update_pk = bool(meta.has_auto_field and not pk_set) --> 700 result = self._do_insert(cls._base_manager, using, fields, update_pk, raw) 701 if update_pk: ipdb> update_pk False ipdb> meta.has_auto_field True ipdb> pk_set True ipdb> ...if we don't need to update the pk, and it is set .. why are we inserting it? Walking through a second time with this knowledge ... and stepping into _do_update(), I end up with filtered = base_qs.filter(pk=pk_val) being equal to [] because the entry in the db has a pk=5, and it is filtering for pk=1 So when return filtered._update(values) > 0 returns, it returns false because nothing was updated because the pk's didn't match. Where I am stuck at now is not understanding how fixture loading manages the pks... Rich On Wednesday, May 4, 2016 at 4:22:33 PM UTC-7, Tim Graham wrote: > > Hi Rich, django-users is the appropriate place to ask "is it a bug?" type > questions. We try not to use this mailing list as a second level support > channel, otherwise it'd get really noisy. Thanks for understanding. > > By the way, Django 1.7 is no longer supported. Please make sure you can > reproduce the issue on Django master so we don't spend time debugging > issues that have since been fixed. > > On Wednesday, May 4, 2016 at 7:13:42 PM UTC-4, Rich Rauenzahn wrote: >> >> >> I'm in the middle of trying to track down a problem with loading fixtures >> during unit tests -- I'm hesitant to call it a bug in Django 1.7, but the >> inconsistent behavior is really stumping me. >> >> Essentially I've made a fixture via >> >>manage dumpdata --indent=3 -e sessions -e admin -e contenttypes -e >> auth.Permission > test-fixtures.json >> >> If I add that fixtures to my TestCase, it sometimes works if I run each >> test individually (using Django Nose) -- >> >>manage test --failfast test_it:TestClass.test_detail >>manage test --failfast test_it:TestClass.test_list >> >> But if I run them together, >> >>manage test --failfast test_it:TestClass >> >> I get errors about duplicate/unique problems. Essentially a row is >> attempted to be added twice. >> >> IntegrityError: Problem installing fixture 'test-fixtures.json': >> Could not load app.Branch(pk=1): duplicate key value violates unique >> constraint "app_branch_name_49810fc21046d2e2_uniq" >> DETAIL: Key (name)=(mock) already exists. >> >> (I've also posted this earlier today on django-users, where I also >> included some postgres output). The tests within the TestCase (or >> TransactionTestCase) can be
Re: failure to load fixtures during unit tests
Hi Rich, django-users is the appropriate place to ask "is it a bug?" type questions. We try not to use this mailing list as a second level support channel, otherwise it'd get really noisy. Thanks for understanding. By the way, Django 1.7 is no longer supported. Please make sure you can reproduce the issue on Django master so we don't spend time debugging issues that have since been fixed. On Wednesday, May 4, 2016 at 7:13:42 PM UTC-4, Rich Rauenzahn wrote: > > > I'm in the middle of trying to track down a problem with loading fixtures > during unit tests -- I'm hesitant to call it a bug in Django 1.7, but the > inconsistent behavior is really stumping me. > > Essentially I've made a fixture via > >manage dumpdata --indent=3 -e sessions -e admin -e contenttypes -e > auth.Permission > test-fixtures.json > > If I add that fixtures to my TestCase, it sometimes works if I run each > test individually (using Django Nose) -- > >manage test --failfast test_it:TestClass.test_detail >manage test --failfast test_it:TestClass.test_list > > But if I run them together, > >manage test --failfast test_it:TestClass > > I get errors about duplicate/unique problems. Essentially a row is > attempted to be added twice. > > IntegrityError: Problem installing fixture 'test-fixtures.json': Could > not load app.Branch(pk=1): duplicate key value violates unique constraint > "app_branch_name_49810fc21046d2e2_uniq" > DETAIL: Key (name)=(mock) already exists. > > (I've also posted this earlier today on django-users, where I also > included some postgres output). The tests within the TestCase (or > TransactionTestCase) can be empty ("pass") and still reproduce. > > As best I can tell it doesn't only happen when combined -- sometimes I can > get it to happen in a class with a single TestCase. And it isn't always > the same model that has the conflict. > > Has anyone seen anything like this behavior before? It's as if sometimes > the fixtures are installed in different order each time, which makes me > think of some dict.keys() that doesn't return the same order every time. > > Rich > > > > -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at https://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/410627d6-ab8c-4b6a-b468-54a3511e6c48%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
failure to load fixtures during unit tests
I'm in the middle of trying to track down a problem with loading fixtures during unit tests -- I'm hesitant to call it a bug in Django 1.7, but the inconsistent behavior is really stumping me. Essentially I've made a fixture via manage dumpdata --indent=3 -e sessions -e admin -e contenttypes -e auth.Permission > test-fixtures.json If I add that fixtures to my TestCase, it sometimes works if I run each test individually (using Django Nose) -- manage test --failfast test_it:TestClass.test_detail manage test --failfast test_it:TestClass.test_list But if I run them together, manage test --failfast test_it:TestClass I get errors about duplicate/unique problems. Essentially a row is attempted to be added twice. IntegrityError: Problem installing fixture 'test-fixtures.json': Could not load app.Branch(pk=1): duplicate key value violates unique constraint "app_branch_name_49810fc21046d2e2_uniq" DETAIL: Key (name)=(mock) already exists. (I've also posted this earlier today on django-users, where I also included some postgres output). The tests within the TestCase (or TransactionTestCase) can be empty ("pass") and still reproduce. As best I can tell it doesn't only happen when combined -- sometimes I can get it to happen in a class with a single TestCase. And it isn't always the same model that has the conflict. Has anyone seen anything like this behavior before? It's as if sometimes the fixtures are installed in different order each time, which makes me think of some dict.keys() that doesn't return the same order every time. Rich -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at https://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/37075277-2390-476c-9d32-3b3e594d2e4f%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.