Would you mind checking if the bug occurs in Django 1.5? If it doesn't, it's a regression introduced by the new transaction management in Django 1.6, and it's a release blocker.
-- Aymeric. On 25 juin 2013, at 22:30, Yo-Yo Ma <baxterstock...@gmail.com> wrote: > I should actually note, this bug affects all versions of Postgres, and > presumably all other supported RDBMSs as well. > > On Sunday, June 23, 2013 7:35:45 PM UTC-4, Yo-Yo Ma wrote: > Minor correction: I placed Atomic.__exit__ to verify - the transaction is > commited every time *starting* on the second object (the third stack printed > in the previous post) - it happens at > https://github.com/django/django/blob/master/django/db/transaction.py#L288. > > > On Sunday, June 23, 2013 7:24:40 PM UTC-4, Yo-Yo Ma wrote: > Hi again Russell, > > I did a little digging. I'm not sure, but I may have uncovered the problem. A > transaction block (using `commit_on_success_unless_managed`) is entered and > exited during each fixture object loaded, due to the calls to the > aforementioned method that exist in various model methods (namely, > `save_base`, in this case). Because of this, the transaction is committed > immedately after each object is loaded, despite the attempt to wrap > `commit_on_success_unless_managed` around the context of the `loaddata` call > in the management command. > > The following are the results of my placing print statements (I know that's > old-school - pdb is just too time consuming) inside > `commit_on_success_unless_managed`. In each call, I added: > > print 'AUTOCOMMIT', connection.autocommit > print 'IN ATOMIC BLOCK', connection.in_atomic_block > for frame in inspect.stack(): > print frame[1], frame[3], frame[2] > > as well as a print after the stack of whether atomic() was returned or > _transaction_func() was returned (for easier reading): > > > AUTOCOMMIT False > IN ATOMIC BLOCK False > > django/db/transaction.py commit_on_success_unless_managed 492 > django/core/management/commands/loaddata.py handle 53 > django/core/management/base.py execute 283 > django/core/management/base.py run_from_argv 240 > django/core/management/__init__.py execute 392 > django/core/management/__init__.py execute_from_command_line 399 > manage.py <module> 10 > > ----RETURNING TRANSACTION FUNC > > =========================================================== > > AUTOCOMMIT False > IN ATOMIC BLOCK False > > django/db/transaction.py commit_on_success_unless_managed 492 > django/db/models/base.py save_base 573 > django/core/serializers/base.py save 165 > django/core/management/commands/loaddata.py process_dir 225 > django/core/management/commands/loaddata.py load_label 169 > django/core/management/commands/loaddata.py loaddata 102 > django/core/management/commands/loaddata.py handle 54 > django/core/management/base.py execute 283 > django/core/management/base.py run_from_argv 240 > django/core/management/__init__.py execute 392 > django/core/management/__init__.py execute_from_command_line 399 > manage.py <module> 10 > > ----RETURNING TRANSACTION FUNC > > =========================================================== > > SAVEPOINT False > AUTOCOMMIT True > IN ATOMIC BLOCK False > > ||||||||||||||||||||||||||||||||||||||||||||||| > django/db/transaction.py commit_on_success_unless_managed 492 > django/db/models/base.py save_base 573 > django/core/serializers/base.py save 165 > django/core/management/commands/loaddata.py process_dir 225 > django/core/management/commands/loaddata.py load_label 169 > django/core/management/commands/loaddata.py loaddata 102 > django/core/management/commands/loaddata.py handle 54 > django/core/management/base.py execute 283 > django/core/management/base.py run_from_argv 240 > django/core/management/__init__.py execute 392 > django/core/management/__init__.py execute_from_command_line 399 > manage.py <module> 10 > > ----RETURNING ATOMIC > > =========================================================== > > > The remaining calls were exactly like call 3 (including "IN ATOMIC BLOCK > False", despite the 3rd call having returned `atomic()`). My prima facie > opinion is that `with atomic()` is needed in `loaddata`, instead of `with > commit_on_success_unless_managed`, since the latter acts funky when nested > calls occur (as see in save_base in the stacks printed above). However, the > issue might be something that needs to be resolved in the > transitioning-to-atomic code. I don't fully understand all of this yet, but > it's a start. > > > On Friday, June 21, 2013 4:34:14 PM UTC-4, Yo-Yo Ma wrote: > Pardon one typo: I meant `python manage.py loaddata test_data` in my previous > post. > > On Friday, June 21, 2013 4:32:33 PM UTC-4, Yo-Yo Ma wrote: > Hi Russel, > > Thanks for taking the time to explain that. I tried that same day to > reproduce the issue in a testing env with the simplified models I typed > above, but my hosting provider had some erroneous networking nonsense that > ruined my test after I spent a couple hours setting everything up. I figured > I'm come back to it... and here I am. > > I didn't set up an entire test env and test app this time, just made a fresh > database and ran my apps fixtures on it, but I did test my app again, using a > fresh database without any data. The models and fixtures for which are as > follows (minus most of the decimals, chars, and other non-FK-type fields, > none of which should be related to this problem): > > > # account/models.py > class Account(models.Model): > name = models.CharField(_(u'name'), max_length=255) > > > # orders/models.py > class Order(models.Model): > account = models.ForeignKey('account.Account', verbose_name=_(u'account')) > number = models.IntegerField(_(u'number')) > bill_address = models.OneToOneField( > 'orders.OrderAddress', > null=True, > on_delete=models.SET_NULL, > related_name='bill_address_order', > verbose_name=_(u'bill to address') > ) > > class OrderAddress(models.Model): > account = models.ForeignKey('account.Account', verbose_name=_(u'account')) > order = models.ForeignKey('orders.Order', verbose_name=_(u'order')) > country = models.CharField(_(u'country'), max_length=2) > > > // orders/fixtures/test_data.json > [ > { > "model": "orders.order", > "pk": 1, > "fields": { > "account": 1, > "number": 1, > "bill_address": 1 > } > }, > { > "model": "orders.orderaddress", > "pk": 1, > "fields": { > "account": 1, > "order": 1, > "country": "US", > } > } > ] > > > (an Account with the primary key of 1 already exists at the time of > ``loaddata``) > > > The error I get with `python manage.py loaddata test_data orders` is: > > django.db.utils.IntegrityError: Problem installing fixture > '/opt/myproject/apps/orders/fixtures/test_data.json': Could not load > orders.Order(pk=1): insert or update on table "orders_order" violates foreign > key constraint "bill_address_id_refs_id_3a4d3fef" > DETAIL: Key (bill_address_id)=(1) is not present in table > "orders_orderaddress". > > > The above fixtures load locally, and they load during test running (with > Postgres) a number of times, but for some reason I get that error when using > `manage.py loaddata ...`. > > > On Sunday, June 16, 2013 7:40:02 PM UTC-4, Russell Keith-Magee wrote: > > Circular dependencies *shouldn't* be a problem on PostgreSQL because all > constraints are set DEFERABLE INITIALLY DEFERRED; that means no constrain > checks should be performed are performed until the transaction boundary, so > all circular references shouldn't be a problem. > > Ticket #3615 exists because MySQL's implementation of DEFERABLE INITIALLY > DEFERRED under InnoDB is, to use the technical term, "Broken". It's unrelated > to any problem you may have found in PostgreSQL, because PostgreSQL gets the > underlying behaviour right. > > Beyond that, we need a specific test case to take this any further. As it > stands, I'm not aware of any problems loading fixtures into PostgreSQL. If > you are able to construct and provide a set of models (which you have done) > and simple fixture (which you haven't) that fails reliably, we have a new bug > on our hands, and you should open a ticket with all the details you can > provide. Confirming whether this is a problem with the alpha, or an ongoing > problem would also be helpful. > > Yours, > Russ Magee %-) > > On Mon, Jun 17, 2013 at 6:22 AM, Yo-Yo Ma <baxters...@gmail.com> wrote: > There doesn't appear to be a way to load fixtures from JSON (using Postgres - > works fine in sqlite3) for the following models: > > > class Collection(models.Model): > main_thing = models.OneToOneField( > 'things.Thing', > null=True, > on_delete=models.SET_NULL > ) > > class Thing(models.Model): > collection = models.ForeignKey( > 'collections.Collection' > ) > > > Here is the exception: > > Problem installing fixture 'my_fixture.json': Could not load > collections.Collection(pk=1): insert or update on table > "collections_collection" violates foreign key constraint > "main_thing_id_refs_id_3a4d3fef" > DETAIL: Key (main_thing_id)=(1) is not present in table "things_thing". > > I'm not sure if the issue is due to the unique constraint implied by a > OneToOneField, or if it's just related to this issue: > https://code.djangoproject.com/ticket/3615 (seems like that ticket and > related ones have been closed for years, so possibly not related). > > Any thoughts? > > Note: I'm using @1.6a1 > > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to django-develop...@googlegroups.com. > To post to this group, send email to django-d...@googlegroups.com. > Visit this group at http://groups.google.com/group/django-developers. > For more options, visit https://groups.google.com/groups/opt_out. > > > > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to django-developers+unsubscr...@googlegroups.com. > To post to this group, send email to django-developers@googlegroups.com. > Visit this group at http://groups.google.com/group/django-developers. > For more options, visit https://groups.google.com/groups/opt_out. > > -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers. For more options, visit https://groups.google.com/groups/opt_out.
