I should actually note, this bug affects all versions of Postgres, and presumably all other supported RDBMSs as well.
On Sunday, June 23, 2013 7:35:45 PM UTC-4, Yo-Yo Ma wrote: > > Minor correction: I placed Atomic.__exit__ to verify - the transaction is > commited every time *starting* on the second object (the third stack > printed in the previous post) - it happens at > https://github.com/django/django/blob/master/django/db/transaction.py#L288 > . > > > On Sunday, June 23, 2013 7:24:40 PM UTC-4, Yo-Yo Ma wrote: >> >> Hi again Russell, >> >> I did a little digging. I'm not sure, but I may have uncovered the >> problem. A transaction block (using `commit_on_success_unless_managed`) is >> entered and exited during each fixture object loaded, due to the calls to >> the aforementioned method that exist in various model methods (namely, >> `save_base`, in this case). Because of this, the transaction is committed >> immedately after each object is loaded, despite the attempt to wrap >> `commit_on_success_unless_managed` around the context of the `loaddata` >> call in the management command. >> >> The following are the results of my placing print statements (I know >> that's old-school - pdb is just too time consuming) inside >> `commit_on_success_unless_managed`. In each call, I added: >> >> print 'AUTOCOMMIT', connection.autocommit >> print 'IN ATOMIC BLOCK', connection.in_atomic_block >> for frame in inspect.stack(): >> print frame[1], frame[3], frame[2] >> >> as well as a print after the stack of whether atomic() was returned or >> _transaction_func() was returned (for easier reading): >> >> >> AUTOCOMMIT False >> IN ATOMIC BLOCK False >> >> django/db/transaction.py commit_on_success_unless_managed 492 >> django/core/management/commands/loaddata.py handle 53 >> django/core/management/base.py execute 283 >> django/core/management/base.py run_from_argv 240 >> django/core/management/__init__.py execute 392 >> django/core/management/__init__.py execute_from_command_line 399 >> manage.py <module> 10 >> >> ----RETURNING TRANSACTION FUNC >> >> =========================================================== >> >> AUTOCOMMIT False >> IN ATOMIC BLOCK False >> >> django/db/transaction.py commit_on_success_unless_managed 492 >> django/db/models/base.py save_base 573 >> django/core/serializers/base.py save 165 >> django/core/management/commands/loaddata.py process_dir 225 >> django/core/management/commands/loaddata.py load_label 169 >> django/core/management/commands/loaddata.py loaddata 102 >> django/core/management/commands/loaddata.py handle 54 >> django/core/management/base.py execute 283 >> django/core/management/base.py run_from_argv 240 >> django/core/management/__init__.py execute 392 >> django/core/management/__init__.py execute_from_command_line 399 >> manage.py <module> 10 >> >> ----RETURNING TRANSACTION FUNC >> >> =========================================================== >> >> SAVEPOINT False >> AUTOCOMMIT True >> IN ATOMIC BLOCK False >> >> ||||||||||||||||||||||||||||||||||||||||||||||| >> django/db/transaction.py commit_on_success_unless_managed 492 >> django/db/models/base.py save_base 573 >> django/core/serializers/base.py save 165 >> django/core/management/commands/loaddata.py process_dir 225 >> django/core/management/commands/loaddata.py load_label 169 >> django/core/management/commands/loaddata.py loaddata 102 >> django/core/management/commands/loaddata.py handle 54 >> django/core/management/base.py execute 283 >> django/core/management/base.py run_from_argv 240 >> django/core/management/__init__.py execute 392 >> django/core/management/__init__.py execute_from_command_line 399 >> manage.py <module> 10 >> >> ----RETURNING ATOMIC >> >> =========================================================== >> >> >> The remaining calls were exactly like call 3 (including "IN ATOMIC BLOCK >> False", despite the 3rd call having returned `atomic()`). My prima facie >> opinion is that `with atomic()` is needed in `loaddata`, instead of `with >> commit_on_success_unless_managed`, since the latter acts funky when nested >> calls occur (as see in save_base in the stacks printed above). However, the >> issue might be something that needs to be resolved in the >> transitioning-to-atomic code. I don't fully understand all of this yet, but >> it's a start. >> >> >> On Friday, June 21, 2013 4:34:14 PM UTC-4, Yo-Yo Ma wrote: >>> >>> Pardon one typo: I meant `python manage.py loaddata test_data` in my >>> previous post. >>> >>> On Friday, June 21, 2013 4:32:33 PM UTC-4, Yo-Yo Ma wrote: >>>> >>>> Hi Russel, >>>> >>>> Thanks for taking the time to explain that. I tried that same day to >>>> reproduce the issue in a testing env with the simplified models I typed >>>> above, but my hosting provider had some erroneous networking nonsense that >>>> ruined my test after I spent a couple hours setting everything up. I >>>> figured I'm come back to it... and here I am. >>>> >>>> I didn't set up an entire test env and test app this time, just made a >>>> fresh database and ran my apps fixtures on it, but I did test my app >>>> again, >>>> using a fresh database without any data. The models and fixtures for which >>>> are as follows (minus most of the decimals, chars, and other non-FK-type >>>> fields, none of which should be related to this problem): >>>> >>>> >>>> # account/models.py >>>> class Account(models.Model): >>>> name = models.CharField(_(u'name'), max_length=255) >>>> >>>> >>>> # orders/models.py >>>> class Order(models.Model): >>>> account = models.ForeignKey('account.Account', >>>> verbose_name=_(u'account')) >>>> number = models.IntegerField(_(u'number')) >>>> bill_address = models.OneToOneField( >>>> 'orders.OrderAddress', >>>> null=True, >>>> on_delete=models.SET_NULL, >>>> related_name='bill_address_order', >>>> verbose_name=_(u'bill to address') >>>> ) >>>> >>>> class OrderAddress(models.Model): >>>> account = models.ForeignKey('account.Account', >>>> verbose_name=_(u'account')) >>>> order = models.ForeignKey('orders.Order', verbose_name=_(u'order')) >>>> country = models.CharField(_(u'country'), max_length=2) >>>> >>>> >>>> // orders/fixtures/test_data.json >>>> [ >>>> { >>>> "model": "orders.order", >>>> "pk": 1, >>>> "fields": { >>>> "account": 1, >>>> "number": 1, >>>> "bill_address": 1 >>>> } >>>> }, >>>> { >>>> "model": "orders.orderaddress", >>>> "pk": 1, >>>> "fields": { >>>> "account": 1, >>>> "order": 1, >>>> "country": "US", >>>> } >>>> } >>>> ] >>>> >>>> >>>> (an Account with the primary key of 1 already exists at the time of >>>> ``loaddata``) >>>> >>>> >>>> The error I get with `python manage.py loaddata test_data orders` is: >>>> >>>> django.db.utils.IntegrityError: Problem installing fixture >>>> '/opt/myproject/apps/orders/fixtures/test_data.json': Could not load >>>> orders.Order(pk=1): insert or update on table "orders_order" violates >>>> foreign key constraint "bill_address_id_refs_id_3a4d3fef" >>>> DETAIL: Key (bill_address_id)=(1) is not present in table >>>> "orders_orderaddress". >>>> >>>> >>>> The above fixtures load locally, and they load during test running >>>> (with Postgres) a number of times, but for some reason I get that error >>>> when using `manage.py loaddata ...`. >>>> >>>> >>>> On Sunday, June 16, 2013 7:40:02 PM UTC-4, Russell Keith-Magee wrote: >>>>> >>>>> >>>>> Circular dependencies *shouldn't* be a problem on PostgreSQL because >>>>> all constraints are set DEFERABLE INITIALLY DEFERRED; that means no >>>>> constrain checks should be performed are performed until the transaction >>>>> boundary, so all circular references shouldn't be a problem. >>>>> >>>>> Ticket #3615 exists because MySQL's implementation of DEFERABLE >>>>> INITIALLY DEFERRED under InnoDB is, to use the technical term, "Broken". >>>>> It's unrelated to any problem you may have found in PostgreSQL, because >>>>> PostgreSQL gets the underlying behaviour right. >>>>> >>>>> Beyond that, we need a specific test case to take this any further. As >>>>> it stands, I'm not aware of any problems loading fixtures into >>>>> PostgreSQL. >>>>> If you are able to construct and provide a set of models (which you have >>>>> done) and simple fixture (which you haven't) that fails reliably, we have >>>>> a >>>>> new bug on our hands, and you should open a ticket with all the details >>>>> you >>>>> can provide. Confirming whether this is a problem with the alpha, or an >>>>> ongoing problem would also be helpful. >>>>> >>>>> Yours, >>>>> Russ Magee %-) >>>>> >>>>> On Mon, Jun 17, 2013 at 6:22 AM, Yo-Yo Ma <[email protected]>wrote: >>>>> >>>>>> There doesn't appear to be a way to load fixtures from JSON (using >>>>>> Postgres - works fine in sqlite3) for the following models: >>>>>> >>>>>> >>>>>> class Collection(models.Model): >>>>>> main_thing = models.OneToOneField( >>>>>> 'things.Thing', >>>>>> null=True, >>>>>> on_delete=models.SET_NULL >>>>>> ) >>>>>> >>>>>> class Thing(models.Model): >>>>>> collection = models.ForeignKey( >>>>>> 'collections.Collection' >>>>>> ) >>>>>> >>>>>> >>>>>> Here is the exception: >>>>>> >>>>>> Problem installing fixture 'my_fixture.json': Could not load >>>>>> collections.Collection(pk=1): insert or update on table >>>>>> "collections_collection" violates foreign key constraint >>>>>> "main_thing_id_refs_id_3a4d3fef" >>>>>> DETAIL: Key (main_thing_id)=(1) is not present in table >>>>>> "things_thing". >>>>>> >>>>>> I'm not sure if the issue is due to the unique constraint implied by >>>>>> a OneToOneField, or if it's just related to this issue: >>>>>> https://code.djangoproject.com/ticket/3615 (seems like that ticket >>>>>> and related ones have been closed for years, so possibly not related). >>>>>> >>>>>> Any thoughts? >>>>>> >>>>>> Note: I'm using @1.6a1 >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Django developers" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> To post to this group, send email to [email protected]. >>>>>> Visit this group at http://groups.google.com/group/django-developers. >>>>>> For more options, visit https://groups.google.com/groups/opt_out. >>>>>> >>>>>> >>>>>> >>>>> >>>>> -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/django-developers. For more options, visit https://groups.google.com/groups/opt_out.
