Hi again Russell,
I did a little digging. I'm not sure, but I may have uncovered the problem.
A transaction block (using `commit_on_success_unless_managed`) is entered
and exited during each fixture object loaded, due to the calls to the
aforementioned method that exist in various model methods (namely,
`save_base`, in this case). Because of this, the transaction is committed
immedately after each object is loaded, despite the attempt to wrap
`commit_on_success_unless_managed` around the context of the `loaddata`
call in the management command.
The following are the results of my placing print statements (I know that's
old-school - pdb is just too time consuming) inside
`commit_on_success_unless_managed`. In each call, I added:
print 'AUTOCOMMIT', connection.autocommit
print 'IN ATOMIC BLOCK', connection.in_atomic_block
for frame in inspect.stack():
print frame[1], frame[3], frame[2]
as well as a print after the stack of whether atomic() was returned or
_transaction_func() was returned (for easier reading):
AUTOCOMMIT False
IN ATOMIC BLOCK False
django/db/transaction.py commit_on_success_unless_managed 492
django/core/management/commands/loaddata.py handle 53
django/core/management/base.py execute 283
django/core/management/base.py run_from_argv 240
django/core/management/__init__.py execute 392
django/core/management/__init__.py execute_from_command_line 399
manage.py <module> 10
----RETURNING TRANSACTION FUNC
===========================================================
AUTOCOMMIT False
IN ATOMIC BLOCK False
django/db/transaction.py commit_on_success_unless_managed 492
django/db/models/base.py save_base 573
django/core/serializers/base.py save 165
django/core/management/commands/loaddata.py process_dir 225
django/core/management/commands/loaddata.py load_label 169
django/core/management/commands/loaddata.py loaddata 102
django/core/management/commands/loaddata.py handle 54
django/core/management/base.py execute 283
django/core/management/base.py run_from_argv 240
django/core/management/__init__.py execute 392
django/core/management/__init__.py execute_from_command_line 399
manage.py <module> 10
----RETURNING TRANSACTION FUNC
===========================================================
SAVEPOINT False
AUTOCOMMIT True
IN ATOMIC BLOCK False
|||||||||||||||||||||||||||||||||||||||||||||||
django/db/transaction.py commit_on_success_unless_managed 492
django/db/models/base.py save_base 573
django/core/serializers/base.py save 165
django/core/management/commands/loaddata.py process_dir 225
django/core/management/commands/loaddata.py load_label 169
django/core/management/commands/loaddata.py loaddata 102
django/core/management/commands/loaddata.py handle 54
django/core/management/base.py execute 283
django/core/management/base.py run_from_argv 240
django/core/management/__init__.py execute 392
django/core/management/__init__.py execute_from_command_line 399
manage.py <module> 10
----RETURNING ATOMIC
===========================================================
The remaining calls were exactly like call 3 (including "IN ATOMIC BLOCK
False", despite the 3rd call having returned `atomic()`). My prima facie
opinion is that `with atomic()` is needed in `loaddata`, instead of `with
commit_on_success_unless_managed`, since the latter acts funky when nested
calls occur (as see in save_base in the stacks printed above). However, the
issue might be something that needs to be resolved in the
transitioning-to-atomic code. I don't fully understand all of this yet, but
it's a start.
On Friday, June 21, 2013 4:34:14 PM UTC-4, Yo-Yo Ma wrote:
>
> Pardon one typo: I meant `python manage.py loaddata test_data` in my
> previous post.
>
> On Friday, June 21, 2013 4:32:33 PM UTC-4, Yo-Yo Ma wrote:
>>
>> Hi Russel,
>>
>> Thanks for taking the time to explain that. I tried that same day to
>> reproduce the issue in a testing env with the simplified models I typed
>> above, but my hosting provider had some erroneous networking nonsense that
>> ruined my test after I spent a couple hours setting everything up. I
>> figured I'm come back to it... and here I am.
>>
>> I didn't set up an entire test env and test app this time, just made a
>> fresh database and ran my apps fixtures on it, but I did test my app again,
>> using a fresh database without any data. The models and fixtures for which
>> are as follows (minus most of the decimals, chars, and other non-FK-type
>> fields, none of which should be related to this problem):
>>
>>
>> # account/models.py
>> class Account(models.Model):
>> name = models.CharField(_(u'name'), max_length=255)
>>
>>
>> # orders/models.py
>> class Order(models.Model):
>> account = models.ForeignKey('account.Account',
>> verbose_name=_(u'account'))
>> number = models.IntegerField(_(u'number'))
>> bill_address = models.OneToOneField(
>> 'orders.OrderAddress',
>> null=True,
>> on_delete=models.SET_NULL,
>> related_name='bill_address_order',
>> verbose_name=_(u'bill to address')
>> )
>>
>> class OrderAddress(models.Model):
>> account = models.ForeignKey('account.Account',
>> verbose_name=_(u'account'))
>> order = models.ForeignKey('orders.Order', verbose_name=_(u'order'))
>> country = models.CharField(_(u'country'), max_length=2)
>>
>>
>> // orders/fixtures/test_data.json
>> [
>> {
>> "model": "orders.order",
>> "pk": 1,
>> "fields": {
>> "account": 1,
>> "number": 1,
>> "bill_address": 1
>> }
>> },
>> {
>> "model": "orders.orderaddress",
>> "pk": 1,
>> "fields": {
>> "account": 1,
>> "order": 1,
>> "country": "US",
>> }
>> }
>> ]
>>
>>
>> (an Account with the primary key of 1 already exists at the time of
>> ``loaddata``)
>>
>>
>> The error I get with `python manage.py loaddata test_data orders` is:
>>
>> django.db.utils.IntegrityError: Problem installing fixture
>> '/opt/myproject/apps/orders/fixtures/test_data.json': Could not load
>> orders.Order(pk=1): insert or update on table "orders_order" violates
>> foreign key constraint "bill_address_id_refs_id_3a4d3fef"
>> DETAIL: Key (bill_address_id)=(1) is not present in table
>> "orders_orderaddress".
>>
>>
>> The above fixtures load locally, and they load during test running (with
>> Postgres) a number of times, but for some reason I get that error when
>> using `manage.py loaddata ...`.
>>
>>
>> On Sunday, June 16, 2013 7:40:02 PM UTC-4, Russell Keith-Magee wrote:
>>>
>>>
>>> Circular dependencies *shouldn't* be a problem on PostgreSQL because all
>>> constraints are set DEFERABLE INITIALLY DEFERRED; that means no constrain
>>> checks should be performed are performed until the transaction boundary, so
>>> all circular references shouldn't be a problem.
>>>
>>> Ticket #3615 exists because MySQL's implementation of DEFERABLE
>>> INITIALLY DEFERRED under InnoDB is, to use the technical term, "Broken".
>>> It's unrelated to any problem you may have found in PostgreSQL, because
>>> PostgreSQL gets the underlying behaviour right.
>>>
>>> Beyond that, we need a specific test case to take this any further. As
>>> it stands, I'm not aware of any problems loading fixtures into PostgreSQL.
>>> If you are able to construct and provide a set of models (which you have
>>> done) and simple fixture (which you haven't) that fails reliably, we have a
>>> new bug on our hands, and you should open a ticket with all the details you
>>> can provide. Confirming whether this is a problem with the alpha, or an
>>> ongoing problem would also be helpful.
>>>
>>> Yours,
>>> Russ Magee %-)
>>>
>>> On Mon, Jun 17, 2013 at 6:22 AM, Yo-Yo Ma <[email protected]> wrote:
>>>
>>>> There doesn't appear to be a way to load fixtures from JSON (using
>>>> Postgres - works fine in sqlite3) for the following models:
>>>>
>>>>
>>>> class Collection(models.Model):
>>>> main_thing = models.OneToOneField(
>>>> 'things.Thing',
>>>> null=True,
>>>> on_delete=models.SET_NULL
>>>> )
>>>>
>>>> class Thing(models.Model):
>>>> collection = models.ForeignKey(
>>>> 'collections.Collection'
>>>> )
>>>>
>>>>
>>>> Here is the exception:
>>>>
>>>> Problem installing fixture 'my_fixture.json': Could not load
>>>> collections.Collection(pk=1): insert or update on table
>>>> "collections_collection" violates foreign key constraint
>>>> "main_thing_id_refs_id_3a4d3fef"
>>>> DETAIL: Key (main_thing_id)=(1) is not present in table "things_thing".
>>>>
>>>> I'm not sure if the issue is due to the unique constraint implied by a
>>>> OneToOneField, or if it's just related to this issue:
>>>> https://code.djangoproject.com/ticket/3615 (seems like that ticket and
>>>> related ones have been closed for years, so possibly not related).
>>>>
>>>> Any thoughts?
>>>>
>>>> Note: I'm using @1.6a1
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "Django developers" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to [email protected].
>>>> To post to this group, send email to [email protected].
>>>> Visit this group at http://groups.google.com/group/django-developers.
>>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>>
>>>>
>>>>
>>>
>>>
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.
