#24576: Bad cascading leads to non-deterministic IntegrityError
-------------------------------------+-------------------------------------
Reporter: glic3rinu | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 1.8
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by timgraham):
* needs_better_patch: => 0
* needs_docs: => 0
* needs_tests: => 0
* stage: Unreviewed => Accepted
Old description:
> I've spent all day with a bug on my application that seems to be a bug on
> Django's ORM delete on cascade resoultion order.
>
> Moreover because the order in which related objects are deleted by the
> ORM is non-deterministic (changes every time you start the interpreter)
> it makes even harder to track down.
>
> So far I've been able to reproduce the problem with a few models:
>
> {{{
> from django.db import models
> from django.contrib.contenttypes.fields import GenericForeignKey,
> GenericRelation
> from django.contrib.contenttypes.models import ContentType
> from django.db.models.signals import post_delete
> from django.dispatch import receiver
>
> class Resource(models.Model):
> content_type = models.ForeignKey(ContentType)
> object_id = models.PositiveIntegerField()
> content_object = GenericForeignKey()
>
> class Account(models.Model):
> name = models.CharField(max_length=10)
> resources = GenericRelation(Resource)
>
> class Order(models.Model):
> account = models.ForeignKey(Account)
> content_type = models.ForeignKey(ContentType)
> object_id = models.PositiveIntegerField()
> content_object = GenericForeignKey()
>
> class MetricStorage(models.Model):
> order = models.ForeignKey(Order)
>
> @receiver(post_delete, dispatch_uid="orders.cancel_orders")
> def cancel_orders(sender, **kwargs):
> instance = kwargs['instance']
> print('delete', sender, instance, instance.pk)
> if sender == Resource:
> related = instance.content_object
> if related:
> ct = ContentType.objects.get_for_model(related)
> order = Order.objects.get(content_type=ct,
> object_id=related.pk)
> order.metricstorage_set.create()
> order.save()
> else:
> print('related is None')
> }}}
>
> And this test code
>
> {{{
> from test.models import Account, Resource, Order
>
> account = Account.objects.create(name='John')
> resource = Resource.objects.create(content_object=account)
> order = Order.objects.create(account=account, content_object=account)
> account.delete()
> }}}
>
> In order to properly test this you should make tries restarting the
> interpreter a handfull of times, and then you'll se two different
> results.
>
> This that I consider correct:
>
> {{{
> delete <class 'test.models.Order'> Order object 384
> delete <class 'test.models.Account'> Account object 124
> delete <class 'test.models.Resource'> Resource object 307
> related is None
> }}}
>
> And the IntegrityError which I consider to be a bug
>
> {{{
> delete <class 'test.models.Resource'> Resource object 311
> delete <class 'test.models.Order'> Order object 388
> delete <class 'test.models.Account'> Account object 128
> Traceback (most recent call last):
> File "<console>", line 1, in <module>
> File "/usr/local/lib/python3.4/dist-packages/django/db/models/base.py",
> line 872, in delete
> collector.delete()
> File "/usr/local/lib/python3.4/dist-
> packages/django/db/models/deletion.py", line 314, in delete
> sender=model, instance=obj, using=self.using
> File "/usr/local/lib/python3.4/dist-packages/django/db/transaction.py",
> line 232, in __exit__
> connection.commit()
> File "/usr/local/lib/python3.4/dist-
> packages/django/db/backends/base/base.py", line 173, in commit
> self._commit()
> File "/usr/local/lib/python3.4/dist-
> packages/django/db/backends/base/base.py", line 142, in _commit
> return self.connection.commit()
> File "/usr/local/lib/python3.4/dist-packages/django/db/utils.py", line
> 97, in __exit__
> six.reraise(dj_exc_type, dj_exc_value, traceback)
> File "/usr/local/lib/python3.4/dist-packages/django/utils/six.py", line
> 658, in reraise
> raise value.with_traceback(tb)
> File "/usr/local/lib/python3.4/dist-
> packages/django/db/backends/base/base.py", line 142, in _commit
> return self.connection.commit()
> django.db.utils.IntegrityError: insert or update on table
> "test_metricstorage" violates foreign key constraint
> "test_metricstorage_order_id_730c757c66b8c627_fk_test_order_id"
> DETAIL: Key (order_id)=(388) is not present in table "test_order".
> }}}
>
> Notice how the order in which related objects are deleted is differnet, I
> believe this to be the source of the problem.
>
> I've noticed this problem on 1.7 and now 1.8, not tested with master.
New description:
I've spent all day with a bug on my application that seems to be a bug on
Django's ORM delete on cascade resolution order.
Moreover because the order in which related objects are deleted by the ORM
is non-deterministic (changes every time you start the interpreter) it
makes even harder to track down.
So far I've been able to reproduce the problem with a few models:
{{{
from django.db import models
from django.contrib.contenttypes.fields import GenericForeignKey,
GenericRelation
from django.contrib.contenttypes.models import ContentType
from django.db.models.signals import post_delete
from django.dispatch import receiver
class Resource(models.Model):
content_type = models.ForeignKey(ContentType)
object_id = models.PositiveIntegerField()
content_object = GenericForeignKey()
class Account(models.Model):
name = models.CharField(max_length=10)
resources = GenericRelation(Resource)
class Order(models.Model):
account = models.ForeignKey(Account)
content_type = models.ForeignKey(ContentType)
object_id = models.PositiveIntegerField()
content_object = GenericForeignKey()
class MetricStorage(models.Model):
order = models.ForeignKey(Order)
@receiver(post_delete, dispatch_uid="orders.cancel_orders")
def cancel_orders(sender, **kwargs):
instance = kwargs['instance']
print('delete', sender, instance, instance.pk)
if sender == Resource:
related = instance.content_object
if related:
ct = ContentType.objects.get_for_model(related)
order = Order.objects.get(content_type=ct,
object_id=related.pk)
order.metricstorage_set.create()
order.save()
else:
print('related is None')
}}}
And this test code
{{{
from test.models import Account, Resource, Order
account = Account.objects.create(name='John')
resource = Resource.objects.create(content_object=account)
order = Order.objects.create(account=account, content_object=account)
account.delete()
}}}
In order to properly test this you should make tries restarting the
interpreter a handful of times, and then you'll see two different results.
This that I consider correct:
{{{
delete <class 'test.models.Order'> Order object 384
delete <class 'test.models.Account'> Account object 124
delete <class 'test.models.Resource'> Resource object 307
related is None
}}}
And the IntegrityError which I consider to be a bug
{{{
delete <class 'test.models.Resource'> Resource object 311
delete <class 'test.models.Order'> Order object 388
delete <class 'test.models.Account'> Account object 128
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/usr/local/lib/python3.4/dist-packages/django/db/models/base.py",
line 872, in delete
collector.delete()
File "/usr/local/lib/python3.4/dist-
packages/django/db/models/deletion.py", line 314, in delete
sender=model, instance=obj, using=self.using
File "/usr/local/lib/python3.4/dist-packages/django/db/transaction.py",
line 232, in __exit__
connection.commit()
File "/usr/local/lib/python3.4/dist-
packages/django/db/backends/base/base.py", line 173, in commit
self._commit()
File "/usr/local/lib/python3.4/dist-
packages/django/db/backends/base/base.py", line 142, in _commit
return self.connection.commit()
File "/usr/local/lib/python3.4/dist-packages/django/db/utils.py", line
97, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/usr/local/lib/python3.4/dist-packages/django/utils/six.py", line
658, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.4/dist-
packages/django/db/backends/base/base.py", line 142, in _commit
return self.connection.commit()
django.db.utils.IntegrityError: insert or update on table
"test_metricstorage" violates foreign key constraint
"test_metricstorage_order_id_730c757c66b8c627_fk_test_order_id"
DETAIL: Key (order_id)=(388) is not present in table "test_order".
}}}
Notice how the order in which related objects are deleted is different, I
believe this to be the source of the problem.
I've noticed this problem on 1.7 and now 1.8, not tested with master.
--
Comment:
I could also reproduce on master with your example code.
--
Ticket URL: <https://code.djangoproject.com/ticket/24576#comment:1>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/067.eb666febf6b17de837ef92c8d47558a2%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
