#36282: Prefetched relations are not available used from parent classes in all
cases
------------------------------+-----------------------------------------
Reporter: Take Weiland | Type: Uncategorized
Status: new | Component: Uncategorized
Version: 5.1 | Severity: Normal
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
------------------------------+-----------------------------------------
When using `prefetch_related` to prefetch relations, Django makes some
attempt to use prefetched data present in parent models, but it is very
limited. It is only implemented for `ForeignKey` and only works for the
immediate parent.
Assuming this model hierarchy:
{{{
class Related(models.Model):
pass
class GrandParent(models.Model):
name = models.CharField(max_length=50)
gp_fk = models.ForeignKey(Related, null=True,
on_delete=models.CASCADE, related_name='gp_fk_rel')
gp_m2m = models.ManyToManyField(Related, related_name='gp_m2m_rel')
class Parent(GrandParent):
pass
class Child(Parent):
pass
}}}
I have written these test cases:
{{{
class PrefetchRelatedWorksWithInheritance(TestCase):
@classmethod
def setUpTestData(cls):
cls.related1 = Related.objects.create()
cls.related2 = Related.objects.create()
cls.related3 = Related.objects.create()
cls.child = Child.objects.create(
gp_fk=cls.related1,
)
cls.m2m_child = Child.objects.create()
cls.m2m_child.gp_m2m.set([cls.related1, cls.related2,
cls.related3])
def test_parent_fk_available_in_child(self):
qs =
GrandParent.objects.select_related('parent').prefetch_related(
'gp_fk'
).filter(pk=self.child.pk)
with self.assertNumQueries(2):
results = list(qs)
self.assertEqual(len(results), 1)
# Works, Parent can look into its GrandParent and find the
prefetched data
self.assertEqual(results[0].parent.gp_fk, self.related1)
def test_grandparent_fk_available_in_child(self):
qs = GrandParent.objects.select_related('parent',
'parent__child').prefetch_related(
'gp_fk'
).filter(pk=self.child.pk)
with self.assertNumQueries(2):
results = list(qs)
self.assertEqual(len(results), 1)
# Causes extra query, Child only looks in Parent, not in
GrandParent
self.assertEqual(results[0].parent.child.gp_fk, self.related1)
def test_parent_m2m_available_in_child(self):
qs =
GrandParent.objects.select_related('parent').prefetch_related(
'gp_m2m'
).filter(pk=self.m2m_child.pk)
with self.assertNumQueries(2):
results = list(qs)
self.assertEqual(len(results), 1)
# Causes extra query, M2M never looks in its parents
self.assertEqual(set(results[0].parent.gp_m2m.all()),
{self.related1, self.related2, self.related3})
def test_grandparent_m2m_available_in_child(self):
qs = GrandParent.objects.select_related('parent',
'parent__child').prefetch_related(
'gp_m2m'
).filter(pk=self.m2m_child.pk)
with self.assertNumQueries(2):
results = list(qs)
self.assertEqual(len(results), 1)
# Causes extra query, M2M never looks in its parents
self.assertEqual(set(results[0].parent.child.gp_m2m.all()),
{self.related1, self.related2, self.related3})
}}}
Only the first of the tests passes.
For ForeignKeys the reason is here:
https://github.com/django/django/blob/c3a23aa02faa1cf1d32e43d66858e793cd9ecac4/django/db/models/fields/related_descriptors.py#L229-L240
This only looks into the immediate parent, not any grandparents.
For ManyToManyField the culprit is here:
https://github.com/django/django/blob/c3a23aa02faa1cf1d32e43d66858e793cd9ecac4/django/db/models/fields/related_descriptors.py#L1094-L1098
It only looks in the instance's _prefetched_objects_cache, when it could
walk up to its parents, just like ForwardManyToOneDescriptor does.
I stumbled upon this when implementing support for
[https://github.com/jazzband/django-model-utils/pull/639 prefetch_related
in Django-Model-Utils' InheritanceManager]. It automatically returns
subclasses form a custom QuerySet iterable. But those subclass-instances
then do not see the prefetched data of their parent, so in my fix I had to
manually copy the caches, so that Django would find them.
I would be willing to work on a patch to make this work out of the box in
Django.
--
Ticket URL: <https://code.djangoproject.com/ticket/36282>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion visit
https://groups.google.com/d/msgid/django-updates/01070195e6550f8a-af88b173-d51e-4940-a915-2b5d02e94393-000000%40eu-central-1.amazonses.com.
