#22757: prefetch_related isn't as effecient as it could be with
GenericForeignKey
and proxy models
-------------------------------------+-------------------------------------
Reporter: gwahl@… | Owner: (none)
Type: | Status: new
Cleanup/optimization |
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:
Keywords: prefetch_related | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Nolan Little):
* owner: Nolan Little => (none)
* status: assigned => new
Comment:
I started working on this at DjangoCon US. I had difficulty coming up with
a true optimization. I started from
`GenericForeignKey.get_prefetch_queryset` like Simon recommended. Grouping
the queries by concrete model was fairly straightforward, the difficulty
began with turning all instances back into their correct concrete/proxy
types correctly.
The only method I could come up with required iterating over every object
and checking if its instance type was `proxy`. I created a dict of the
instances content types to instance PK during the query grouping so that I
could use it to identify the correct `ContentType`. In the below code
`fkeys` was a list of grouped ids for all concrete and proxy objects:
{{{
objs = concrete_ct.get_all_objects_for_this_type(pk__in=fkeys)
for obj in objs:
instance = instance_dict[obj.pk]
if instance.content_type.model_class()._meta.proxy:
obj.__dict__.pop("_state") # necessary?
ret_obj =
instance.content_type.model_class()(**obj.__dict__)
else:
ret_obj = obj
ret_val.append(ret_obj)
}}}
There may be a cleaner way to get the correct Instance type back that I'm
unaware of as this piece felt quite ugly:
{{{
obj.__dict__.pop("_state") # necessary?
ret_obj = instance.content_type.model_class()(**obj.__dict__)
}}}
Unless there is a way to avoid iterating over all objects to determine the
correct instance I question the viability of optimizing the amount of
queries here. Realistically it seems that the amount of objects returned
is likely to be far higher than the amount of proxy models queried.
If i'm way off on this approach or if there is a way to get the correct
type for return instances without iterating all objects I'd love to learn
about it! I don't anticipate having a lot of time to make progress on this
in the near future so I'm going to un-assign myself, but I'm happy to pick
it back up in the future if anyone has recommendations on the approach.
--
Ticket URL: <https://code.djangoproject.com/ticket/22757#comment:5>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/0107018b959bf7c2-1ce0877a-e789-46b5-8c2e-35c93399b6aa-000000%40eu-central-1.amazonses.com.
