#19501: Allow for fast path model loading
-------------------------------------+-------------------------------------
Reporter: akaariai | Owner: akaariai
Type: | Status: new
Cleanup/optimization | Version: master
Component: Database layer | Resolution:
(models, ORM) | Triage Stage: Accepted
Severity: Normal | Needs documentation: 0
Keywords: | Patch needs improvement: 0
Has patch: 1 | UI/UX: 0
Needs tests: 0 |
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Changes (by akaariai):
* needs_better_patch: 1 => 0
Comment:
I updated the code and docs based on review comments. The code was
mistakenly creating a new model._state in from_db() method, when just
updating the attributes of the existing ._state instance is enough. Now
the slowdown (using djangobench) is:
{{{
Running 'query_select_related' benchmark ...
Min: 0.054252 -> 0.058567: 1.0795x slower
Avg: 0.055689 -> 0.059967: 1.0768x slower
Significant (t=-8.076647)
Stddev: 0.00271 -> 0.00258: 1.0494x smaller (N = 50)
Running 'query_all' benchmark ...
Min: 0.021309 -> 0.022659: 1.0633x slower
Avg: 0.022763 -> 0.024240: 1.0649x slower
Significant (t=-2.598991)
Stddev: 0.00285 -> 0.00283: 1.0042x smaller (N = 50)
Running 'query_raw_deferred' benchmark ...
Min: 0.016543 -> 0.017822: 1.0773x slower
Avg: 0.017149 -> 0.018389: 1.0723x slower
Significant (t=-4.172737)
Stddev: 0.00150 -> 0.00147: 1.0174x smaller (N = 50)
}}}
So, the code itself causes a minor slowdown. The benchmarks done in above
comments and the ticket's description test various fast-path
implementations. Basically, one can do the following for their model:
{{{
class Empty(object):
pass
class MyModel(models.Model):
...
@classmethod
def from_db(cls, db, field_names, values):
empty = Empty()
empty.__class__ = cls
empty.__dict__ = zip(field_names, values)
empty._state = ModelState()
empty._state.adding = False
return empty
}}}
and this results in around 2x better performance for query_all benchmark,
and 4x speedup for deferred model loading (query_defer benchmark).
This ticket discussed various ways to do automatically the above
optimization - it is possible to do if the model doesn't have customized
``__init__``, doesn't have pre/post init signals, and calling setattr
isn't required for the model. Detecting this seems too complex, so the
automation isn't there any more. However, users who require absolute
performance can do the optimization manually if needed.
--
Ticket URL: <https://code.djangoproject.com/ticket/19501#comment:9>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/066.b212805aa835e1a388cb0605b21b3cc3%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
