#31202: Bulk update suffers from poor performance with large numbers of models
and
columns
-------------------------------------+-------------------------------------
Reporter: Tom Forbes | Owner: Tom
Type: | Forbes
Cleanup/optimization | Status: assigned
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by jerch):
Recently stumbled again over the perf issue with `bulk_update` in my
computedfields library. And to my surprise - it is even much slower than
applying flat `update` calls to a bunch of objects. Since I got complaints
from users I added a `flat_update` to my django-fast-update package (still
in PR: https://github.com/netzkolchose/django-fast-update/pull/25).
Here are some perf numbers to digest:
{{{
(env) ➜ example git:(drop_bulk_update) ✗ DBENGINE=sqlite ./manage.py perf
10 instances
bulk_update: 0.05896315574645996
fast_update: 0.002216744422912598
flat_update: 0.006779623031616211
100 instances
bulk_update: 0.5576666355133056
fast_update: 0.01461637020111084
flat_update: 0.06025240421295166
1000 instances
bulk_update: 5.557345986366272
fast_update: 0.13169689178466798
flat_update: 0.5667753458023072
10000 instances
bulk_update: 60.25784921646118
fast_update: 1.262794852256775
flat_update: 5.409043550491333
(env) ➜ example git:(drop_bulk_update) ✗ DBENGINE=postgres ./manage.py
perf
10 instances
bulk_update: 0.05660002231597901
fast_update: 0.004314589500427246
copy_update: 0.007386398315429687
flat_update: 0.017342567443847656
100 instances
bulk_update: 0.5351178169250488
fast_update: 0.01944422721862793
copy_update: 0.017321300506591798
flat_update: 0.13300635814666747
1000 instances
bulk_update: 5.777743911743164
fast_update: 0.1764376640319824
copy_update: 0.10273151397705078
flat_update: 1.348728108406067
10000 instances
bulk_update: 100.42072892189026
fast_update: 1.751427412033081
copy_update: 0.6608501672744751
flat_update: 13.84923231601715
}}}
(Perf test case is also in the PR.)
What sticks out is the fact, that `bulk_update` is even so much worse than
plain `update` chaining. Which makes me wonder what was its actual use
case in first place and reason to add it to django core?
--
Ticket URL: <https://code.djangoproject.com/ticket/31202#comment:19>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-updates+unsubscr...@googlegroups.com.
To view this discussion visit
https://groups.google.com/d/msgid/django-updates/010701986c634d8f-6510480b-2365-4961-a4fd-2662f5302721-000000%40eu-central-1.amazonses.com.
