#31202: Bulk update suffers from poor performance with large numbers of models
and
columns
-------------------------------------+-------------------------------------
Reporter: Tom Forbes | Owner: Tom
Type: | Forbes
Cleanup/optimization | Status: assigned
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by jerch):
I have started to implement an UPDATE FROM VALUES variant in
https://github.com/netzkolchose/django-
computedfields/blob/master/computedfields/fast_update.py, as I needed
faster updates in the lib.
First, the speedup is huge, it performs 10 - 25 times better than the
current bulk_update implementation.
But ofc, there are several issues with this:
- only supported by newer DBMS versions (SQLITE 3.33+, MariaDB 10.3+,
Mysql 8, Oracle unclear)
- not official ANSI SQL, thus the backends need their very own SQL
templates
- MariaDB and Mysql8 have different templates, which is not covered by
Django's backend distinction
- not easy to fit into the ORM's SQL compiler templates, thus I went with
string formatting for now
- F expressions cannot be used in VALUES at all
For Postgres imho COPY FROM would be even faster, but I did not do it due
to the needed complexity with totally different code paths for just
postgres.
--
Ticket URL: <https://code.djangoproject.com/ticket/31202#comment:9>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/061.4debbe02f2bfdb131c118821078fd4d1%40djangoproject.com.
