#31202: Bulk update suffers from poor performance with large numbers of models
and
columns
-------------------------------------+-------------------------------------
Reporter: Tom Forbes | Owner: Tom
Type: | Forbes
Cleanup/optimization | Status: assigned
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by jerch):
Some early tests with COPY FROM indicate, that it even outperforms UPDATE
FROM VALUES by far (at least 4 times faster). But the code is even more
degenerated and violates django paradigms in almost every aspect. This is
probably nothing for django itself, but maybe a third party lib, that
people can use if they are aware of basic restrictions.
What I find bothersome with COPY FROM:
- needs dealing with temporary tables, columns partially bootstrapped from
target tables
- might need own cast/escape rules for more complex field types (have not
yet looked at psycopg3's copy cursor mechanics)
- might not work for all field types / payloads (this most likely depends
on alot on the used transport format/delimiters)
- impact of index on pk field of temp table uncertain (whether index
creation outweighs the index gain on the final update)
Yeah well, this needs alot more investigation, before it will be useable
with a nice interface...
--
Ticket URL: <https://code.djangoproject.com/ticket/31202#comment:10>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/061.2aa3bc4e270edf6418038adfdd02c1d3%40djangoproject.com.
