#35975: New feature: faster fixture loading via loaddata command
-------------------------------+-----------------------------------------
Reporter: JorisBenschop | Type: Uncategorized
Status: new | Component: Uncategorized
Version: 5.1 | Severity: Normal
Keywords: loaddata | Triage Stage: Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+-----------------------------------------
As per [https://forum.djangoproject.com/t/feature-proposal-faster-fixture-
loading-via-loaddata-command/36972/4 this forum discussion], I have
created a patch to improve load times for the loaddata command under some
circumstances.
Currently the “loaddata” management command uses the obj.save() method for
each deserialized object within a fixture. This function first tries an
UPDATE statement and, if that fails, tries an INSERT statement.
I propose to add two optional flags to the loaddata command:
–force-insert adds the “force_insert=True” to the save() method. This
reduces load time by ~50% but has an increased risk of the upload failing
in case the record already exists.
–bulk_create: This option groups records in the fixture by the model,
and insert the group as a single bulk_create statement. For large
fixtures, I have achieved a 1000-fold improvement in loading time. It has
a number of risks (already described in the bulk_create section), notably
that it skips some routines covered in the save() method of the model.
Both these flags are supposed to be run by people who know that above-
mentioned issues will not be a problem for them, and should not be enabled
by default.
--
Ticket URL: <https://code.djangoproject.com/ticket/35975>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-updates+unsubscr...@googlegroups.com.
To view this discussion visit
https://groups.google.com/d/msgid/django-updates/01070193975152a8-e158f025-d3a8-4da9-8c29-32a4d3d7a0e5-000000%40eu-central-1.amazonses.com.
