#24670: Django migrations are not scaling well with number of operations in
migration
--------------------------------------+------------------------------------
Reporter: kurevin | Owner: nobody
Type: Cleanup/optimization | Status: new
Component: Migrations | Version: 1.8
Severity: Normal | Keywords: migrations
| optimization
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
--------------------------------------+------------------------------------
In our project there are around 200 models, most of them are in a single
app.
I've been testing potencial performance improvements of migration runner
in django 1.8.
When running migration that adds 17 fields this happens:
Rendering model states... DONE (23.982s)
Applying xxx... OK (28.930s)
50 seconds to add just 17 fields to a completely empty DB.
Similar results shows current django master.
In comparison, migration that adds a single field:
Rendering model states... DONE (2.994s)
Applying xxx2... OK (1.540s)
Profiling of 17 fields migration:
Rendering model states... DONE (27.341s)
Applying xxx... OK (33.287s)
{{{
26957269 function calls (26828616 primitive calls) in 61.414
seconds
Ordered by: cumulative time
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.007 0.007 61.414 61.414
django/core/management/commands/migrate.py:55(handle)
1 0.000 0.000 60.634 60.634
django/db/migrations/executor.py:65(migrate)
525 0.003 0.000 51.490 0.098
django/db/migrations/operations/fields.py:43(state_forwards)
582 0.030 0.000 51.487 0.088
django/db/migrations/state.py:88(reload_model)
54 0.016 0.000 39.538 0.732
django/db/migrations/state.py:240(render_multiple)
6174 0.251 0.000 39.521 0.006
django/db/migrations/state.py:507(render)
1 0.005 0.005 33.287 33.287
django/db/migrations/executor.py:132(apply_migration)
1 0.055 0.055 33.260 33.260
django/db/migrations/migration.py:86(apply)
15525 1.869 0.000 25.929 0.002
django/apps/registry.py:334(clear_cache)
40 0.001 0.000 25.852 0.646
django/db/migrations/migration.py:72(mutate_state)
7889/6140 0.648 0.000 23.327 0.004
django/db/models/base.py:72(__new__)
137224 0.725 0.000 22.687 0.000
django/db/migrations/state.py:476(construct_fields)
2049773 19.569 0.000 19.569 0.000
django/db/models/options.py:711(_expire_cache)
7889 0.040 0.000 13.726 0.002
django/db/migrations/state.py:273(register_model)
7616 0.028 0.000 12.333 0.002
django/db/migrations/state.py:281(unregister_model)
25408 0.135 0.000 10.822 0.000
django/db/models/fields/related.py:1867(deconstruct)
25408 0.250 0.000 10.687 0.000
django/db/models/fields/related.py:1551(deconstruct)
28344 5.208 0.000 10.398 0.000
django/db/models/fields/related.py:309(swappable_setting)
43915 0.147 0.000 9.827 0.000
django/utils/lru_cache.py:94(wrapper)
21213 1.501 0.000 9.416 0.000
django/apps/registry.py:153(get_models)
118321/106078 0.231 0.000 9.062 0.000
django/db/models/base.py:321(add_to_class)
3985834 3.512 0.000 7.534 0.000
django/apps/config.py:164(get_models)
19 0.001 0.000 7.105 0.374
django/db/migrations/state.py:142(clone)
19 0.007 0.000 6.920 0.364
django/db/migrations/state.py:145(<dictcomp>)
3572 0.056 0.000 6.913 0.002
django/db/migrations/state.py:496(clone)
127520 4.815 0.000 5.706 0.000
django/db/models/fields/__init__.py:358(deconstruct)
1891 0.024 0.000 5.029 0.003
django/db/models/fields/related.py:2518(contribute_to_class)
1749 0.092 0.000 4.856 0.003
django/db/models/fields/related.py:2048(create_many_to_many_intermediary
_model)
741163 2.485 0.000 3.776 0.000
/usr/lib/python2.7/collections.py:104(values)
}}}
We also have a initial migration that adds ~400 FK fields and it takes up
to 10 minutes to execute.
--
Ticket URL: <https://code.djangoproject.com/ticket/24670>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/050.eae875b537e843e7df3f03254e26b18a%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
