Re: slow migrations

2017-02-07 Thread charettes
Hello Venelin, Thanks for thanking the time to investigate the source of the slowdown you were experiencing on your project. I know you mentioned it would require a lot of effort to port your project to Django 1.10 but I'm pretty sure recent changes to RenameModel.state_forward would solve

Re: slow migrations

2017-02-07 Thread Venelin Stoykov
Hello all, I had the same problem with Django 1.8 project with 76 applications and 229 migrations in total. Even when the project was fully migrated and you execute *manage.py migrate* it takes 90 seconds after the message "*No migrations to apply.*" to finish. If I upgrade to Django 1.9 then

Re: slow migrations

2016-01-10 Thread Shai Berger
On Sunday 10 January 2016 08:09:33 Carl Meyer wrote: > On 01/09/2016 09:29 AM, Shai Berger wrote: > > There are two kinds of data migrations, generally speaking. > > > > [...] > > > > The other is migrations which *create* data -- fill in tables for > > database- implemented-enums, for example.

Re: slow migrations

2016-01-10 Thread René Fleschenberg
Hi, On Saturday 09 January 2016 18:29:32 Shai Berger wrote: > The other is migrations which *create* data -- fill in tables for > database- implemented-enums, for example. If you remove these, you are > going to break your tests (if you do this and haven't broken your tests, > your tests are

Re: slow migrations

2016-01-09 Thread Carl Meyer
On 01/09/2016 09:29 AM, Shai Berger wrote: > There are two kinds of data migrations, generally speaking. > > One is the kind that is needed to facilitate some schema migrations. Changing > the type of some field, for example, usually involves creating the new field > (schema migration),

Re: slow migrations

2016-01-09 Thread charettes
Shai, I think I have a viable solution for the the second kind of data migration your are mentioning. https://code.djangoproject.com/ticket/26064#ticket Simon Le samedi 9 janvier 2016 11:30:01 UTC-5, Shai Berger a écrit : > > On Saturday 09 January 2016 04:56:11 'Hugo Osvaldo Barrera' via

Re: slow migrations

2016-01-09 Thread Shai Berger
On Saturday 09 January 2016 04:56:11 'Hugo Osvaldo Barrera' via Django developers (Contributions to Django itself) wrote: > > In my case, once data migrations have run in staging/production, they're > useless and can be ignored forever, so there's no point in keeping them > in later squashed

Re: slow migrations

2016-01-08 Thread 'Hugo Osvaldo Barrera' via Django developers (Contributions to Django itself)
On Thu, Jan 7, 2016, at 19:12, Carl Meyer wrote: > On 01/07/2016 03:03 PM, Aymeric Augustin wrote: > > As far as I understand, the CPU cost comes from generating a full set of > > model classes for each step of the migration history. That’s consistent > > with the profile sent by Florian. > > >

Re: slow migrations

2016-01-08 Thread Markus Holtermann
Yes, the more relationships your models have the more time expensive migrations become. Adding/Altering/Removing a field (ForeignKey or not) always requires all related models to be recreated. See https://github.com/django/django/blob/master/django/db/migrations/state.py#L53 for the reasoning

Re: slow migrations

2016-01-07 Thread Peter Baumgartner
Reporting back on some additional findings for what it's worth. SmileyChris dumped and recreated the project in question's migrations, manually ordering them to minimize dependencies. It was not a major reduction in total migrations: 82 to 58 (mostly third-party migrations and initials), but the

Re: slow migrations

2016-01-07 Thread Carl Meyer
On 01/07/2016 03:03 PM, Aymeric Augustin wrote: > As far as I understand, the CPU cost comes from generating a full set of > model classes for each step of the migration history. That’s consistent > with the profile sent by Florian. > > I usually end up throwing away the migration history and

Re: slow migrations

2016-01-07 Thread Andrew Godwin
Yes, it's basically a fundamental design flaw of having the migrations represented this way - it makes autodetection and code generation very accurate, but at the expense of calculation time. There is some optimisation work that can be done to try and avoid building intermediate states, but it's a

Re: slow migrations

2016-01-07 Thread Aymeric Augustin
As far as I understand, the CPU cost comes from generating a full set of model classes for each step of the migration history. That’s consistent with the profile sent by Florian. I usually end up throwing away the migration history and regenerating a new set of migrations when I get to that

Re: slow migrations

2016-01-07 Thread Florian Apolloner
Also, is there any chance that I can get access to this project to profile a little bit more and identify some hotspots? On Thursday, January 7, 2016 at 9:10:26 PM UTC+1, Florian Apolloner wrote: > > Just looking at the topmost three entries: > *_expire_cache* spends 18 second in delattr, can

Re: slow migrations

2016-01-07 Thread Tim Graham
As far as I know, no one is actively working on this area at the moment. There is at least one open issue: https://code.djangoproject.com/ticket/22608 You can look through all open migrations tickets too: