Re: Migrations in Django 1.7 make unit testing models harder

There is no such flag, at least not in 1.11. I wrote my own "migratetest" and "cleandb" commands because my DBAs don't let me drop and recreate databases. It is a simple matter if you use the testsuite, but it would probably be better development to call create_test_db manually and such. On

Re: Migrations in Django 1.7 make unit testing models harder

Thanks all! I remember reading about the zero downtime deployments many months back and just needed a refresher. I appreciate it. To sum up for myself: we need to test at every step of the change that the model changes are in fact backwards compatible with the old state of the DB. Makes sense.

Re: Migrations in Django 1.7 make unit testing models harder

On Sat, Sep 14, 2019, 12:44 Adam Johnson wrote: > (Fixed Ryan's link: https://github.com/aspiredu/django-safemigrate ) > Doh. Thanks. > -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe

Re: Migrations in Django 1.7 make unit testing models harder

(Fixed Ryan's link: https://github.com/aspiredu/django-safemigrate ) On Sat, 14 Sep 2019 at 17:12, Adam Johnson wrote: > I'm interested in your use case: why do you need the model changes without >> the migrations? I.e. why would you want to be in a state that's >> inconsistent between the ORM

Re: Migrations in Django 1.7 make unit testing models harder

> > I'm interested in your use case: why do you need the model changes without > the migrations? I.e. why would you want to be in a state that's > inconsistent between the ORM and the DB? Is it just so you can recover > more easily if something goes wrong? > It's not that one *wants* to be in

Re: Migrations in Django 1.7 make unit testing models harder

On Sat, Sep 14, 2019 at 10:09 AM Dylan Young wrote: > why do you need the model changes without the migrations? I.e. why would > you want to be in a state that's inconsistent between the ORM and the DB? > Is it just so you can recover more easily if something goes wrong? > I expect that its to

Re: Migrations in Django 1.7 make unit testing models harder

Thanks for the tips; I'll take a look at those code paths. I'm interested in your use case: why do you need the model changes without the migrations? I.e. why would you want to be in a state that's inconsistent between the ORM and the DB? Is it just so you can recover more easily if something

Re: Migrations in Django 1.7 make unit testing models harder

Hi Dylan, In my experience, high availability deployments often need to commit model changes without migration changes. For example, when removing a (nulable) field, the best approach is to: 1. Remove the field from Django 2. Test 3. Deploy 4. Run makemigrations to generate the

Re: Migrations in Django 1.7 make unit testing models harder

This is an old issue that (as far as I can see) was never resolved. My proposed solution would be a flag to the test command to automatically `makemigrations && migrate`. When this behaviour is enabled, it would also roll back any generated migrations at the end of the test run. The flag

Re: Migrations in Django 1.7 make unit testing models harder

I have found a good alternative to the old initial data feature. It's called django-update-initial-data. https://github.com/ambitioninc/django-dynamic-initial-data MySQL compatible version: https://github.com/minervaproject/django-dynamic-initial-data/tree/mysql It's actually better than the

Re: Migrations in Django 1.7 make unit testing models harder

I don't know of any tickets or plans to address this. On Wednesday, May 13, 2015 at 12:51:59 AM UTC-4, Pradeek J wrote: > > I'm facing the same speed issue here. Like Andrew mentioned above, if > initial data is going to be removed and data migrations are the way > forward, even having an

Re: Migrations in Django 1.7 make unit testing models harder

I'm facing the same speed issue here. Like Andrew mentioned above, if initial data is going to be removed and data migrations are the way forward, even having an option to skip migrations is a problem because we'd need that data to be populated. Is there something planned for this for Django

Re: Migrations in Django 1.7 make unit testing models harder

At work we use a --no-migrate flag on our test runner. It's helpful when 1) we're developing a new app but the models aren't nailed down yet, and 2) it speeds up the test suite time significantly. The --keepdb option is sufficient for scenario 2, but doesn't do much for scenario 1. Admittedly

Re: Migrations in Django 1.7 make unit testing models harder

In pull request 3791 , I started working on the code removals for Django 1.9 to better understand how the database/migrations code will look and what APIs we still need to deprecate. The good news is that we can keep support for apps without

Re: Migrations in Django 1.7 make unit testing models harder

There's django-admin test --keepdb for this purpose (in Django 1.8). -- Aymeric. > Le 20 déc. 2014 à 18:41, Bernard Sumption a écrit : > > A proposition: the problem isn't that migrations take 80 seconds, it's that > this 80 seconds is repeated every time a developer

Re: Migrations in Django 1.7 make unit testing models harder

I did some research, and I can hit the logic in question, if app_label == settings.AUTH_USER_MODEL and ignore_swappable: continue ... on both master and with Claude's patch if I follow the steps in this comment: https://code.djangoproject.com/ticket/22563#comment:8 Andrew, first you added

Re: Migrations in Django 1.7 make unit testing models harder

A proposition: the problem isn't that migrations take 80 seconds, it's that this 80 seconds is repeated every time a developer runs tests. How about serialising the structure and content of the database to disk after applying migrations, then whenever you need a migrated test database, load it

Re: Migrations in Django 1.7 make unit testing models harder

On Friday, December 19, 2014 6:30:32 PM UTC+1, Tim Graham wrote: > > Yes. Claude has worked on the deprecation in > https://github.com/django/django/pull/3220 but it requires adding more > migrations to our test suite and he noted that each migration we add to > Django's test suite adds up to

Re: Migrations in Django 1.7 make unit testing models harder

For what it's worth, for my team, it just comes down to speed (so a single in-memory migration per app is great, and so is speeding up migrations overall). I have a stopwatch at my desk and did some quick (unscientific) timing of creating the test DB for my own project: *Without migrations*

Re: Migrations in Django 1.7 make unit testing models harder

Yes. Claude has worked on the deprecation in https://github.com/django/django/pull/3220 but it requires adding more migrations to our test suite and he noted that each migration we add to Django's test suite adds up to ~3.5 seconds to the run time of the test suite. He also worked on some

Re: Migrations in Django 1.7 make unit testing models harder

Hi Tim, I can have a look, but I can't be certain about hitting any deadlines. I do want to get that deprecation in, though... Did you want it with a view to us being able to drop that in for tests rather than making migrations for every test app, I presume? Andrew On Fri, Dec 19, 2014 at 3:06

Re: Migrations in Django 1.7 make unit testing models harder

Andrew, I've thought of something similar to the in-memory migrations idea you've proposed. It would be great not to have to add and maintain migrations for all of the apps in Django's test suite. Do you think you might be able to investigate this solution in the next month or so before 1.8

Re: Migrations in Django 1.7 make unit testing models harder

I agree that migrations are slower than syncdb - that's perhaps the only thing worse about them - but the reason we plan to deprecate the other methods is code simplicity; migrations does not share almost any code with the old DatabaseCreation backends, and if we don't deprecate it we're going to

Re: Migrations in Django 1.7 make unit testing models harder

At the risk of reviving an old topic, I wanted to add one significant point in favor of (and mitigation for) running tests with migrations disabled: Speed. Creating a test DB in sqlite for our project (~100 database tables) takes approximately 1-2 minutes on most developer machines. 1-2

Re: Migrations in Django 1.7 make unit testing models harder

Well, having stayed up til 4AM on Monday night, I managed to get the whole book upgraded to Django 1.7, and published. I've shared some thoughts on the migrations / tdd stuff we've been discussing, here: http://www.obeythetestinggoat.com/book-upgraded-to-django-17.html I'd be very interested

Re: Migrations in Django 1.7 make unit testing models harder

Yes, you can - the "migrations or not" switch is currently "is there a migrations directory", but bear in mind this will eventually turn from "use the old way" to "ignore it completely", probably also in 1.9. Andrew On Mon, Mar 31, 2014 at 4:15 PM, Harry Percival

Re: Migrations in Django 1.7 make unit testing models harder

Just found out that you can make Django behave in the "old way" by just deleting the migrations folder when you first do your `startapp`, and then you can pretend migrations don't exist until you need them. Don't think that's necessarily a good idea though... On 30 March 2014 19:42, Andrew

Re: Migrations in Django 1.7 make unit testing models harder

You're roughly right, yes. String fields are a little odd, though, in that you can have them blank=True without null=True and then the default should be an empty string (Django's separation of blank and null irks me still) - the migrations should correctly detect this and insert blank strings for

Re: Migrations in Django 1.7 make unit testing models harder

On Sunday 30 March 2014 15:08:17 Harry Percival wrote: > Ah, so the reason I was confused is because it *looks* like the default is > the empty string, because that's what you get if you initialise an object, > by default. But at the database level, the default for the column is NULL. > Is that

Re: Migrations in Django 1.7 make unit testing models harder

Ah, so the reason I was confused is because it *looks* like the default is the empty string, because that's what you get if you initialise an object, by default. But at the database level, the default for the column is NULL. Is that right? So, I realise we're getting sidetracked here, but, how

Re: Migrations in Django 1.7 make unit testing models harder

> > I thought TextField did have a default, the empty string? > > Like every other field, the "default default" is None (NULL). -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from

Re: Migrations in Django 1.7 make unit testing models harder

OK, I've now run into the problem IRL, and sure enough, it's different to what i'm used to. Am trying to overcome my knee-jerk reactions of "why did it change! i hate it!". ignoring that for a moment then: One thing I did find surprising was that I'm going thru TDD in small steps like this:

Re: Migrations in Django 1.7 make unit testing models harder

I suspect you're probably right. Having to run makemigrations in between making changes to model code and running tests isn't the end of the world i suppose. Will know better what I'm talking about when I've actually got to that part of the book... On 29 March 2014 18:23, Andrew Godwin

Re: Migrations in Django 1.7 make unit testing models harder

No, there is no way to turn off migrations for tests - some of the core tests won't work without them turned on, in fact, and adding that option would be weird (why only tests? what would it do? how do you load data in now initial_data is gone?). The only complaint I've seen - the one that Bernie

Re: Migrations in Django 1.7 make unit testing models harder

Am just working on updating my book on TDD to django 1.7 based on the beta. Currently half-way thru, not run into any problems because I don't use migrations until a later chapter, but when I do I will run into the same problems Bernie mentions. Will share more once I've finished the

Re: Migrations in Django 1.7 make unit testing models harder

> > South's `--update` also rolled the previous migration back, changed it and > then reapplied it to the current database. > OK, in that case I can very much see how it's useful for people who develop against a persistent database. That's probably most people. Anyway, the result of this

Re: Migrations in Django 1.7 make unit testing models harder

South's `--update` also rolled the previous migration back, changed it and then reapplied it to the current database. M On 28 March 2014 10:48, Bernie Sumption wrote: > > That script would be bad if you'd run any of those migrations against your >> development db (yes

Re: Migrations in Django 1.7 make unit testing models harder

> That script would be bad if you'd run any of those migrations against your > development db (yes it should be "throwaway" or rebuildable but...) > I'd think the same could be said of --update? As I understand it, --update is the equivalent of deleting the most recent migration and

Re: Migrations in Django 1.7 make unit testing models harder

That script would be bad if you'd run any of those migrations against your development db (yes it should be "throwaway" or rebuildable but...) Personally, I'm strongly in favour of Shai's suggestion and also in favour of --update, mainly as I like being able to capture (most) migrations has

Re: Migrations in Django 1.7 make unit testing models harder

OK, it turns out that the "safe update migrations script" too simple to even qualify as a "script": git clean myapp/migrations -f && python manage.py makemigrations Perhaps the solution is to document this on the testing page as a solution to the "accumulation of many small migrations during

Re: Migrations in Django 1.7 make unit testing models harder

The problem with --update is that if overwrites the most recent migration, then it might be used to modify a committed and distributed migration, which is a Bad Thing. The flag would probably be useful to people with my use case, if they trust themselves to use the flag with care and remember

Re: Migrations in Django 1.7 make unit testing models harder

If I need to take either of these options I'd tend towards Shai's one - we don't want to allow people to just disable migrations on a per-database basis, that's bound to get someone into trouble. That said, the flag is going to be weird to explain to people. Just to establish a baseline, would

Re: Migrations in Django 1.7 make unit testing models harder

On Thursday 27 March 2014 01:39:53 Bernie Sumption wrote: > > I'd be willing to keep the current contract of "things without a > > migrations directory don't get migrated", but I suspect you're doing > > things > > on apps that already have migrations (which makes my reticence to add a > > setting

Re: Migrations in Django 1.7 make unit testing models harder

> > I'd be willing to keep the current contract of "things without a > migrations directory don't get migrated", but I suspect you're doing things > on apps that already have migrations (which makes my reticence to add a > setting even bigger - if you "syncdb" an app with migrations to a main

Re: Migrations in Django 1.7 make unit testing models harder

When I practice TDD I write the test to spec, and then write the model and view code, so I usually have about the same amount of model changes as otherwise (as, having written the tests, I usually have a clearer idea of what fields I need). I agree that if you're incrementally writing tests on top

Re: Migrations in Django 1.7 make unit testing models harder

> > we can't promote adding random strings to MIGRATION_MODULES as the > suggested way to "get around" migrations for tests. > I agree, my workaround is a hack. It would be better to introduce a flag or setting designed specifically for this use case. > In my opinion, the whole point of

Re: Migrations in Django 1.7 make unit testing models harder

On Tue, Mar 25, 2014 at 4:02 PM, Florian Apolloner wrote: > On Tuesday, March 25, 2014 9:12:42 PM UTC+1, Andrew Godwin wrote: >> >> So, the functionality whereby you can have apps which do not use >> migrations (i.e. that use the old creation backends) is meant to go away

Re: Migrations in Django 1.7 make unit testing models harder

On Tuesday, March 25, 2014 9:12:42 PM UTC+1, Andrew Godwin wrote: > > So, the functionality whereby you can have apps which do not use > migrations (i.e. that use the old creation backends) is meant to go away in > 1.9. > Uhm, strong -1 here unless you have really convincing arguments, I really

Re: Migrations in Django 1.7 make unit testing models harder

I'll update the deprecation document to include more direct information about DatabaseCreation and the legacy app sync method. I'm not sure "TDD is a bit harder" is a release blocker - the TDD I do generally doesn't have that much model creation, and it's relatively easy to just run

Re: Migrations in Django 1.7 make unit testing models harder

I don't see any meaningful notes about apps being required to ship migrations beginning with 1.9. I do see deprecation notes about the syncdb signals changing and the syncdb command itself is clearly deprecated. This legacy behavior is handled by sync_apps in the migrate command but there

Re: Migrations in Django 1.7 make unit testing models harder

I just read the deprecation timeline and the very brief mention of syncdb command and signals going away doesn't really seem to sufficiently detail the "side-effects" you mention. Anyone who hasn't read your email is going to be unpleasantly surprised. I also don't see any deprecation warnings in

Re: Migrations in Django 1.7 make unit testing models harder

So, the functionality whereby you can have apps which do not use migrations (i.e. that use the old creation backends) is meant to go away in 1.9 (i.e. the standard three-release deprecation cycle). Most of the side-effects of this are detailed in

Re: Migrations in Django 1.7 make unit testing models harder

Andrew, Can you clarify exactly what is deprecated in this work-around? I don't see anything in the deprecation timeline to remove MIGRATION_MODULES nor any pending deprecations related to its usage. It seems like could probably be replaced by something that uses the app-loading/app-configs

Re: Migrations in Django 1.7 make unit testing models harder

On Tuesday, March 25, 2014 7:14:55 PM UTC+1, Andrew Godwin wrote: > > Yes, the new behaviour is by design, in the sense that the workaround you > mentioned will be deprecated in 1.9 (along with all syncdb-related > functionality). > What exactly will get deprecated here? -- You received this

Re: Migrations in Django 1.7 make unit testing models harder

Do we have an equivalent of south's --update? This would mean you don't get many files. We don't want to make it too hard for people to work in a strict TDD fashion. M On 25 Mar 2014 18:15, "Andrew Godwin" wrote: > Yes, the new behaviour is by design, in the sense that the

Re: Migrations in Django 1.7 make unit testing models harder

Yes, the new behaviour is by design, in the sense that the workaround you mentioned will be deprecated in 1.9 (along with all syncdb-related functionality). This way, tests always run against the version of your models that production would run, so you don't run the risk of the tests passing