Re: Data Migrations and Testing

[Andrew Godwin]

If you read the section on this:
https://docs.djangoproject.com/en/dev/topics/testing/overview/#test-case-serialized-rollback
the emulation is only performed if you set serialized_rollback = True on
the TestCase (as not all tests need this and it slows tests down by 3x, so
we decided to make it opt-in not opt-out). Did you turn that on and it's
still failing?

Andrew


On Sat, Jun 21, 2014 at 5:07 PM, Rafał Pitoń  wrote:

> Ticket claims this is fixed, but as I've moved my project to Django 1.7,
> I've seen story from OP's mail repeat. My migrations create tables, then
> populate them with data. Then tests that read this data from DB blurp with
> "DoesNotExist".
>
> > Any initial data loaded in migrations will only be available in TestCase 
> > tests
> and not in TransactionTestCase tests, and additionally only on backends
> where transactions are supported (the most important exception being
> MyISAM).
>
> My tests inherit from TestCase and are ran against @PostgreSQL 9.3 which
> according to doc should support migrations data.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/7e071bc2-c563-439a-9f8f-88e24d150945%40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFwN1uocbMcOjvDr6JwqHNbXZ3qYEMqRoPKE6ee9Kd3wrh1T0w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Data Migrations and Testing

[Rafał Pitoń]

Ticket claims this is fixed, but as I've moved my project to Django 1.7, 
I've seen story from OP's mail repeat. My migrations create tables, then 
populate them with data. Then tests that read this data from DB blurp with 
"DoesNotExist".

> Any initial data loaded in migrations will only be available in TestCase 
> tests 
and not in TransactionTestCase tests, and additionally only on backends 
where transactions are supported (the most important exception being 
MyISAM).

My tests inherit from TestCase and are ran against @PostgreSQL 9.3 which 
according to doc should support migrations data.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/7e071bc2-c563-439a-9f8f-88e24d150945%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Data Migrations and Testing

[Andrew Godwin]

There's two balancing forces here - I don't want to delay the release as we
always do that, but then again we always make sure we ship correct, not on
time.

I'll investigate this as much as I can this week, and see if I can get the
dump-and-restore-to-fixture stuff working, as I think that's the best
solution we have here, as it "emulates" the right behaviour on databases
with no transactions and should do the right thing for
TransactionTestCases. I've opened a ticket here:
https://code.djangoproject.com/ticket/22487

Andrew


On Tue, Apr 22, 2014 at 2:02 AM, Christian Schmitt  wrote:

> Hm, I’m not a core developer, but i wouldn’t prefer the documentation that
> explains the edge cases.
>
> As Russel Keith-Magee already said that Django is a tool that ensures
> quality.
> We should definitly make a Ticket and try to clear that thing up before
> the 1.7 release. I mean this is not a new feature since Django 1.7 should
> already support Data Migrations even in TestCases.
>
> Also I don’t think that people will read every part of the documentation.
> I don’t know how many people will use Migrations and especially Data
> Migrations and rely on Django’s TestCases but i don’t think that i’m the
> only one, so the 1.7 release wouldn’t be that useful for these people as it
> will be to other people that don’t use data migrations and/or don’t test
> their applications.
>
> —
> Best Regards
>
> Christian Schmitt
> c.schm...@briefdomain.de
>
>
>
>
> Am 22.04.2014 um 03:22 schrieb Russell Keith-Magee <
> russ...@keith-magee.com>:
>
> deferred
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFwN1upf4tve3ZW8Yufb8QCL3Q8VJdmpDTSbY%3DsQMT40a9y0ww%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Data Migrations and Testing

[Christian Schmitt]

Hm, I’m not a core developer, but i wouldn’t prefer the documentation that 
explains the edge cases.

As Russel Keith-Magee already said that Django is a tool that ensures quality.
We should definitly make a Ticket and try to clear that thing up before the 1.7 
release. I mean this is not a new feature since Django 1.7 should already 
support Data Migrations even in TestCases.

Also I don’t think that people will read every part of the documentation. I 
don’t know how many people will use Migrations and especially Data Migrations 
and rely on Django’s TestCases but i don’t think that i’m the only one, so the 
1.7 release wouldn’t be that useful for these people as it will be to other 
people that don’t use data migrations and/or don’t test their applications.

—
Best Regards

Christian Schmitt
c.schm...@briefdomain.de




Am 22.04.2014 um 03:22 schrieb Russell Keith-Magee :

> deferred



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Data Migrations and Testing

[Russell Keith-Magee]

On Mon, Apr 21, 2014 at 11:35 PM, Andrew Godwin  wrote:

> Yes, no matter what it's too late to add anything to 1.7, which is a
> massive shame - we'll have to just heavily document this for now, and then
> investigate the dump/load data stuff for the next cycle (I think it should
> work everywhere initial_data did, at least, as they're both fixtures).
>

Are we absolutely sure about this? Process is meant to be a tool to ensure
quality, not the other way around.

Put it this way - if we push out 1.7 with migrations with their current
feature set, are we going to be mobbed by people saying that migrations are
broken?

As much as a deferred release would pain me, I'd rather see a few weeks
delay than release documentation that has to explain a bunch of edge cases
that get cleaned up in the next release.

Yours,
Russ Magee %-)

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAJxq8499%3D7ytzMEBbjmtPd7zgc4JJKn1HiC8Fvjf6QErgP_qGQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Data Migrations and Testing

[Andrew Godwin]

Yes, the test suite is basically the biggest obstacle to full usage of
migrations - I've tried to make it see sense, but given my limited time at
the moment and the fact that it's a tortuous mess of hacks in places it
means that I can't see that happening before the RC.

I'd like to have started the Creation deprecation cycle this release, but
it's probably fine if we let that slip a release (it's going to be harder
to remove than many other features). Migrations will hopefully drive the
adoption of SchemaEditor by developers and third-party backends anyway.

Andrew


On Mon, Apr 21, 2014 at 9:21 AM, Tim Graham  wrote:

> I have been thinking that maybe we should delay #22340 (Legacy Table
> Creation Methods Not Properly Deprecated) until 1.8 so that migrations
> won't be required until Django 2.0. I'm not sure how feasible it is to
> remove Django's test suite usage of it in the next week and a half, if we
> are still shooting for RC1 on May 1 or shortly thereafter.
>
>
> On Monday, April 21, 2014 12:06:10 PM UTC-4, Michael Manfre wrote:
>
>> Are you thinking the next cycle would be 1.7.1 or 1.8?
>>
>>
>> On Mon, Apr 21, 2014 at 11:35 AM, Andrew Godwin wrote:
>>
>>> Yes, no matter what it's too late to add anything to 1.7, which is a
>>> massive shame - we'll have to just heavily document this for now, and then
>>> investigate the dump/load data stuff for the next cycle (I think it should
>>> work everywhere initial_data did, at least, as they're both fixtures).
>>>
>>> I will try and remove that first flush before tests start, though, which
>>> would at least let you use the data inside normal TestCases on
>>> transactional DBs, though I fear removing it might be too
>>> backwards-incompatible.
>>>
>>> Andrew
>>>
>>>
>>> On Sun, Apr 20, 2014 at 12:23 AM, Anssi Kääriäinen 
>>> wrote:
>>>
 On 04/19/2014 06:54 PM, Andrew Godwin wrote:

> I agree we can just say that initial_data can be used as a fixture for
> tests rather than being auto-loaded - and we could perhaps even put it in
> the base testcase so it always auto-applied somehow - but that doesn't get
> over the fact that you can't rely on data migrations to set up your
> database for tests (for example, to add in required base rows or choices 
> to
> EAV designs).
>
> I remember we reordered TransactionTestCase with respect to TestCase -
> what did we do? If we had them all at the end then we could at least have
> things work for TestCase if we remove that initial flush, but 
> unfortunately
> I suspect option 3 might have to be the solution, and I really don't want
> to do anything with fixtures (though I guess the only performance hit is
> writing the fixture, as we would have been reading one in every time 
> before
> anyway - initial_data). Perhaps we could cache the fixture in memory and
> then spool out to disk if it gets too big.
>
>  The test case ordering is currently TestCase, SimpleTestCase and then
 the rest (including TransactionTestCases).

 Using fixtures for dumping contents of the DB before first flush, and
 then reloading it back after each flush seems to be doable. I am not sure
 how well it will perform, or if it will be actually backwards compatible
 for users in all cases. It is extremely likely there will be at least some
 corner cases that can't be handled by simple dump-and-reload, so there is
 likely more work than just adding the dump and reload calls in to Django.

 As for spooling out to disk - I don't think we need to do that. If the
 file is small enough to stay in memory, then reading from the file should
 be fast, too. If it is large, then the operating system knows better than
 us when to write it out to disk. In addition, if a fixture file is too
 large to stay in memory, then dump-and-restore will be too slow to be
 usable in any case.

 If we want to support large amounts of data, then an approach I worked
 on to speed up Django's test suite some time ago might be useful. The idea
 was that only tables which were changed by a test were flushed after that
 test. It worked pretty well, but it was also somewhat complex. However, if
 we add an ability to have static data for tests this might need a revisit.
 The approach should be extremely effective for static test data. If you
 have any large amount of static test data then you don't want to reload it
 after each transactional test.

 I have had a need for static data for tests in some projects, so a big
 +1 for this feature. I don't see how we can make this feature reliable for
 1.7 in the time available. Can we punt this to 1.8?


  - Anssi

 --
 You received this message because you are subscribed to the Google
 Groups "Django developers" group.
 To unsubscribe from this 

Re: Data Migrations and Testing

[Tim Graham]

I have been thinking that maybe we should delay #22340 (Legacy Table 
Creation Methods Not Properly Deprecated) until 1.8 so that migrations 
won't be required until Django 2.0. I'm not sure how feasible it is to 
remove Django's test suite usage of it in the next week and a half, if we 
are still shooting for RC1 on May 1 or shortly thereafter.

On Monday, April 21, 2014 12:06:10 PM UTC-4, Michael Manfre wrote:
>
> Are you thinking the next cycle would be 1.7.1 or 1.8?
>
>
> On Mon, Apr 21, 2014 at 11:35 AM, Andrew Godwin 
>  > wrote:
>
>> Yes, no matter what it's too late to add anything to 1.7, which is a 
>> massive shame - we'll have to just heavily document this for now, and then 
>> investigate the dump/load data stuff for the next cycle (I think it should 
>> work everywhere initial_data did, at least, as they're both fixtures).
>>
>> I will try and remove that first flush before tests start, though, which 
>> would at least let you use the data inside normal TestCases on 
>> transactional DBs, though I fear removing it might be too 
>> backwards-incompatible.
>>
>> Andrew
>>
>>
>> On Sun, Apr 20, 2014 at 12:23 AM, Anssi Kääriäinen 
>> > > wrote:
>>
>>> On 04/19/2014 06:54 PM, Andrew Godwin wrote:
>>>
 I agree we can just say that initial_data can be used as a fixture for 
 tests rather than being auto-loaded - and we could perhaps even put it in 
 the base testcase so it always auto-applied somehow - but that doesn't get 
 over the fact that you can't rely on data migrations to set up your 
 database for tests (for example, to add in required base rows or choices 
 to 
 EAV designs).

 I remember we reordered TransactionTestCase with respect to TestCase - 
 what did we do? If we had them all at the end then we could at least have 
 things work for TestCase if we remove that initial flush, but 
 unfortunately 
 I suspect option 3 might have to be the solution, and I really don't want 
 to do anything with fixtures (though I guess the only performance hit is 
 writing the fixture, as we would have been reading one in every time 
 before 
 anyway - initial_data). Perhaps we could cache the fixture in memory and 
 then spool out to disk if it gets too big.

  The test case ordering is currently TestCase, SimpleTestCase and then 
>>> the rest (including TransactionTestCases).
>>>
>>> Using fixtures for dumping contents of the DB before first flush, and 
>>> then reloading it back after each flush seems to be doable. I am not sure 
>>> how well it will perform, or if it will be actually backwards compatible 
>>> for users in all cases. It is extremely likely there will be at least some 
>>> corner cases that can't be handled by simple dump-and-reload, so there is 
>>> likely more work than just adding the dump and reload calls in to Django.
>>>
>>> As for spooling out to disk - I don't think we need to do that. If the 
>>> file is small enough to stay in memory, then reading from the file should 
>>> be fast, too. If it is large, then the operating system knows better than 
>>> us when to write it out to disk. In addition, if a fixture file is too 
>>> large to stay in memory, then dump-and-restore will be too slow to be 
>>> usable in any case.
>>>
>>> If we want to support large amounts of data, then an approach I worked 
>>> on to speed up Django's test suite some time ago might be useful. The idea 
>>> was that only tables which were changed by a test were flushed after that 
>>> test. It worked pretty well, but it was also somewhat complex. However, if 
>>> we add an ability to have static data for tests this might need a revisit. 
>>> The approach should be extremely effective for static test data. If you 
>>> have any large amount of static test data then you don't want to reload it 
>>> after each transactional test.
>>>
>>> I have had a need for static data for tests in some projects, so a big 
>>> +1 for this feature. I don't see how we can make this feature reliable for 
>>> 1.7 in the time available. Can we punt this to 1.8?
>>>
>>>
>>>  - Anssi
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Django developers" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to django-develop...@googlegroups.com .
>>> To post to this group, send email to 
>>> django-d...@googlegroups.com
>>> .
>>> Visit this group at http://groups.google.com/group/django-developers.
>>> To view this discussion on the web visit https://groups.google.com/d/
>>> msgid/django-developers/535375E1.8060602%40thl.fi.
>>>
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>  -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to 

Re: Data Migrations and Testing

[Michael Manfre]

Are you thinking the next cycle would be 1.7.1 or 1.8?


On Mon, Apr 21, 2014 at 11:35 AM, Andrew Godwin  wrote:

> Yes, no matter what it's too late to add anything to 1.7, which is a
> massive shame - we'll have to just heavily document this for now, and then
> investigate the dump/load data stuff for the next cycle (I think it should
> work everywhere initial_data did, at least, as they're both fixtures).
>
> I will try and remove that first flush before tests start, though, which
> would at least let you use the data inside normal TestCases on
> transactional DBs, though I fear removing it might be too
> backwards-incompatible.
>
> Andrew
>
>
> On Sun, Apr 20, 2014 at 12:23 AM, Anssi Kääriäinen <
> anssi.kaariai...@thl.fi> wrote:
>
>> On 04/19/2014 06:54 PM, Andrew Godwin wrote:
>>
>>> I agree we can just say that initial_data can be used as a fixture for
>>> tests rather than being auto-loaded - and we could perhaps even put it in
>>> the base testcase so it always auto-applied somehow - but that doesn't get
>>> over the fact that you can't rely on data migrations to set up your
>>> database for tests (for example, to add in required base rows or choices to
>>> EAV designs).
>>>
>>> I remember we reordered TransactionTestCase with respect to TestCase -
>>> what did we do? If we had them all at the end then we could at least have
>>> things work for TestCase if we remove that initial flush, but unfortunately
>>> I suspect option 3 might have to be the solution, and I really don't want
>>> to do anything with fixtures (though I guess the only performance hit is
>>> writing the fixture, as we would have been reading one in every time before
>>> anyway - initial_data). Perhaps we could cache the fixture in memory and
>>> then spool out to disk if it gets too big.
>>>
>>>  The test case ordering is currently TestCase, SimpleTestCase and then
>> the rest (including TransactionTestCases).
>>
>> Using fixtures for dumping contents of the DB before first flush, and
>> then reloading it back after each flush seems to be doable. I am not sure
>> how well it will perform, or if it will be actually backwards compatible
>> for users in all cases. It is extremely likely there will be at least some
>> corner cases that can't be handled by simple dump-and-reload, so there is
>> likely more work than just adding the dump and reload calls in to Django.
>>
>> As for spooling out to disk - I don't think we need to do that. If the
>> file is small enough to stay in memory, then reading from the file should
>> be fast, too. If it is large, then the operating system knows better than
>> us when to write it out to disk. In addition, if a fixture file is too
>> large to stay in memory, then dump-and-restore will be too slow to be
>> usable in any case.
>>
>> If we want to support large amounts of data, then an approach I worked on
>> to speed up Django's test suite some time ago might be useful. The idea was
>> that only tables which were changed by a test were flushed after that test.
>> It worked pretty well, but it was also somewhat complex. However, if we add
>> an ability to have static data for tests this might need a revisit. The
>> approach should be extremely effective for static test data. If you have
>> any large amount of static test data then you don't want to reload it after
>> each transactional test.
>>
>> I have had a need for static data for tests in some projects, so a big +1
>> for this feature. I don't see how we can make this feature reliable for 1.7
>> in the time available. Can we punt this to 1.8?
>>
>>
>>  - Anssi
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-developers+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-developers@googlegroups.com.
>> Visit this group at http://groups.google.com/group/django-developers.
>> To view this discussion on the web visit https://groups.google.com/d/
>> msgid/django-developers/535375E1.8060602%40thl.fi.
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAFwN1uqxhOXD%2B6D0zGVpMaF2bKTX4vzwpavDHa_Y892PXrTOMw%40mail.gmail.com
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 

Re: Data Migrations and Testing

[Andrew Godwin]

Yes, no matter what it's too late to add anything to 1.7, which is a
massive shame - we'll have to just heavily document this for now, and then
investigate the dump/load data stuff for the next cycle (I think it should
work everywhere initial_data did, at least, as they're both fixtures).

I will try and remove that first flush before tests start, though, which
would at least let you use the data inside normal TestCases on
transactional DBs, though I fear removing it might be too
backwards-incompatible.

Andrew


On Sun, Apr 20, 2014 at 12:23 AM, Anssi Kääriäinen
wrote:

> On 04/19/2014 06:54 PM, Andrew Godwin wrote:
>
>> I agree we can just say that initial_data can be used as a fixture for
>> tests rather than being auto-loaded - and we could perhaps even put it in
>> the base testcase so it always auto-applied somehow - but that doesn't get
>> over the fact that you can't rely on data migrations to set up your
>> database for tests (for example, to add in required base rows or choices to
>> EAV designs).
>>
>> I remember we reordered TransactionTestCase with respect to TestCase -
>> what did we do? If we had them all at the end then we could at least have
>> things work for TestCase if we remove that initial flush, but unfortunately
>> I suspect option 3 might have to be the solution, and I really don't want
>> to do anything with fixtures (though I guess the only performance hit is
>> writing the fixture, as we would have been reading one in every time before
>> anyway - initial_data). Perhaps we could cache the fixture in memory and
>> then spool out to disk if it gets too big.
>>
>>  The test case ordering is currently TestCase, SimpleTestCase and then
> the rest (including TransactionTestCases).
>
> Using fixtures for dumping contents of the DB before first flush, and then
> reloading it back after each flush seems to be doable. I am not sure how
> well it will perform, or if it will be actually backwards compatible for
> users in all cases. It is extremely likely there will be at least some
> corner cases that can't be handled by simple dump-and-reload, so there is
> likely more work than just adding the dump and reload calls in to Django.
>
> As for spooling out to disk - I don't think we need to do that. If the
> file is small enough to stay in memory, then reading from the file should
> be fast, too. If it is large, then the operating system knows better than
> us when to write it out to disk. In addition, if a fixture file is too
> large to stay in memory, then dump-and-restore will be too slow to be
> usable in any case.
>
> If we want to support large amounts of data, then an approach I worked on
> to speed up Django's test suite some time ago might be useful. The idea was
> that only tables which were changed by a test were flushed after that test.
> It worked pretty well, but it was also somewhat complex. However, if we add
> an ability to have static data for tests this might need a revisit. The
> approach should be extremely effective for static test data. If you have
> any large amount of static test data then you don't want to reload it after
> each transactional test.
>
> I have had a need for static data for tests in some projects, so a big +1
> for this feature. I don't see how we can make this feature reliable for 1.7
> in the time available. Can we punt this to 1.8?
>
>
>  - Anssi
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/django-developers/535375E1.8060602%40thl.fi.
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFwN1uqxhOXD%2B6D0zGVpMaF2bKTX4vzwpavDHa_Y892PXrTOMw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Data Migrations and Testing

[Anssi Kääriäinen]

On 04/19/2014 06:54 PM, Andrew Godwin wrote:
I agree we can just say that initial_data can be used as a fixture for 
tests rather than being auto-loaded - and we could perhaps even put it 
in the base testcase so it always auto-applied somehow - but that 
doesn't get over the fact that you can't rely on data migrations to 
set up your database for tests (for example, to add in required base 
rows or choices to EAV designs).


I remember we reordered TransactionTestCase with respect to TestCase - 
what did we do? If we had them all at the end then we could at least 
have things work for TestCase if we remove that initial flush, but 
unfortunately I suspect option 3 might have to be the solution, and I 
really don't want to do anything with fixtures (though I guess the 
only performance hit is writing the fixture, as we would have been 
reading one in every time before anyway - initial_data). Perhaps we 
could cache the fixture in memory and then spool out to disk if it 
gets too big.


The test case ordering is currently TestCase, SimpleTestCase and then 
the rest (including TransactionTestCases).


Using fixtures for dumping contents of the DB before first flush, and 
then reloading it back after each flush seems to be doable. I am not 
sure how well it will perform, or if it will be actually backwards 
compatible for users in all cases. It is extremely likely there will be 
at least some corner cases that can't be handled by simple 
dump-and-reload, so there is likely more work than just adding the dump 
and reload calls in to Django.


As for spooling out to disk - I don't think we need to do that. If the 
file is small enough to stay in memory, then reading from the file 
should be fast, too. If it is large, then the operating system knows 
better than us when to write it out to disk. In addition, if a fixture 
file is too large to stay in memory, then dump-and-restore will be too 
slow to be usable in any case.


If we want to support large amounts of data, then an approach I worked 
on to speed up Django's test suite some time ago might be useful. The 
idea was that only tables which were changed by a test were flushed 
after that test. It worked pretty well, but it was also somewhat 
complex. However, if we add an ability to have static data for tests 
this might need a revisit. The approach should be extremely effective 
for static test data. If you have any large amount of static test data 
then you don't want to reload it after each transactional test.


I have had a need for static data for tests in some projects, so a big 
+1 for this feature. I don't see how we can make this feature reliable 
for 1.7 in the time available. Can we punt this to 1.8?


 - Anssi

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/535375E1.8060602%40thl.fi.
For more options, visit https://groups.google.com/d/optout.

Re: Data Migrations and Testing

[Andrew Godwin]

I agree we can just say that initial_data can be used as a fixture for
tests rather than being auto-loaded - and we could perhaps even put it in
the base testcase so it always auto-applied somehow - but that doesn't get
over the fact that you can't rely on data migrations to set up your
database for tests (for example, to add in required base rows or choices to
EAV designs).

I remember we reordered TransactionTestCase with respect to TestCase - what
did we do? If we had them all at the end then we could at least have things
work for TestCase if we remove that initial flush, but unfortunately I
suspect option 3 might have to be the solution, and I really don't want to
do anything with fixtures (though I guess the only performance hit is
writing the fixture, as we would have been reading one in every time before
anyway - initial_data). Perhaps we could cache the fixture in memory and
then spool out to disk if it gets too big.

Andrew


On Sat, Apr 19, 2014 at 5:20 AM, Andrey Antukh  wrote:

> 2014-04-19 14:15 GMT+02:00 Anssi Kääriäinen :
>
> On 04/19/2014 10:52 AM, Anssi Kääriäinen wrote:
>>
>>  Getting dump-and-reload support for 1.7 seems hard, so from the above
>>> options this leaves just 2) - not deprecating initial_data yet. Or we
>>> need to find some other solution not listed above.
>>>
>> Hmmh, there is actually a pretty straightforward upgrade path for this
>> case. Just rename your initial_data fixtures to something else, and add
>> them as normal testing fixtures in your TestCases. Of course, this requires
>> some work to do, but it is a viable upgrade path.
>>
>> It is clear that migrations can't be used as a replacement for
>> initial_data when testing, so we need some improvements to the release
>> notes in any case.
>>
>
> Completely agree with you.
>
> +1
>
>
>>
>>
>
>>  - Anssi
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-developers+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-developers@googlegroups.com.
>> Visit this group at http://groups.google.com/group/django-developers.
>> To view this discussion on the web visit https://groups.google.com/d/
>> msgid/django-developers/535268EF.3010903%40thl.fi.
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> --
> Andrey Antukh - Андрей Антух -  / <
> n...@niwi.be>
> http://www.niwi.be 
> https://github.com/niwibe
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAKn%3DmOOE34RsCMprPCLfCT0o8KUArNoXGKXrDRapbNNSg4u%2BkA%40mail.gmail.com
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFwN1uobwrJ_rcguNfD3wxwZ0eF9gd5cKvCmbUndV5Vpw3gang%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Data Migrations and Testing

[Andrey Antukh]

2014-04-19 14:15 GMT+02:00 Anssi Kääriäinen :

> On 04/19/2014 10:52 AM, Anssi Kääriäinen wrote:
>
>  Getting dump-and-reload support for 1.7 seems hard, so from the above
>> options this leaves just 2) - not deprecating initial_data yet. Or we
>> need to find some other solution not listed above.
>>
> Hmmh, there is actually a pretty straightforward upgrade path for this
> case. Just rename your initial_data fixtures to something else, and add
> them as normal testing fixtures in your TestCases. Of course, this requires
> some work to do, but it is a viable upgrade path.
>
> It is clear that migrations can't be used as a replacement for
> initial_data when testing, so we need some improvements to the release
> notes in any case.
>

Completely agree with you.

+1


>
>

>  - Anssi
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/django-developers/535268EF.3010903%40thl.fi.
>
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Andrey Antukh - Андрей Антух -  / 
http://www.niwi.be 
https://github.com/niwibe

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAKn%3DmOOE34RsCMprPCLfCT0o8KUArNoXGKXrDRapbNNSg4u%2BkA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Data Migrations and Testing

[Andrey Antukh]

Hi Marc.

Thanks for you explanation.


2014-04-19 13:46 GMT+02:00 Marc Tamlyn :

> Hi Andrey,
>
> There are limited use cases where initial data is a good idea. Examples
> might be where you have a list of countries for relating to or some other
> fixed set of data.
>
> The idea is that now instead if having a hard to maintain fixture for that
> table, you populate it with a data migration instead. This means if I for
> example add a new column to that table, you get a migration to add the
> column then one to populate it (then one to make it not null) as opposed to
> just the one migration which you then run, manually back around the
> database to get it populated, update the fixture dump, deploy and then
> reload that (potentially overwriting old data).
>
I understand that use of intial_data is not he best approach and +1 for
deprecate/remove it, but put migrations as replacement (or suggest it on
docs) is really confusing.
My purpose for this, is removing the suggestion on release notes to use
migrations as replacement of initial_data, because migrations just as word
says is for data migrations not inital data.

Personally I think the new version is much better. That said, there are
> limited use cases for this feature.
>
The current version is much better because removes features with bad usage
practices. But as I previously said, do not suggest an other bad approach
as replacement.

> Marc
> On 19 Apr 2014 12:21, "Andrey Antukh"  wrote:
>
>>  Hi Christian.
>>
>> But, as far as I know, data migrations are "some logic" for translate
>> data from old scheme to new scheme and initial_data are files for load
>> intial data. I do not see where they could serve for the same purpose. Is
>> really confusing.
>>
>> Personally, putting initial data of my application mixed with migration
>> logic is bad approach for me. But is my opinion. :D
>>
>> Greetings.
>> Andrey
>>
>>
>>
>>
>>
>> 2014-04-19 13:13 GMT+02:00 Christian Schmitt :
>>
>>> hm, at first i didn't even read the release notes..
>>> But i think we should definitly make a blocker issue in trac.
>>> Currently re-introduce initial_data is the worst thing we could do,
>>> since django 1.9 requires migrations and do deprecate that behavior:
>>> > Deprecated since version 1.7: If an application uses migrations,
>>> there is no automatic loading of fixtures. Since migrations will be
>>> required for applications in Django 1.9, this behavior is
>>> > considered deprecated. If you want to load initial data for an app,
>>> consider doing it in a migration.
>>>
>>> Currently I think that the third solution could be the best, in my case
>>> we have like a dataset of 50 or more entries as initial data, which could
>>> easily be mocked and/or dumped and reloaded. It won't take that much time.
>>> And as you already said for bigger applications fixtures didn't work, too.
>>> But I don't think it will make it in Django 1.7 and since data
>>> migrations are unusable if you rely on testing and initial_data this
>>> wouldn't be too good.
>>>
>>> > At this time I haven't touched the new migrations system for django.
>>> But now, reading the releases notes and this thread...
>>> > I don't understand how data migrations can replace initial_data, are
>>> two things completely different and they have completely different scope.
>>> I'm slightly confusing.
>>>
>>> Data Migrations are the same thing than initial_data i mean they can be
>>> used to load a data set and drop that dataset, thats way more flexible. But
>>> currently Django's TestSuite isn't prepared for Data Migrations. Thats
>>> infact a problem.
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Django developers" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to django-developers+unsubscr...@googlegroups.com.
>>> To post to this group, send email to django-developers@googlegroups.com.
>>> Visit this group at http://groups.google.com/group/django-developers.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/django-developers/6c14d75b-6720-4c97-b697-93adb759f134%40googlegroups.com
>>> .
>>>
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>
>>
>> --
>> Andrey Antukh - Андрей Антух -  / <
>> n...@niwi.be>
>> http://www.niwi.be 
>> https://github.com/niwibe
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-developers+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-developers@googlegroups.com.
>> Visit this group at 

Re: Data Migrations and Testing

[Anssi Kääriäinen]

On 04/19/2014 10:52 AM, Anssi Kääriäinen wrote:


Getting dump-and-reload support for 1.7 seems hard, so from the above
options this leaves just 2) - not deprecating initial_data yet. Or we
need to find some other solution not listed above.
Hmmh, there is actually a pretty straightforward upgrade path for this 
case. Just rename your initial_data fixtures to something else, and add 
them as normal testing fixtures in your TestCases. Of course, this 
requires some work to do, but it is a viable upgrade path.


It is clear that migrations can't be used as a replacement for 
initial_data when testing, so we need some improvements to the release 
notes in any case.


 - Anssi

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/535268EF.3010903%40thl.fi.
For more options, visit https://groups.google.com/d/optout.

Re: Data Migrations and Testing

[Marc Tamlyn]

Hi Andrey,

There are limited use cases where initial data is a good idea. Examples
might be where you have a list of countries for relating to or some other
fixed set of data.

The idea is that now instead if having a hard to maintain fixture for that
table, you populate it with a data migration instead. This means if I for
example add a new column to that table, you get a migration to add the
column then one to populate it (then one to make it not null) as opposed to
just the one migration which you then run, manually back around the
database to get it populated, update the fixture dump, deploy and then
reload that (potentially overwriting old data).

Personally I think the new version is much better. That said, there are
limited use cases for this feature.

Marc
On 19 Apr 2014 12:21, "Andrey Antukh"  wrote:

> Hi Christian.
>
> But, as far as I know, data migrations are "some logic" for translate data
> from old scheme to new scheme and initial_data are files for load intial
> data. I do not see where they could serve for the same purpose. Is really
> confusing.
>
> Personally, putting initial data of my application mixed with migration
> logic is bad approach for me. But is my opinion. :D
>
> Greetings.
> Andrey
>
>
>
>
>
> 2014-04-19 13:13 GMT+02:00 Christian Schmitt :
>
>> hm, at first i didn't even read the release notes..
>> But i think we should definitly make a blocker issue in trac.
>> Currently re-introduce initial_data is the worst thing we could do, since
>> django 1.9 requires migrations and do deprecate that behavior:
>> > Deprecated since version 1.7: If an application uses migrations, there
>> is no automatic loading of fixtures. Since migrations will be required for
>> applications in Django 1.9, this behavior is
>> > considered deprecated. If you want to load initial data for an app,
>> consider doing it in a migration.
>>
>> Currently I think that the third solution could be the best, in my case
>> we have like a dataset of 50 or more entries as initial data, which could
>> easily be mocked and/or dumped and reloaded. It won't take that much time.
>> And as you already said for bigger applications fixtures didn't work, too.
>> But I don't think it will make it in Django 1.7 and since data migrations
>> are unusable if you rely on testing and initial_data this wouldn't be too
>> good.
>>
>> > At this time I haven't touched the new migrations system for django.
>> But now, reading the releases notes and this thread...
>> > I don't understand how data migrations can replace initial_data, are
>> two things completely different and they have completely different scope.
>> I'm slightly confusing.
>>
>> Data Migrations are the same thing than initial_data i mean they can be
>> used to load a data set and drop that dataset, thats way more flexible. But
>> currently Django's TestSuite isn't prepared for Data Migrations. Thats
>> infact a problem.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-developers+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-developers@googlegroups.com.
>> Visit this group at http://groups.google.com/group/django-developers.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-developers/6c14d75b-6720-4c97-b697-93adb759f134%40googlegroups.com
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> --
> Andrey Antukh - Андрей Антух -  / <
> n...@niwi.be>
> http://www.niwi.be 
> https://github.com/niwibe
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAKn%3DmOP9_PoEZTmmm7PF7c-2_ikmkSXHk151m_iEYhB%2Broivwg%40mail.gmail.com
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.

Re: Data Migrations and Testing

[Andrey Antukh]

Hi Christian.

But, as far as I know, data migrations are "some logic" for translate data
from old scheme to new scheme and initial_data are files for load intial
data. I do not see where they could serve for the same purpose. Is really
confusing.

Personally, putting initial data of my application mixed with migration
logic is bad approach for me. But is my opinion. :D

Greetings.
Andrey





2014-04-19 13:13 GMT+02:00 Christian Schmitt :

> hm, at first i didn't even read the release notes..
> But i think we should definitly make a blocker issue in trac.
> Currently re-introduce initial_data is the worst thing we could do, since
> django 1.9 requires migrations and do deprecate that behavior:
> > Deprecated since version 1.7: If an application uses migrations, there
> is no automatic loading of fixtures. Since migrations will be required for
> applications in Django 1.9, this behavior is
> > considered deprecated. If you want to load initial data for an app,
> consider doing it in a migration.
>
> Currently I think that the third solution could be the best, in my case we
> have like a dataset of 50 or more entries as initial data, which could
> easily be mocked and/or dumped and reloaded. It won't take that much time.
> And as you already said for bigger applications fixtures didn't work, too.
> But I don't think it will make it in Django 1.7 and since data migrations
> are unusable if you rely on testing and initial_data this wouldn't be too
> good.
>
> > At this time I haven't touched the new migrations system for django. But
> now, reading the releases notes and this thread...
> > I don't understand how data migrations can replace initial_data, are two
> things completely different and they have completely different scope. I'm
> slightly confusing.
>
> Data Migrations are the same thing than initial_data i mean they can be
> used to load a data set and drop that dataset, thats way more flexible. But
> currently Django's TestSuite isn't prepared for Data Migrations. Thats
> infact a problem.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/6c14d75b-6720-4c97-b697-93adb759f134%40googlegroups.com
> .
>
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Andrey Antukh - Андрей Антух -  / 
http://www.niwi.be 
https://github.com/niwibe

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAKn%3DmOP9_PoEZTmmm7PF7c-2_ikmkSXHk151m_iEYhB%2Broivwg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Data Migrations and Testing

[Christian Schmitt]

hm, at first i didn't even read the release notes..
But i think we should definitly make a blocker issue in trac.
Currently re-introduce initial_data is the worst thing we could do, since 
django 1.9 requires migrations and do deprecate that behavior:
> Deprecated since version 1.7: If an application uses migrations, there is 
no automatic loading of fixtures. Since migrations will be required for 
applications in Django 1.9, this behavior is 
> considered deprecated. If you want to load initial data for an app, 
consider doing it in a migration.

Currently I think that the third solution could be the best, in my case we 
have like a dataset of 50 or more entries as initial data, which could 
easily be mocked and/or dumped and reloaded. It won't take that much time. 
And as you already said for bigger applications fixtures didn't work, too.
But I don't think it will make it in Django 1.7 and since data migrations 
are unusable if you rely on testing and initial_data this wouldn't be too 
good.

> At this time I haven't touched the new migrations system for django. But 
now, reading the releases notes and this thread... 
> I don't understand how data migrations can replace initial_data, are two 
things completely different and they have completely different scope. I'm 
slightly confusing.

Data Migrations are the same thing than initial_data i mean they can be 
used to load a data set and drop that dataset, thats way more flexible. But 
currently Django's TestSuite isn't prepared for Data Migrations. Thats 
infact a problem. 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/6c14d75b-6720-4c97-b697-93adb759f134%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Data Migrations and Testing

[Andrey Antukh]

Hi!

At this time I haven't touched the new migrations system for django. But
now, reading the releases notes and this thread...
I don't understand how data migrations can replace initial_data, are two
things completely different and they have completely different scope. I'm
slightly confusing.

In any case, great work.
Thanks
Andrey






2014-04-19 9:52 GMT+02:00 Anssi Kääriäinen :

> On 04/18/2014 07:41 PM, Andrew Godwin wrote:
>
>> Ah yes, flushing, I forgot we did that for lesser DBs.
>>
>> I can think of several solutions:
>>  - Run the entire migration set every time you flush the database. This
>> is, obviously, not practicable.
>>  - Re-introduce initial_data fixtures; I'd rather not, as these require
>> constant upkeep and interact badly with migrations.
>>  - Dump the content of all apps at the start of tests into a file and
>> restore from that after flushes. This is a bit hacky and might use a lot of
>> disk space, but would get the intended effect.
>>  - Say that you can't use datamigrations if you have a database without
>> transactions (not ideal)
>>
>> Nothing here is really a great solution, alas. The fourth one is the
>> easiest solution - tell people to use factories in tests or fixtures if
>> they want that stuff - but slightly undermines the idea of loading initial
>> data in a migration (which is the better way of doing it).
>>
>> Perhaps we should look into deprecating flushing? It's not a behaviour
>> that can be easily replicated any more, as it relied on the old system of
>> one-shot schema and load a single file, but it's also kind of crucial to
>> non-transactional tests...
>>
>
> Unfortunately deprecating flushing isn't possible. For each test in
> TransactionTestCase Django flushes the database. So, this problem hits
> everybody using initial_data and TransactionTestCase independent of
> transaction support of the database used.
>
> As for why things break for normal TestCase, too - after creating the
> database and running all the migrations Django flushes the database - so
> all migration data is flushed, too.
>
> Of the above choices that leaves only the first three, and of those the
> first is already ruled out.
>
> Option 3) might actually work pretty well. Dump all data for managed
> tables just after migrations have ran, then load it back after each flush.
> If the data amounts are too large to make this practical, then they surely
> were impractical for initial_data or fixtures, too. In addition we could
> use faster forms of data loading (COPY for PostgreSQL, similar
> optimizations exists for other databases, too).
>
> Even if one isn't using any initial data in their project, Django does it
> with contenttypes and permissions. We already optimized this for Django's
> test suite, and this resulted in more than halving the runtime of the suite
> (granted, Django's test suite has more models than normal projects).
>
> Getting dump-and-reload support for 1.7 seems hard, so from the above
> options this leaves just 2) - not deprecating initial_data yet. Or we need
> to find some other solution not listed above.
>
>  - Anssi
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/django-developers/53522B3C.3050901%40thl.fi.
>
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Andrey Antukh - Андрей Антух -  / 
http://www.niwi.be 
https://github.com/niwibe

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAKn%3DmOMtrLNBZ%2BNfjz6GCpg-8sKQQ7iEKBhP40fOsCR_KtWfyQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Data Migrations and Testing

[Anssi Kääriäinen]

On 04/18/2014 07:41 PM, Andrew Godwin wrote:

Ah yes, flushing, I forgot we did that for lesser DBs.

I can think of several solutions:
 - Run the entire migration set every time you flush the database. 
This is, obviously, not practicable.
 - Re-introduce initial_data fixtures; I'd rather not, as these 
require constant upkeep and interact badly with migrations.
 - Dump the content of all apps at the start of tests into a file and 
restore from that after flushes. This is a bit hacky and might use a 
lot of disk space, but would get the intended effect.
 - Say that you can't use datamigrations if you have a database 
without transactions (not ideal)


Nothing here is really a great solution, alas. The fourth one is the 
easiest solution - tell people to use factories in tests or fixtures 
if they want that stuff - but slightly undermines the idea of loading 
initial data in a migration (which is the better way of doing it).


Perhaps we should look into deprecating flushing? It's not a behaviour 
that can be easily replicated any more, as it relied on the old system 
of one-shot schema and load a single file, but it's also kind of 
crucial to non-transactional tests...


Unfortunately deprecating flushing isn't possible. For each test in 
TransactionTestCase Django flushes the database. So, this problem hits 
everybody using initial_data and TransactionTestCase independent of 
transaction support of the database used.


As for why things break for normal TestCase, too - after creating the 
database and running all the migrations Django flushes the database - so 
all migration data is flushed, too.


Of the above choices that leaves only the first three, and of those the 
first is already ruled out.


Option 3) might actually work pretty well. Dump all data for managed 
tables just after migrations have ran, then load it back after each 
flush. If the data amounts are too large to make this practical, then 
they surely were impractical for initial_data or fixtures, too. In 
addition we could use faster forms of data loading (COPY for PostgreSQL, 
similar optimizations exists for other databases, too).


Even if one isn't using any initial data in their project, Django does 
it with contenttypes and permissions. We already optimized this for 
Django's test suite, and this resulted in more than halving the runtime 
of the suite (granted, Django's test suite has more models than normal 
projects).


Getting dump-and-reload support for 1.7 seems hard, so from the above 
options this leaves just 2) - not deprecating initial_data yet. Or we 
need to find some other solution not listed above.


 - Anssi

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/53522B3C.3050901%40thl.fi.
For more options, visit https://groups.google.com/d/optout.

Re: Data Migrations and Testing

[Christian Schmitt]

The first solution is definitly the worst, if you make integration tests 
you already have a "long running" test suite (especially with over 20-30 
models).
initial_data fixtures having a problem, if you change your database the 
initial_data will change over time, which is somehow bad to integrate.
Also dumping the content to a file could be really really slow. As said the 
initial db creation of our tests already takes like 5 seconds.

btw. currently a database (that supports transactions) that has initial 
data at the migration level won't contain your initial data even with 
TestCase. So maybe this is a bug or PostgreSQL won't have transactions??!? 
I even tried the AUTOCOMMIT False / True, w/e still no effect. My Initial 
Data is lost. 
So to make your fourth solution work, we would need to fix it. That initial 
Data stays in the database if running TestCase instead of 
TransactionTestCase.

And yes, the fourth one is somehow dump, but it would definitely tell me 
the things that i heard in many talks. "The Database is hot lava". I would 
change my test suite completely to only have function tests and "real" 
unittests without any database dependency..

The last thing would be great, but i don't think its something that could 
make it into 1.7, so we would rely on your forth solution. Tell people that 
data migrations shouldn't be used if you have a database without 
transactions if you want to keep your test suite up and running (and fix 
the problem that datamigrations on databases with transactions won't get 
lost in _pre_setup())


Am Freitag, 18. April 2014 18:41:06 UTC+2 schrieb Andrew Godwin:
>
> Ah yes, flushing, I forgot we did that for lesser DBs.
>
> I can think of several solutions:
>  - Run the entire migration set every time you flush the database. This 
> is, obviously, not practicable.
>  - Re-introduce initial_data fixtures; I'd rather not, as these require 
> constant upkeep and interact badly with migrations.
>  - Dump the content of all apps at the start of tests into a file and 
> restore from that after flushes. This is a bit hacky and might use a lot of 
> disk space, but would get the intended effect.
>  - Say that you can't use datamigrations if you have a database without 
> transactions (not ideal)
>
> Nothing here is really a great solution, alas. The fourth one is the 
> easiest solution - tell people to use factories in tests or fixtures if 
> they want that stuff - but slightly undermines the idea of loading initial 
> data in a migration (which is the better way of doing it).
>
> Perhaps we should look into deprecating flushing? It's not a behaviour 
> that can be easily replicated any more, as it relied on the old system of 
> one-shot schema and load a single file, but it's also kind of crucial to 
> non-transactional tests...
>
> Andrew
>
>
> On Fri, Apr 18, 2014 at 7:56 AM, Christian Schmitt 
>  > wrote:
>
>> The case is really simple and i think its definitely that would block the 
>> release.
>>
>> First off, I have a model:
>>
>> from django.db import models
>>
>> # Create your models here.
>> class SimpleModel(models.Model):
>> name = models.CharField(max_length=10)
>>
>> Then I will make some migrations:
>> # encoding: utf8
>> from django.db import models, migrations
>>
>>
>> class Migration(migrations.Migration):
>>
>> dependencies = [
>> ]
>>
>> operations = [
>> migrations.CreateModel(
>> name='SimpleModel',
>> fields=[
>> ('id', models.AutoField(serialize=False, 
>> primary_key=True, verbose_name='ID', auto_created=True)),
>> ('name', models.CharField(max_length=10)),
>> ],
>> options={
>> },
>> bases=(models.Model,),
>> ),
>> ]
>> Now the second one that loads data:
>> # encoding: utf8
>>  from django.db import models, migrations
>>
>>
>> def add_data(apps, schema_editor):
>> SimpleModel = apps.get_model("new_app", "SimpleModel")
>>  SimpleModel.objects.create(name="simple")
>>
>> class Migration(migrations.Migration):
>>
>> dependencies = [
>> ('new_app', '0001_initial'),
>> ]
>>
>> operations = [
>> migrations.RunPython(add_data),
>> ]
>>
>> Now I have a view that makes a get query:
>> from django.http import HttpResponse
>> from .models import SimpleModel
>>
>> # Create your views here.
>> def simple(request):
>> simple = SimpleModel.objects.get(name="simple")
>> return HttpResponse(simple)
>>
>> The urls will have the / url pointing to our view.
>>
>> And finally the test:
>> from django.test import TransactionTestCase
>> from .models import SimpleModel
>>
>> # Create your tests here.
>> class SimpleTestView(TransactionTestCase):
>>
>> def test_data_migration(self):
>> response = self.client.get('/')
>> self.assertEqual(response, not False)
>>
>> And it will fail.
>> But it will not fail cause of maybe a bad test, even if my test 

Re: Data Migrations and Testing

[Andrew Godwin]

Ah yes, flushing, I forgot we did that for lesser DBs.

I can think of several solutions:
 - Run the entire migration set every time you flush the database. This is,
obviously, not practicable.
 - Re-introduce initial_data fixtures; I'd rather not, as these require
constant upkeep and interact badly with migrations.
 - Dump the content of all apps at the start of tests into a file and
restore from that after flushes. This is a bit hacky and might use a lot of
disk space, but would get the intended effect.
 - Say that you can't use datamigrations if you have a database without
transactions (not ideal)

Nothing here is really a great solution, alas. The fourth one is the
easiest solution - tell people to use factories in tests or fixtures if
they want that stuff - but slightly undermines the idea of loading initial
data in a migration (which is the better way of doing it).

Perhaps we should look into deprecating flushing? It's not a behaviour that
can be easily replicated any more, as it relied on the old system of
one-shot schema and load a single file, but it's also kind of crucial to
non-transactional tests...

Andrew


On Fri, Apr 18, 2014 at 7:56 AM, Christian Schmitt  wrote:

> The case is really simple and i think its definitely that would block the
> release.
>
> First off, I have a model:
>
> from django.db import models
>
> # Create your models here.
> class SimpleModel(models.Model):
> name = models.CharField(max_length=10)
>
> Then I will make some migrations:
> # encoding: utf8
> from django.db import models, migrations
>
>
> class Migration(migrations.Migration):
>
> dependencies = [
> ]
>
> operations = [
> migrations.CreateModel(
> name='SimpleModel',
> fields=[
> ('id', models.AutoField(serialize=False, primary_key=True,
> verbose_name='ID', auto_created=True)),
> ('name', models.CharField(max_length=10)),
> ],
> options={
> },
> bases=(models.Model,),
> ),
> ]
> Now the second one that loads data:
> # encoding: utf8
> from django.db import models, migrations
>
>
> def add_data(apps, schema_editor):
> SimpleModel = apps.get_model("new_app", "SimpleModel")
>  SimpleModel.objects.create(name="simple")
>
> class Migration(migrations.Migration):
>
> dependencies = [
> ('new_app', '0001_initial'),
> ]
>
> operations = [
> migrations.RunPython(add_data),
> ]
>
> Now I have a view that makes a get query:
> from django.http import HttpResponse
> from .models import SimpleModel
>
> # Create your views here.
> def simple(request):
> simple = SimpleModel.objects.get(name="simple")
> return HttpResponse(simple)
>
> The urls will have the / url pointing to our view.
>
> And finally the test:
> from django.test import TransactionTestCase
> from .models import SimpleModel
>
> # Create your tests here.
> class SimpleTestView(TransactionTestCase):
>
> def test_data_migration(self):
> response = self.client.get('/')
> self.assertEqual(response, not False)
>
> And it will fail.
> But it will not fail cause of maybe a bad test, even if my test wouldn’t
> be correct it would still fail.
> It will fail cause of this:
> ==
> ERROR: test_data_migration (new_app.tests.SimpleTestView)
> --
> Traceback (most recent call last):
>   File "/Users/schmitch/PycharmProjects/testing/new_app/tests.py", line 8,
> in test_data_migration
> response = self.client.get('/')
>   File
> "/Users/schmitch/PycharmProjects/envisia_erp_server/.venv/lib/python3.4/site-packages/django/test/client.py",
> line 467, in get
> **extra)
>   File
> "/Users/schmitch/PycharmProjects/envisia_erp_server/.venv/lib/python3.4/site-packages/django/test/client.py",
> line 285, in get
> return self.generic('GET', path, secure=secure, **r)
>   File
> "/Users/schmitch/PycharmProjects/envisia_erp_server/.venv/lib/python3.4/site-packages/django/test/client.py",
> line 355, in generic
> return self.request(**r)
>   File
> "/Users/schmitch/PycharmProjects/envisia_erp_server/.venv/lib/python3.4/site-packages/django/test/client.py",
> line 437, in request
> six.reraise(*exc_info)
>   File
> "/Users/schmitch/PycharmProjects/envisia_erp_server/.venv/lib/python3.4/site-packages/django/utils/six.py",
> line 536, in reraise
> raise value
>   File
> "/Users/schmitch/PycharmProjects/envisia_erp_server/.venv/lib/python3.4/site-packages/django/core/handlers/base.py",
> line 113, in get_response
> response = wrapped_callback(request, *callback_args, **callback_kwargs)
>   File "/Users/schmitch/PycharmProjects/testing/new_app/views.py", line 6,
> in simple
> simple = SimpleModel.objects.get(name="simple")
>   File
> 

Re: Data Migrations and Testing

[Christian Schmitt]

The case is really simple and i think its definitely that would block the 
release.

First off, I have a model:

from django.db import models

# Create your models here.
class SimpleModel(models.Model):
name = models.CharField(max_length=10)

Then I will make some migrations:
# encoding: utf8
from django.db import models, migrations


class Migration(migrations.Migration):

dependencies = [
]

operations = [
migrations.CreateModel(
name='SimpleModel',
fields=[
('id', models.AutoField(serialize=False, primary_key=True, 
verbose_name='ID', auto_created=True)),
('name', models.CharField(max_length=10)),
],
options={
},
bases=(models.Model,),
),
]
Now the second one that loads data:
# encoding: utf8
from django.db import models, migrations


def add_data(apps, schema_editor):
SimpleModel = apps.get_model("new_app", "SimpleModel")
SimpleModel.objects.create(name="simple")

class Migration(migrations.Migration):

dependencies = [
('new_app', '0001_initial'),
]

operations = [
migrations.RunPython(add_data),
]

Now I have a view that makes a get query:
from django.http import HttpResponse
from .models import SimpleModel

# Create your views here.
def simple(request):
simple = SimpleModel.objects.get(name="simple")
return HttpResponse(simple)

The urls will have the / url pointing to our view.

And finally the test:
from django.test import TransactionTestCase
from .models import SimpleModel

# Create your tests here.
class SimpleTestView(TransactionTestCase):

def test_data_migration(self):
response = self.client.get('/')
self.assertEqual(response, not False)

And it will fail.
But it will not fail cause of maybe a bad test, even if my test wouldn’t be 
correct it would still fail.
It will fail cause of this:
==
ERROR: test_data_migration (new_app.tests.SimpleTestView)
--
Traceback (most recent call last):
  File "/Users/schmitch/PycharmProjects/testing/new_app/tests.py", line 8, in 
test_data_migration
response = self.client.get('/')
  File 
"/Users/schmitch/PycharmProjects/envisia_erp_server/.venv/lib/python3.4/site-packages/django/test/client.py",
 line 467, in get
**extra)
  File 
"/Users/schmitch/PycharmProjects/envisia_erp_server/.venv/lib/python3.4/site-packages/django/test/client.py",
 line 285, in get
return self.generic('GET', path, secure=secure, **r)
  File 
"/Users/schmitch/PycharmProjects/envisia_erp_server/.venv/lib/python3.4/site-packages/django/test/client.py",
 line 355, in generic
return self.request(**r)
  File 
"/Users/schmitch/PycharmProjects/envisia_erp_server/.venv/lib/python3.4/site-packages/django/test/client.py",
 line 437, in request
six.reraise(*exc_info)
  File 
"/Users/schmitch/PycharmProjects/envisia_erp_server/.venv/lib/python3.4/site-packages/django/utils/six.py",
 line 536, in reraise
raise value
  File 
"/Users/schmitch/PycharmProjects/envisia_erp_server/.venv/lib/python3.4/site-packages/django/core/handlers/base.py",
 line 113, in get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/Users/schmitch/PycharmProjects/testing/new_app/views.py", line 6, in 
simple
simple = SimpleModel.objects.get(name="simple")
  File 
"/Users/schmitch/PycharmProjects/envisia_erp_server/.venv/lib/python3.4/site-packages/django/db/models/manager.py",
 line 92, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
  File 
"/Users/schmitch/PycharmProjects/envisia_erp_server/.venv/lib/python3.4/site-packages/django/db/models/query.py",
 line 352, in get
self.model._meta.object_name)
new_app.models.DoesNotExist: SimpleModel matching query does not exist.


I don’t even need to test the view. i could test another view. The Test client 
will always fail if there is a view somewhere that makes some get queries where 
the data isn’t there.

here the next test i made:
def test_another_thing(self):
self.user = User.objects.create_user("testuser", "u...@test.com", 
"sample_pass")
login = self.client.login(username="testuser", password="sample_pass")
self.assertEqual(True, True)

It shouldn’t fail. I mean i didn’t even query the view. i just login a user and 
get the same error.

I don’t think this is the way how tests should run. I also tried 
TransactionTestCase and TestCase on PostgreSQL, Sqlite3, and MySQL. Same 
results, every database.

Btw. if I have some more Logic applied around the get it wouldn’t be a problem, 
but we have some views where NOBODY could delete these get() queries, so we 
rely on them and they will fail, if we try to make tests.
Too bad our application is now too big to think about application design, i 
also 

Re: Data Migrations and Testing

[Anssi Kääriäinen]

The problem might be that initial data is loaded after each database 
flush, but the same isn't true for migrations. If we want to have RunSQL 
as a complete replacement for initial data, then we need to have some 
way to mark data loading "migrations" to be loaded after each database 
flush. Of course, then some migrations aren't actually migrations... 
Another approach is to recommend using fixtures instead of initial data.


 - Anssi

On 04/17/2014 11:42 PM, Andrew Godwin wrote:

Hi Christian,

Can you explain your situation more, and perhaps give us some example 
files or console output? When you run tests they should run through 
all the migrations first, so any initial data should get loaded in 
before tests begin.


Andrew


On Thu, Apr 17, 2014 at 1:16 PM, Christian Schmitt 
> wrote:


Hello,

currently I try to make a TestSuite for a already grown
application and it is horrible to do so.
But thats not why I'm writing here.

Our Application has some data migrations, previously we used
South, now we use the Django 1.7 Migrations with RunPython to do so.

In the Docs there is also a Article how to do it:
https://docs.djangoproject.com/en/dev/topics/migrations/#data-migrations
Currently If you load initial data with a RunPython command that
your App Depends on (there are some cases, not many, but some) you
will fail hard.
Your Unittests starting to fail. And that is not good, so i think
we should either change something on the Test Suites or we should
make it clear that Data Migrations for initial data is a bad idea
or maybe warn about the drawbacks with Test Cases.
I'm really new to testing and I really needed to figure out whats
going on first until I realized that this behavior comes from the
TestSuite.
-- 
You received this message because you are subscribed to the Google

Groups "Django developers" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to django-developers+unsubscr...@googlegroups.com
.
To post to this group, send email to
django-developers@googlegroups.com
.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit

https://groups.google.com/d/msgid/django-developers/768969da-8f75-4402-80a5-c30c27c74997%40googlegroups.com

.
For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google 
Groups "Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to django-developers+unsubscr...@googlegroups.com 
.
To post to this group, send email to 
django-developers@googlegroups.com 
.

Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFwN1uqasqm17EKHg1PrL5HbJaE3W2hVn%3DBe3wd45yxKRDR04w%40mail.gmail.com 
.

For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/5350D836.3040607%40thl.fi.
For more options, visit https://groups.google.com/d/optout.

Re: Data Migrations and Testing

[Andrew Godwin]

Hi Christian,

Can you explain your situation more, and perhaps give us some example files
or console output? When you run tests they should run through all the
migrations first, so any initial data should get loaded in before tests
begin.

Andrew


On Thu, Apr 17, 2014 at 1:16 PM, Christian Schmitt  wrote:

> Hello,
>
> currently I try to make a TestSuite for a already grown application and it
> is horrible to do so.
> But thats not why I'm writing here.
>
> Our Application has some data migrations, previously we used South, now we
> use the Django 1.7 Migrations with RunPython to do so.
>
> In the Docs there is also a Article how to do it:
> https://docs.djangoproject.com/en/dev/topics/migrations/#data-migrations
> Currently If you load initial data with a RunPython command that your App
> Depends on (there are some cases, not many, but some) you will fail hard.
> Your Unittests starting to fail. And that is not good, so i think we
> should either change something on the Test Suites or we should make it
> clear that Data Migrations for initial data is a bad idea or maybe warn
> about the drawbacks with Test Cases.
> I'm really new to testing and I really needed to figure out whats going on
> first until I realized that this behavior comes from the TestSuite.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/768969da-8f75-4402-80a5-c30c27c74997%40googlegroups.com
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFwN1uqasqm17EKHg1PrL5HbJaE3W2hVn%3DBe3wd45yxKRDR04w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.