Hey Josh, glad the package can help in the mean time!

> Is there anyway to determine the pickle-ability of something without just 
trying to pickle it? I wouldn't be keen on that overhead.

Not that I'm aware off but unfortunately. There really shouldn't be much 
overhead though because the deepcopy is only performed on instance 
attribute access. That means that tests that are only creating test data 
without accessing attributes assigned during setUpTestData() are not going 
to be affected by this change at all. In other words, I suggest only doing 
the deep copy if required for the requested attributes so if you define a 
complex data set in setUpTestData() then only the few attributes and their 
related objects would get deepcopied on instance attribute accesses.

> Could you just capture any copy exceptions, raise a deprecation warning, 
and abandon the copy for that attribute?

Yeah that's the exact plan for the deprecation phase; warn and return the 
actual attribute. From Django 3.1 this would result in an AttributeError.

I've updated the branch with the deprecation warnings[0] approach.

I'll give it one or two more weeks to gather a bit more feedback but it 
looks like this could be a viable option so far.

Cheers,
Simon

[0] 
https://github.com/django/django/compare/master...charettes:testdata#diff-5d7d8ead1a907fe91ffc121f830f2a49R1032

Le mercredi 28 novembre 2018 21:40:53 UTC-5, Josh Smeaton a écrit :
>
> Our project also suffers extensively with mutating objects assigned from 
> setUp, preventing us from moving most of our tests to setUpTestData. I'll 
> likely begin using your pypi package right away, thanks Simon!
>
> Backward compat issues are probably likely - but they'd be in test cases 
> exclusively, making them extremely easy to find during an upgrade. That 
> said, a deprecation warning is probably the most sensible path forward to 
> prevent the need for immediate action.
>
> Is there anyway to determine the pickle-ability of something without just 
> trying to pickle it? I wouldn't be keen on that overhead. Could you just 
> capture any copy exceptions, raise a deprecation warning, and abandon the 
> copy for that attribute?
>
> On Saturday, 24 November 2018 14:29:33 UTC+11, charettes wrote:
>>
>> Dear developers,
>>
>> Django 1.8 introduced the `TestCase.setUpTestData()` class method as a 
>> mean to
>> speed up test fixtures initialization as compared to using `setUp()`[0].
>>
>> As I've come to use this feature and review changes from peers using it in
>> different projects the fact that test data assigned during its execution
>> couldn't be safely altered by test methods without compromising test 
>> isolation
>> has often be the source of confusion and frustration.
>>
>> While the `setUpTestData` documentation mentions this limitation[1] and 
>> ways to
>> work around it by using `refresh_from_db()` in `setUp()` I believe it 
>> defeats
>> the whole purpose of the feature; avoiding unnecessary roundtrips to the
>> database to speed up execution. Given `TestCase` goes through great 
>> lengths to
>> ensure database level data isolation I believe it should do the same with 
>> class
>> level in-memory data assigned during `setUpTestData`.
>>
>> In order to get rid of this caveat of the feature I'd like to propose an
>> adjustment to ensure such in-memory test data isolation.
>>
>> What I suggest doing is wrapping all attributes assigned during 
>> `setUpTestData`
>> in descriptors that lazily return `copy.deepcopy()`ed values on instance
>> attribute accesses. By attaching the `deepcopy()`'s memo on test 
>> instances we
>> can ensure that the reference graph between objects is preserved and thus
>> backward compatible.
>>
>> In other words, the following test would pass even if `self.book` is a 
>> deep
>> copy of `cls.book`.
>>
>> class BookTests(TestCase):
>>     @classmethod
>>     def setUpTestData(cls):
>>         cls.author = Author.objects.create()
>>         cls.book = cls.author.books.create()
>>
>>     def test_relationship_preserved(self):
>>         self.assertIs(self.book.author, self.author)
>>
>> Lazily returning `deepcopy'ies and caching returned values in `__dict__` 
>> à la
>> `cached_property` should also make sure the slight performance overhead 
>> this
>> incurs is minimized.
>>
>> From a check against a few projects and Django's test suite[2] I have only
>> identified a single issue which is that attributes assigned during
>> `setUpTestData` would now have to be `deepcopy()`able but it shouldn't be
>> a blocker given `Model` instance are.
>>
>> In order to allow other possible issues from being identified against 
>> existing
>> projects I packaged the proposed feature[3] and made it available on 
>> pypi[4]. It
>> requires decorating `setUpTestData` methods but it shouldn't be too hard 
>> to
>> apply to your projects if you want to give it a try.
>>
>> Given this reaches consensus that this could be a great addition I'd file
>> a ticket and finalize what I have so far[2].
>>
>> Thank your for your time,
>> Simon
>>
>> [0] 
>> https://docs.djangoproject.com/en/1.8/releases/1.8/#testcase-data-setup
>> [1] 
>> https://docs.djangoproject.com/en/2.1/topics/testing/tools/#django.test.TestCase.setUpTestData
>> [2] 
>> https://github.com/charettes/django/compare/setuptestdata...charettes:testdata
>> [3] https://github.com/charettes/django-testdata
>> [4] https://pypi.org/project/django-testdata/
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/61c1f65d-d675-4733-aa85-d3495b261cfc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Reply via email to