Hi Simon,

Intriguing proposal! Thanks for bringing it up. This is certainly something
I've struggled with on older projects we've continued to maintain.

My first impression (and it may be only that) is that this seems a big
magical and potentially confusing, especially in the event the copy
operation fails. I would want some way to disable the behavior, or even
have "disabled" as the default, at least for a couple releases (e.g., by
keeping it as a decorator the way you have it now). Alternatively, is there
a reason this couldn't continue to live as an external package?

I'm also not sure I understand the argument about how the current
implementation doesn't help reduce round trips to the database. Is this
just a case of the docs offering a less-than-optimal recommendation? When
tests do need to modify a record in the database that was created in
setUpTestData(), it's easy enough to replace self.obj.foo = bar;
self.obj.save() with Model.objects.filter(pk=self.obj.pk).update(foo=bar),
which will avoid modifying the Python object and be *at least* as efficient
as the original code, without the need for any additional overhead. I'll
admit tracking down the offending code can be a pain, but it can be done.

Could we, for example, include something along these lines as the preferred
approach in the docs
<https://docs.djangoproject.com/en/2.1/topics/testing/tools/#django.test.TestCase.setUpTestData>,
and/or even find a way to raise an error or warning when a test does modify
an object that it shouldn't?

Tobias

On Wed, Nov 28, 2018 at 11:42 PM charettes <charett...@gmail.com> wrote:

> Hey Josh, glad the package can help in the mean time!
>
> > Is there anyway to determine the pickle-ability of something without
> just trying to pickle it? I wouldn't be keen on that overhead.
>
> Not that I'm aware off but unfortunately. There really shouldn't be much
> overhead though because the deepcopy is only performed on instance
> attribute access. That means that tests that are only creating test data
> without accessing attributes assigned during setUpTestData() are not going
> to be affected by this change at all. In other words, I suggest only doing
> the deep copy if required for the requested attributes so if you define a
> complex data set in setUpTestData() then only the few attributes and their
> related objects would get deepcopied on instance attribute accesses.
>
> > Could you just capture any copy exceptions, raise a deprecation warning,
> and abandon the copy for that attribute?
>
> Yeah that's the exact plan for the deprecation phase; warn and return the
> actual attribute. From Django 3.1 this would result in an AttributeError.
>
> I've updated the branch with the deprecation warnings[0] approach.
>
> I'll give it one or two more weeks to gather a bit more feedback but it
> looks like this could be a viable option so far.
>
> Cheers,
> Simon
>
> [0]
> https://github.com/django/django/compare/master...charettes:testdata#diff-5d7d8ead1a907fe91ffc121f830f2a49R1032
>
> Le mercredi 28 novembre 2018 21:40:53 UTC-5, Josh Smeaton a écrit :
>>
>> Our project also suffers extensively with mutating objects assigned from
>> setUp, preventing us from moving most of our tests to setUpTestData. I'll
>> likely begin using your pypi package right away, thanks Simon!
>>
>> Backward compat issues are probably likely - but they'd be in test cases
>> exclusively, making them extremely easy to find during an upgrade. That
>> said, a deprecation warning is probably the most sensible path forward to
>> prevent the need for immediate action.
>>
>> Is there anyway to determine the pickle-ability of something without just
>> trying to pickle it? I wouldn't be keen on that overhead. Could you just
>> capture any copy exceptions, raise a deprecation warning, and abandon the
>> copy for that attribute?
>>
>> On Saturday, 24 November 2018 14:29:33 UTC+11, charettes wrote:
>>>
>>> Dear developers,
>>>
>>> Django 1.8 introduced the `TestCase.setUpTestData()` class method as a
>>> mean to
>>> speed up test fixtures initialization as compared to using `setUp()`[0].
>>>
>>> As I've come to use this feature and review changes from peers using it
>>> in
>>> different projects the fact that test data assigned during its execution
>>> couldn't be safely altered by test methods without compromising test
>>> isolation
>>> has often be the source of confusion and frustration.
>>>
>>> While the `setUpTestData` documentation mentions this limitation[1] and
>>> ways to
>>> work around it by using `refresh_from_db()` in `setUp()` I believe it
>>> defeats
>>> the whole purpose of the feature; avoiding unnecessary roundtrips to the
>>> database to speed up execution. Given `TestCase` goes through great
>>> lengths to
>>> ensure database level data isolation I believe it should do the same
>>> with class
>>> level in-memory data assigned during `setUpTestData`.
>>>
>>> In order to get rid of this caveat of the feature I'd like to propose an
>>> adjustment to ensure such in-memory test data isolation.
>>>
>>> What I suggest doing is wrapping all attributes assigned during
>>> `setUpTestData`
>>> in descriptors that lazily return `copy.deepcopy()`ed values on instance
>>> attribute accesses. By attaching the `deepcopy()`'s memo on test
>>> instances we
>>> can ensure that the reference graph between objects is preserved and thus
>>> backward compatible.
>>>
>>> In other words, the following test would pass even if `self.book` is a
>>> deep
>>> copy of `cls.book`.
>>>
>>> class BookTests(TestCase):
>>>     @classmethod
>>>     def setUpTestData(cls):
>>>         cls.author = Author.objects.create()
>>>         cls.book = cls.author.books.create()
>>>
>>>     def test_relationship_preserved(self):
>>>         self.assertIs(self.book.author, self.author)
>>>
>>> Lazily returning `deepcopy'ies and caching returned values in `__dict__`
>>> à la
>>> `cached_property` should also make sure the slight performance overhead
>>> this
>>> incurs is minimized.
>>>
>>> From a check against a few projects and Django's test suite[2] I have
>>> only
>>> identified a single issue which is that attributes assigned during
>>> `setUpTestData` would now have to be `deepcopy()`able but it shouldn't be
>>> a blocker given `Model` instance are.
>>>
>>> In order to allow other possible issues from being identified against
>>> existing
>>> projects I packaged the proposed feature[3] and made it available on
>>> pypi[4]. It
>>> requires decorating `setUpTestData` methods but it shouldn't be too hard
>>> to
>>> apply to your projects if you want to give it a try.
>>>
>>> Given this reaches consensus that this could be a great addition I'd file
>>> a ticket and finalize what I have so far[2].
>>>
>>> Thank your for your time,
>>> Simon
>>>
>>> [0]
>>> https://docs.djangoproject.com/en/1.8/releases/1.8/#testcase-data-setup
>>> [1]
>>> https://docs.djangoproject.com/en/2.1/topics/testing/tools/#django.test.TestCase.setUpTestData
>>> [2]
>>> https://github.com/charettes/django/compare/setuptestdata...charettes:testdata
>>> [3] https://github.com/charettes/django-testdata
>>> [4] https://pypi.org/project/django-testdata/
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/61c1f65d-d675-4733-aa85-d3495b261cfc%40googlegroups.com
> <https://groups.google.com/d/msgid/django-developers/61c1f65d-d675-4733-aa85-d3495b261cfc%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAMGFDKSoU7anDWcV5Wz5QmDemZfY8k4nq9o7Q9odUZ9i-xuBtw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Reply via email to