[issue20804] Sentinels identity lost when pickled (unittest.mock)

Changes by Donald Stufft : -- pull_requests: +979 ___ Python tracker ___ ___

[issue20804] Sentinels identity lost when pickled (unittest.mock)

Serhiy Storchaka added the comment: Thank you for your review Michael. -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker

[issue20804] Sentinels identity lost when pickled (unittest.mock)

Roundup Robot added the comment: New changeset 398b73dbb1a0 by Serhiy Storchaka in branch '3.5': Issue #20804: Document the limitation of the unittest.mock.sentinel attributes. https://hg.python.org/cpython/rev/398b73dbb1a0 -- ___ Python tracker

[issue20804] Sentinels identity lost when pickled (unittest.mock)

Roundup Robot added the comment: New changeset 98f061402fcf by Serhiy Storchaka in branch 'default': Issue #20804: The unittest.mock.sentinel attributes now preserve their https://hg.python.org/cpython/rev/98f061402fcf -- nosy: +python-dev ___ Python

[issue20804] Sentinels identity lost when pickled (unittest.mock)

Michael Foord added the comment: LGTM Serhiy - thank you for your work. Appreciated. -- ___ Python tracker ___

[issue20804] Sentinels identity lost when pickled (unittest.mock)

Serhiy Storchaka added the comment: Updated patch contains doc changes. -- type: behavior -> enhancement versions: -Python 3.5, Python 3.6 Added file: http://bugs.python.org/file46260/mock_sentinel_pickle2.patch ___ Python tracker

[issue20804] Sentinels identity lost when pickled (unittest.mock)

Michael Foord added the comment: It's a new feature. -- ___ Python tracker ___ ___ Python-bugs-list mailing

[issue20804] Sentinels identity lost when pickled (unittest.mock)

Serhiy Storchaka added the comment: sentinel-doc.patch documents the contrary behaviour -- not preserving an identity and equality. I haven't included doc changes because I don't know how to treat this change. As a bug fix, as a new feature, or as a new feature backported to maintained

[issue20804] Sentinels identity lost when pickled (unittest.mock)

Michael Foord added the comment: I think option 3 is the correct semantic behaviour for sentinels, and if there are already examples of this in the standard library then it *doesn't* violate expected behaviour of pickling and copying (sentinel and singleton objects can be permitted to retain

[issue20804] Sentinels identity lost when pickled (unittest.mock)

Serhiy Storchaka added the comment: I just don't know well the purpose of sentinels, and don't . I see fourth possible options: 1. Raise an exception when copy or pickle a sentinel. This is an option when we want to be sure that copying and pickling are not involved and we get the same

[issue20804] Sentinels identity lost when pickled (unittest.mock)

Michael Foord added the comment: So my thinking has evolved slightly on it. If it's *possible* for sentinels to be copied and pickled and preserve identity then I'm happy for it. I think the right semantics for copying a sentinel is that you get the original object back. If you pickle a

[issue20804] Sentinels identity lost when pickled (unittest.mock)

Vlastimil Zíma added the comment: Since the issue was dug up, I've created a patch for documentation. I still think, that it is worth stating the equality is not preserved for sentinels when copied as the message produced by tests is rather confusing: class SentinelTestCase(TestCase):

[issue20804] Sentinels identity lost when pickled (unittest.mock)

Serhiy Storchaka added the comment: Sorry, I had wrote this patch for issue29229 and didn't read the above discussion. Copying and pickling preserve identity of some objects: functions, classes, enums, builtin singletons (None, False, True, ...). It was not obvious to me that this shouldn't

[issue20804] Sentinels identity lost when pickled (unittest.mock)

Davin Potts added the comment: Serhiy: The above discussion seemed to converge on the perspective that object identity should not survive pickling and that the point of a sentinel is object identity. While your proposed patch may mechanically work, I believe it is in conflict with the

[issue20804] Sentinels identity lost when pickled (unittest.mock)

Serhiy Storchaka added the comment: Proposed patch makes mock sentinels supporting pickling and copying. This allows passing them in multiprocessing. I'm not sure whether this is a bug fix or a new feature. The patch works only with 3.5+. Older versions need more complex solution. --

[issue20804] Sentinels identity lost when pickled (unittest.mock)

Vlastimil Zíma added the comment: David, first part of your comment should be added to docs. Purpose of sentinels isn't much intuitive and in my opinion documentation doesn't quite describe it. An exception in case of pickle or copy would be much easier to understand apart from unexpected

[issue20804] Sentinels identity lost when pickled (unittest.mock)

Michael Foord added the comment: I'm not sure to what extent mock documentation should explain Python semantics. The docs *do* make clear that sentinel is useful for where you want to test (compare) objects by *identity*. Problems with copying and pickling them come from the fact that those

[issue20804] Sentinels identity lost when pickled (unittest.mock)

New submission from Vlastimil Zíma: Sentinels lose their identity when they are pickled, meaning they are no longer equal to sentinel with the same name. from mock import sentinel import pickle sentinel.foo == sentinel.foo True pickle.loads(pickle.dumps(sentinel.foo)) == sentinel.foo

[issue20804] Sentinels identity lost when pickled (unittest.mock)

R. David Murray added the comment: Since the point of a sentinel is *object identity*, there is no way that you can get the same sentinel back after a pickle. This is fundamental to the nature of pickle. -- nosy: +michael.foord, r.david.murray ___

[issue20804] Sentinels identity lost when pickled (unittest.mock)

Vlastimil Zíma added the comment: I don't question pickle, but I'd expect sentinels to keep their equality when they are pickled or copied. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20804

[issue20804] Sentinels identity lost when pickled (unittest.mock)

R. David Murray added the comment: Sentinels are not equal unless they are the *same object*, just as for any python object that does not have a defined __eq__. And as I said, this is part of the point of sentinels. -- ___ Python tracker

[issue20804] Sentinels identity lost when pickled (unittest.mock)

Michael Foord added the comment: Maybe making sentinels unpickleable would be a better solution. On the other hand I'd love to be able to [properly] customise object creation when unpickling. It would solve various other problems I've had from time to time. But that's well outside the scope

[issue20804] Sentinels identity lost when pickled (unittest.mock)

R. David Murray added the comment: In case anyone is wondering, I'm leaving this open in case Michael thinks it is a good idea to add some sort of documentation note about this. I'm not sure it is, because this is fundamental to the way Python works. On the other hand, I can't think of any

[issue20804] Sentinels identity lost when pickled (unittest.mock)

R. David Murray added the comment: Our posts crossed. Making them unpickleable sounds like it might be sensible. Since the sentinel *can't* survive pickling/unpickling and still perform its documented function, it seems better to raise an error if someone tries. --

[issue20804] Sentinels identity lost when pickled (unittest.mock)

Peter Otten added the comment: From looking at the sentinel code it appears a sentinel's identity is controlled by its name alone, i. e. sentinel.foo is sentinel.foo If that's the desired behaviour it is well possible to make that indentity survive pickling. I have attached a demo script

[issue20804] Sentinels identity lost when pickled (unittest.mock)

R. David Murray added the comment: Personally I think that's a bad idea. Anything that implies that object identity can survive pickle/unpickle is IMO bad, since pickle is a serialization protocol and in real life (as opposed to tests) the process that does the unpickle is almost always not

[issue20804] Sentinels identity lost when pickled (unittest.mock)

Peter Otten added the comment: I have no experience with unittest.mock, so I'm in no position to contradict... Vlastimil, could you give your use case? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20804

[issue20804] Sentinels identity lost when pickled (unittest.mock)

Vlastimil Zíma added the comment: I encountered this problem when I tested code with cache, something like def test_foo(self): cache.set('a_key', sentinel.status) # Performs pickle self.assertEqual( cache.get('a_key'), # Performs unpickle

[issue20804] Sentinels identity lost when pickled (unittest.mock)

R. David Murray added the comment: Yes, exactly, you don't use is for equality comparison. It tests object identity, which is why it makes sense to use it with a sentinel. The 'name' of the sentinel is purely a way to store and retrieve particular sentinel objects. The sentinel itself does