[issue26906] format(object.__reduce__) fails intermittently

Serhiy Storchaka added the comment: Yet one demonstration of this bug: $ ./python -IS >>> import operator >>> operator.length_hint(iter("abc")) 0 >>> import collections.abc >>> operator.length_hint(iter("abc")) 3 -- ___ Python tracker

[issue26906] format(object.__reduce__) fails intermittently

Guido van Rossum added the comment: Serhiy -- please do what do you think we should do. At this point I'm open to just about anything, but I don't feel comfortable creating or reviewing patches any more. -- assignee: gvanrossum -> serhiy.storchaka

[issue26906] format(object.__reduce__) fails intermittently

Serhiy Storchaka added the comment: Yet one similar bug: issue11702. -- ___ Python tracker ___ ___

[issue26906] format(object.__reduce__) fails intermittently

Serhiy Storchaka added the comment: Similar bug just was introduced in issue21124. -- priority: normal -> high versions: +Python 3.7 ___ Python tracker

[issue26906] format(object.__reduce__) fails intermittently

Antti Haapala added the comment: Could it be possible to to make the debug build absolutely abort on any usage of PyType's that are not readied, usage including instantiating them. Then, instead of changing all `static` linkages to `extern`s (as in Serhiy's first patch) one could rather make

[issue26906] format(object.__reduce__) fails intermittently

Guido van Rossum added the comment: Because the data structure that defines a type is just data, and at some point PyType_Ready() must be called. The question is how to do this, given that nobody can (or needs to) produce a definitive list of all types. --

[issue26906] format(object.__reduce__) fails intermittently

Antti Haapala added the comment: I am not an expert on PyType internals, so I am wondering why is the PyType_Ready'ing done implicitly at all? -- ___ Python tracker

[issue26906] format(object.__reduce__) fails intermittently

Guido van Rossum added the comment: Probably. -- ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue26906] format(object.__reduce__) fails intermittently

Serhiy Storchaka added the comment: There is one test (ClassPropertiesAndMethods.test_mutable_bases_with_failing_mro in test_descr) that crashes with the code from issue551412 because _PyType_Lookup() is recursive called from PyType_Ready(). Is this the reason? My patch prevents recursive

[issue26906] format(object.__reduce__) fails intermittently

Guido van Rossum added the comment: But the problem isn't limited to format()... Why would format() be special? -- ___ Python tracker ___

[issue26906] format(object.__reduce__) fails intermittently

Terry J. Reedy added the comment: Is there a way to have format() try to force the initialization, by explicitly doing the equivalent of obj.__format__, at least for types, instead of raising the TypeError? -- nosy: +terry.reedy ___ Python tracker

[issue26906] format(object.__reduce__) fails intermittently

Guido van Rossum added the comment: Sadly it's been a very long time since I wrote that code and I don't recall much about it. I presume there was a good reason for not to do it in _PyType_Lookup(), but who knows -- maybe the oroginal approach was just too naive and nobody cared? I'm not excited

[issue26906] format(object.__reduce__) fails intermittently

Antti Haapala added the comment: And to the other things failing, I was trying to find out which of the magic method ops fail, and for that tried to find out the `dir` of list iterator. Well... % python3.5 -S Python 3.5.0+ (default, Oct 11 2015, 09:05:38) [GCC 5.2.1

[issue26906] format(object.__reduce__) fails intermittently

Antti Haapala added the comment: I can reproduce the bug in 3.5.0+ Ubuntu with list iterator, if I execute python with -S: % python3.5 -S Python 3.5.0+ (default, Oct 11 2015, 09:05:38) [GCC 5.2.1 20151010] on linux >>> format(iter([])) Traceback (most recent call last):

[issue26906] format(object.__reduce__) fails intermittently

Serhiy Storchaka added the comment: The problem is that format() fails for instances of some classes, because the type still is not initialized. The simplest example -- list iterator. >>> format(iter([])) Traceback (most recent call last): File "", line 1, in TypeError: Type listiterator

[issue26906] format(object.__reduce__) fails intermittently

Serhiy Storchaka added the comment: Added a check for Py_TPFLAGS_READYING to prevent recursive calling. -- Added file: http://bugs.python.org/file42718/init_type_in_pytype_lookup.patch ___ Python tracker

[issue26906] format(object.__reduce__) fails intermittently

Changes by Serhiy Storchaka : Removed file: http://bugs.python.org/file42717/init_type_in_pytype_lookup.patch ___ Python tracker ___

[issue26906] format(object.__reduce__) fails intermittently

Guido van Rossum added the comment: Serhiy, I'm happy to help, but I'm not sure what you're asking me to do. Decide between different patches? I can't even repro the issue. -- ___ Python tracker

[issue26906] format(object.__reduce__) fails intermittently

Serhiy Storchaka added the comment: An alternative way is just call PyType_Ready from _PyType_Lookup if type->tp_mro is NULL. Here is a patch against 2.7 that restores the solution from issue551412, but returns NULL if type->tp_mro is still NULL after calling PyType_Ready. I found one place

[issue26906] format(object.__reduce__) fails intermittently

Serhiy Storchaka added the comment: A number of other types are not initialized until you request an attribute. Here is larger patch for 2.7 that makes 38 types to be explicitly initialized. -- Added file: http://bugs.python.org/file42716/init_types-2.7.patch

[issue26906] format(object.__reduce__) fails intermittently

Serhiy Storchaka added the comment: There is similar issue on 3.x: >>> import array >>> it = iter(array.array('i')) >>> format(it) Traceback (most recent call last): File "", line 1, in TypeError: Type arrayiterator doesn't define __format__ >>> type(it).__format__ >>> format(it) ''

[issue26906] format(object.__reduce__) fails intermittently

Serhiy Storchaka added the comment: Proposed patch makes method descriptors types to be explicitly initialized as in 3.x. -- components: +Interpreter Core keywords: +patch nosy: +serhiy.storchaka stage: -> patch review Added file:

[issue26906] format(object.__reduce__) fails intermittently

Changes by Eric V. Smith : -- nosy: +eric.smith type: -> behavior ___ Python tracker ___

[issue26906] format(object.__reduce__) fails intermittently

Antti Haapala added the comment: s/explicitly do/explicitly access/ -- ___ Python tracker ___ ___

[issue26906] format(object.__reduce__) fails intermittently

New submission from Antti Haapala: This is an annoying heisenbug; it seems that some objects cannot be formatted until you explicitly do obj.__format__. For example `object.__reduce__` behaves like this: Python 2.7.10 (default, Oct 14 2015, 16:09:02) [GCC 5.2.1 20151010] on linux2