[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Change by Terry J. Reedy : -- versions: +Python 3.10, Python 3.9 -Python 2.7, Python 3.3, Python 3.4 ___ Python tracker ___ ___

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Mark Dickinson added the comment: > The other way to solve my problem would be to provide an operator module > function (operator.as_float?) that does a duck-typed conversion of an > arbitrary Python object to a float. This does feel like the *right* solution to me. See #40801 and the

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Mark Dickinson added the comment: [Serhiy] > * Undeprecate accepting __index__ and __int__ returning instances of int > sublasses. There is no difference from the side of using int and index(), but > it can simplify user implementations of __index__ and __int__. I'm not sure about this.

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Serhiy Storchaka added the comment: The current status: * Decimal and Fraction are no longer automatically converted to int when pass to functions implemented in C. PyLong_AsLong() etc no longer call __int__. (see issue36048 and issue37999) * operator.index() and PyNumber_Index() always

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

STINNER Victor added the comment: It started to write a new issue, but then I found this issue issue (created in 2013!) which is still open. So let me write my comment here instead. The code to convert a number to an integer is quite complex in Python. There are *many* ways to do that and

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Serhiy Storchaka added the comment: Mark, I think you can reopen the PR and merge it in 3.9 now. As for my proposition to use the FutureWarning first, I think it is not necessary. The behavior change is very subtle and will affects only int subclasses with overridden __index__. Similar

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Mark Dickinson added the comment: I've closed the PR. Reassigning back to Ethan. -- assignee: mark.dickinson -> ethan.furman nosy: +ethan.furman ___ Python tracker ___

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Change by Mark Dickinson : -- nosy: -mark.dickinson ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Serhiy Storchaka added the comment: > Can we at least switch to PyLong_CheckExact? This is a behavior change and as such should be preceded by a period of warning. If we go this way I propose to add a FutureWarning for int subclasses with overridden __index__. As for turning the deprecated

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Mark Dickinson added the comment: > Can we at least switch to PyLong_CheckExact? +1 > I am not sure that converting to an exact int in low-level C API functions is > the best option. I am sure. :-) The number of naturally-occurring cases where we're actually passing a subtype of `int`

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Raymond Hettinger added the comment: Can we at least switch to PyLong_CheckExact? That would fix Barry's original issue and should run slightly faster. -- nosy: +rhettinger ___ Python tracker

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Serhiy Storchaka added the comment: I am not sure that raising an error is the best option. We can just convert an integer subclass to an exact int using _PyLong_Copy(). I am not sure that converting to an exact int in low-level C API functions is the best option. In many cases we use only

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Change by Mark Dickinson : -- pull_requests: +13623 pull_request: https://github.com/python/cpython/pull/13740 ___ Python tracker ___

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Change by Mark Dickinson : -- assignee: ethan.furman -> mark.dickinson ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Mark Dickinson added the comment: I'm working on a PR that finally changes the DeprecationWarnings that Serhiy introduced to TypeErrors; I think that should be acceptable, four Python versions and some years later. With that PR: - int will always return something of exact type `int` (or

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Serhiy Storchaka added the comment: See also issue33039. -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Changes by Ethan Furman et...@stoneleaf.us: -- nosy: -ethan.furman ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17576 ___ ___ Python-bugs-list

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Nick Coghlan added the comment: OK, something appears to have gotten confused along the way here. Barry's original problem report was that operator.index() was returning a different answer than operator.__index__() for int subclasses. Absolutely nothing to do with the int builtin at all.

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Serhiy Storchaka added the comment: The tests are checking that they are the same value (8) and the same type (int)? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17576 ___

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Manuel Jacob added the comment: Maybe I'm missing something, but it seems to me that test_int_subclass_with_index() is testing for the exactly wrong behaviour. Isn't the point of this issue that operator.index(a) should be equal to a.__index__()? Why are the tests checking that they are

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Serhiy Storchaka added the comment: Ah, it just checks current behavior. So we will know when this will be changed. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17576 ___

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Changes by Ethan Furman et...@stoneleaf.us: -- assignee: - ethan.furman ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17576 ___ ___

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Serhiy Storchaka added the comment: I have doubts about this issue, so I have unassigned it from myself. -- assignee: serhiy.storchaka - ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17576

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Roundup Robot added the comment: New changeset a3de2b3881c1 by Serhiy Storchaka in branch '3.3': Issue #17576: Removed deprecation warnings added in changeset 618cca51a27e. http://hg.python.org/cpython/rev/a3de2b3881c1 -- ___ Python tracker

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Serhiy Storchaka added the comment: PyLong_AsUnsignedLong() and PyLong_AsLong() can return different values for same object. Why __int__ and __index__ should be used for int subclasses at all? I'm not sure this is right solution. -- ___ Python

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Roundup Robot added the comment: New changeset 618cca51a27e by Serhiy Storchaka in branch '3.3': Issue #17576: Deprecation warning emitted now when __int__() or __index__() http://hg.python.org/cpython/rev/618cca51a27e New changeset 59fb79d0411e by Serhiy Storchaka in branch 'default': Issue

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Serhiy Storchaka added the comment: I first committed safe part of patch, which doesn't change behavior, only adds warnings and tests. Here is other part (very simple), which change behavior (in sum they are equal to issue17576_v3.patch with minor changes). * PyNumber_Index() now calls

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Serhiy Storchaka added the comment: Even with last patch int() can return non-int: class A(int): ... def __int__(self): return True ... def __repr__(self): return 'A' ... class B: ... def __int__(self): return A() ... class C: ... def __trunc__(self): return A() ...

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Serhiy Storchaka added the comment: Here is updated patch. There is no more overhead in PyLong_As* functions. Simplified PyNumber_Index(). assertWarns() now used instead of support.check_warnings(). Added new tests. -- Added file: http://bugs.python.org/file33077/issue17576_v3.patch

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Nick Coghlan added the comment: Took me a while to figure out that one of the code paths was being deleted as redundant because the type machinery will always fill in nb_int for int subclasses, but Serhiy's patch looks good to me. -- assignee: mark.dickinson - serhiy.storchaka

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Mark Dickinson added the comment: Thanks, Serhiy. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17576 ___ ___ Python-bugs-list mailing list

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Serhiy Storchaka added the comment: What about PyLong_AsSsize_t(), PyLong_AsUnsignedLong(), and PyLong_AsSize_t()? They are ignore __int__(). PyLong_AsVoidPtr() calls PyLong_AsLong(), PyLong_AsUnsignedLong(), PyLong_AsLongLong(), or PyLong_AsUnsignedLongLong() (depending on pointer's size

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Nick Coghlan added the comment: The ssize_t functions deliberately ignore lossy int conversions (e.g. from floats) - that's why they only work on types that implement __index__. -- ___ Python tracker rep...@bugs.python.org

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Serhiy Storchaka added the comment: The ssize_t functions deliberately ignore lossy int conversions (e.g. from floats) - that's why they only work on types that implement __index__. They ignore __index__. -- ___ Python tracker

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Mark Dickinson added the comment: Yes, the PyLong_As... conversions are a mess. In theory, they should all use __index__ if available, and ignore __int__. In practice, it's a mess that's difficult to change without breaking things. I think there are already some other issues open on this

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Serhiy Storchaka added the comment: Ping. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17576 ___ ___ Python-bugs-list mailing list

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Mark Dickinson added the comment: Ping. Bah. Sorry; I haven't had time to deal with this. Serhiy: are you interested in taking over? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17576

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Mark Dickinson added the comment: I still need to act on some of Serhiy's comments. I do plan to get this in for 3.4. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17576 ___

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com: -- nosy: +Arfrever ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17576 ___

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Ethan Furman added the comment: Where do we stand with this issue? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17576 ___ ___ Python-bugs-list

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Serhiy Storchaka added the comment: Yet some nitpicks. Currently the code of _PyLong_FromNbInt() is inlined and the do_decref flag is used to prevent needless change refcounts of int objects (see also issue18797). In proposed patch common code is extracted into the _PyLong_FromNbInt()

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Serhiy Storchaka added the comment: On PyPy 1.8.0 operator.index(True) returns 1. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17576 ___ ___

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Serhiy Storchaka added the comment: And yet one nitpick. For int subclasses which doesn't overload the __int__ method the patch calls default int.__int__ which creates a copy of int object. This is redundant in PyLong_As* functions because they only extract C int value and drop Python int

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Nick Coghlan added the comment: On 21 Aug 2013 15:47, Mark Dickinson rep...@bugs.python.org wrote: Mark Dickinson added the comment: Shouldn't it be PendingDeprecationWarning? Hmm. Possibly. I'm not sure what the policy is any more regarding DeprecationWarning versus

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Changes by Serhiy Storchaka storch...@gmail.com: -- stage: - patch review ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17576 ___ ___

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Serhiy Storchaka added the comment: Shouldn't it be PendingDeprecationWarning? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17576 ___ ___

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Mark Dickinson added the comment: Shouldn't it be PendingDeprecationWarning? Hmm. Possibly. I'm not sure what the policy is any more regarding DeprecationWarning versus PendingDeprecationWarning. Nick? -- ___ Python tracker

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Mark Dickinson added the comment: See the related python-dev discussion started by Mark Shannon here: http://mail.python.org/pipermail/python-dev/2013-March/125022.html and continuing well into April here: http://mail.python.org/pipermail/python-dev/2013-April/125042.html The consensus that

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Mark Dickinson added the comment: New patch that replaces the TypeErrors with warnings and fixes a refleak in the original patch. -- Added file: http://bugs.python.org/file31149/issue17576_v2.patch ___ Python tracker rep...@bugs.python.org

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Nick Coghlan added the comment: The deprecation warning version looks good to me. Something I'll mention explicitly (regarding the PyCon discussions that Eric mentioned above), is that we unfortunately couldn't do something like this for the various concrete APIs with overly permissive

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Serhiy Storchaka added the comment: if (PyLong_CheckExact(item) || (PyLong_Check(item) item-ob_type-tp_as_number-nb_index == PyLong_Type.tp_as_number-nb_index)) -- nosy: +serhiy.storchaka ___ Python tracker rep...@bugs.python.org

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

STINNER Victor added the comment: Alex In my opinion that should use PyLong_CheckExact +1 -- nosy: +haypo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17576 ___

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Changes by Terry J. Reedy tjre...@udel.edu: -- versions: -Python 3.1, Python 3.2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17576 ___ ___

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Changes by Ethan Furman et...@stoneleaf.us: -- nosy: +stoneleaf ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17576 ___ ___ Python-bugs-list

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Changes by Mark Dickinson dicki...@gmail.com: -- nosy: +mark.dickinson ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17576 ___ ___

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Barry A. Warsaw added the comment: You also end up with this nice bit of inconsistency: x = myint(7) from operator import index range(10)[6:x] range(6, 7) range(10)[6:x.__index__()] range(6, 8) range(10)[6:index(x)] range(6, 7) Granted, it's insane to have __index__() return a different

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Changes by Barry A. Warsaw ba...@python.org: -- title: PyNumber_Index() is not int-subclass friendly - PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie) ___ Python tracker rep...@bugs.python.org

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Eric Snow added the comment: Would it be okay to do a check on __index__ after the PyLong_Check() succeeds? Something like this: if (PyLong_Check(item) item-ob_type-tp_as_number-nb_index == PyLong_Type.tp_as_number-nb_index) { Py_INCREF(item); return item; }

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Alex Gaynor added the comment: In my opinion that should use PyLong_CheckExact -- nosy: +alex ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17576 ___

[issue17576] PyNumber_Index() is not int-subclass friendly (or operator.index() docos lie)

Barry A. Warsaw added the comment: On Mar 30, 2013, at 12:29 AM, Eric Snow wrote: Would it be okay to do a check on __index__ after the PyLong_Check() succeeds? Something like this: if (PyLong_Check(item) item-ob_type-tp_as_number-nb_index == PyLong_Type.tp_as_number-nb_index)