[issue29159] Regression in bytes constructor

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
pull_requests:  -1003

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29159] Regression in bytes constructor

[Donald Stufft]

Changes by Donald Stufft :


--
pull_requests: +1003

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29159] Regression in bytes constructor

[INADA Naoki]

INADA Naoki added the comment:

> patch LGTM except that arguments of assertEqual are in reversed order.

fixed.

--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29159] Regression in bytes constructor

[Roundup Robot]

Roundup Robot added the comment:

New changeset 505cc50ddc82 by INADA Naoki in branch '3.6':
Issue #29159: Fix regression in bytes(x) when x.__index__() raises Exception.
https://hg.python.org/cpython/rev/505cc50ddc82

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29159] Regression in bytes constructor

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

The code restored by 29159-index-fallback.patch is slightly different from the 
code before issue27704, but the patch LGTM except that arguments of assertEqual 
are in reversed order.

> Can you explain what your patch does?  I don't understand why you single out 
> the OverflowError.  When is __index__ expected to raise it?

PyNumber_AsSsize_t(arg, PyExc_OverflowError) raises an OverflowError when the 
result of __index__ don't fit in Py_ssize_t.

> I wonder if we could check for buffer protocol support before detecting an 
> integer argument?

This would fix an issue with NumPy arrays, but it is much harder to implement.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29159] Regression in bytes constructor

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

On the other hand, the documentation for the bytearray constructor [1] lists 
integer argument first. 

[1]: https://docs.python.org/3/library/functions.html#bytearray

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29159] Regression in bytes constructor

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

@inada.naoki

Sorry, I still don't understand the role of OverflowError.

With respect to my earlier suggestion that buffer protocol should have priority 
over __index__, note that the documentation implies the same order:

>>> help(bytes)

class bytes(object)
 |  bytes(iterable_of_ints) -> bytes
 |  bytes(string, encoding[, errors]) -> bytes
 |  bytes(bytes_or_buffer) -> immutable copy of bytes_or_buffer
 |  bytes(int) -> bytes object of size given by the parameter initialized with 
null bytes
 |  bytes() -> empty bytes object
 |
 |  Construct an immutable array of bytes from:
 |- an iterable yielding integers in range(256)
 |- a text string encoded using the specified encoding
 |- any object implementing the buffer API.
 |- an integer
..

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29159] Regression in bytes constructor

[INADA Naoki]

INADA Naoki added the comment:

@belopolsky
It is backported behavior from Python 3.5, to fix regression.

See code review discussion in #27704.
The first patch kept the behavior, but we simplified it in second patch.
That cause this regression.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29159] Regression in bytes constructor

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

Inada,

Can you explain what your patch does?  I don't understand why you single out 
the OverflowError.  When is __index__ expected to raise it?

>>> (2**300).__index__()
2037035976334486086268445688409378161051468393665936250636140449354381299763336706183397376

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29159] Regression in bytes constructor

[INADA Naoki]

Changes by INADA Naoki :


--
keywords: +patch
Added file: http://bugs.python.org/file46147/29159-index-fallback.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29159] Regression in bytes constructor

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

Also relevant:

  * #20895 - Add bytes.empty_buffer and deprecate bytes(17) for the same purpose
  * PEP 467 - Minor API improvements for binary sequences

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29159] Regression in bytes constructor

[Alexander Belopolsky]

Changes by Alexander Belopolsky :


--
nosy: +ncoghlan

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29159] Regression in bytes constructor

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

I don't think we should put too much effort into preserving numpy behavior.  
Consider this python 3.5 session:

>>> import numpy
>>> a = numpy.array([1])
>>> bytes(a)
__main__:1: VisibleDeprecationWarning: converting an array with ndim > 0 to an 
index will result in an error in the future
b'\x00'
>>> a = numpy.array([2, 2])
>>> bytes(a)
b'\x02\x00\x00\x00\x02\x00\x00\x00'

It looks like this behavior is just an artifact of ndarray providing both 
__index__ and buffer protocol and not something thought out by numpy developers.

I wonder if we could check for buffer protocol support before detecting an 
integer argument?  I also recall a discussion of deprecating bytes(int) 
altogether.  See 
.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29159] Regression in bytes constructor

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Ah, if numpy arrays are affected, this should be restored. But swallowing an 
arbitrary exception doesn't look good to me.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29159] Regression in bytes constructor

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
nosy: +inada.naoki

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29159] Regression in bytes constructor

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Indeed, there is behavior change. It is easy to revert the old behavior. But 
was the old behavior designed? It looks as a side effect of the implementation.

What is a reason of making an array an integer type?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29159] Regression in bytes constructor

[Alexander Belopolsky]

Alexander Belopolsky added the comment:

My test code may seem contrived, but numpy arrays exhibit similar behavior:

>>> a = numpy.array([2, 2])
>>> bytes(a)
Traceback (most recent call last):
  File "", line 1, in 
TypeError: only integer arrays with one element can be converted to an index

Under python 3.5, the result was

>>> bytes(a)
b'\x02\x00\x00\x00\x02\x00\x00\x00'

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29159] Regression in bytes constructor

[Alexander Belopolsky]

New submission from Alexander Belopolsky:

Consider the following code:

$ cat bug.py
from array import array
class C(array):
def __new__(cls):
return array.__new__(cls, 'B', b'abc')
def __index__(self):
raise TypeError
x = C()
print(bytes(x))

It works under python 3.5:

$ python3.5 bug.py
b'abc'

but raises a TypeError under python 3.6:

$ python3.6 bug.py
Traceback (most recent call last):
  File "bug.py", line 8, in 
print(bytes(x))
  File "bug.py", line 6, in __index__
raise TypeError
TypeError

It looks like this was introduced in issue #27704.

(Ref: e/pyq#827)

--
keywords: 3.6regression
messages: 284663
nosy: belopolsky, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Regression in bytes constructor
type: behavior
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com