Changes by Serhiy Storchaka storch...@gmail.com:
--
resolution: - fixed
stage: commit review - resolved
status: open - closed
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23688
___
Roundup Robot added the comment:
New changeset 4dc69e5124f8 by Serhiy Storchaka in branch 'default':
Issue #23688: Added support of arbitrary bytes-like objects and avoided
https://hg.python.org/cpython/rev/4dc69e5124f8
--
nosy: +python-dev
___
Serhiy Storchaka added the comment:
OK, so left it as is if nobody complains.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23688
___
___
Wolfgang Maier added the comment:
I see now that it is just issue21560 that went into 2.7 and that's fine.
As I said: sorry for the noise
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23688
Serhiy Storchaka added the comment:
I think I saw that you committed this also to the 2.7 branch,
I committed only working tests and a fix from issue21560.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23688
Serhiy Storchaka added the comment:
Here is a patch that restores support on non-contiguous memoryviews.
It would be better to drop support of non-contiguous data, because it worked
only by accident. Needed support of only bytes-like memoryviews written by
BufferedWriter.
--
Added
Wolfgang Maier added the comment:
to preserve compatibility:
there is the memoryview.c_contiguous flag. Maybe we should just check it and if
it is False fall back to the old copying behavior ?
--
___
Python tracker rep...@bugs.python.org
Wolfgang Maier added the comment:
something like:
def write(self,data):
self._check_closed()
if self.mode != WRITE:
import errno
raise OSError(errno.EBADF, write() on read-only GzipFile object)
if self.fileobj is None:
raise
Stefan Krah added the comment:
In a sense, the old behavior was an artefact of silently copying the
memoryview to bytes.
It likely wasn't intentional, but tobytes() *is* used to serialize
weird arrays to their C-contiguous representation (following the
logical structure of the array rather
Stefan Krah added the comment:
I just see that non-contiguous arrays didn't work in 2.7 either,
so that was probably the original intention.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23688
Wolfgang Maier added the comment:
Serhiy:
I think I saw that you committed this also to the 2.7 branch, but that would
not work since memoryviews do not have the nbytes attribute (they do not seem
to have cast either). One would have to calculate the length instead from other
properties.
Wolfgang Maier added the comment:
ouch. haven't thought of this.
OTOH, just plain io with your example:
with open('xy', 'wb') as f:
f.write(y)
Traceback (most recent call last):
File pyshell#29, line 2, in module
f.write(y)
BufferError: memoryview: underlying buffer is not
Stefan Krah added the comment:
I think there's a behavior change: Before you could gzip non-contiguous
views directly, now that operation raises BufferError.
--
nosy: +skrah
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23688
Serhiy Storchaka added the comment:
Could you provide an example?
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23688
___
___
Python-bugs-list
Stefan Krah added the comment:
Sure:
import gzip
x = memoryview(b'x' * 10)
y = x[::-1]
with gzip.GzipFile(x, 'w') as f:
f.write(y)
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23688
Serhiy Storchaka added the comment:
In general the patch LGTM.
--
assignee: - serhiy.storchaka
stage: - commit review
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23688
___
Changes by Wolfgang Maier wolfgang.ma...@biologie.uni-freiburg.de:
Added file: http://bugs.python.org/file38650/write_bytes_like_objects_v3.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23688
Wolfgang Maier added the comment:
Here is a revised version of my patch addressing Serhiy's review comments.
--
Added file: http://bugs.python.org/file38639/write_bytes_like_objects_v2.patch
___
Python tracker rep...@bugs.python.org
Wolfgang Maier added the comment:
ok, I've prepared a patch including tests based on my last suggestion, which I
think is ready for getting reviewed.
--
Added file: http://bugs.python.org/file38600/write_bytes_like_objects.patch
___
Python tracker
Wolfgang Maier added the comment:
Thanks everyone for the lively discussion !
I like Serhiy's idea of making write work with arbitrary objects supporting the
buffer protocol. In fact, I noticed before that GzipFile.write misbehaves with
array.array input. It pretends to accept that, but
Serhiy Storchaka added the comment:
Better way is data = data.cast('B').
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23688
___
___
STINNER Victor added the comment:
Better way is data = data.cast('B').
Why is this cast required? Can you please elaborate? If some memoryview must be
rejected, again, we need more unit tests.
--
___
Python tracker rep...@bugs.python.org
Martin Panter added the comment:
I would say that the current patch looks correct enough, in that it would still
get the correct lengths when a memoryview() object is passed in. The zlib
module’s crc32() function and compress() method already seem to support
arbitrary bytes-like objects.
But
STINNER Victor added the comment:
While we are here, it is possible to add the support of general byte-like
objects.
With and without the patch, write() accepts bytes, bytearray and memoryview.
Which other byte-like types do you know?
writeframesraw() method of aifc, sunau and wave modules
Serhiy Storchaka added the comment:
You patch is correct Wolfgang, but with cast('B') the patch would be smaller
(no need to replace len(data) to nbytes).
While we are here, it is possible to add the support of general byte-like
objects.
if not isinstance(data, bytes):
data =
Serhiy Storchaka added the comment:
With and without the patch, write() accepts bytes, bytearray and memoryview.
Which other byte-like types do you know?
The bytes-like object term is used as an alias of an instance of type that
supports buffer protocol. Besides bytes, bytearray and
New submission from Wolfgang Maier:
I thought I'd go back to work on a test patch for issue21560 today, but now I'm
puzzled by the explicit handling of memoryviews in gzip.GzipFile.write.
The method is defined as:
def write(self,data):
self._check_closed()
if self.mode !=
Changes by Wolfgang Maier wolfgang.ma...@biologie.uni-freiburg.de:
--
keywords: +patch
Added file: http://bugs.python.org/file38521/memoryview_write.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23688
STINNER Victor added the comment:
The patch looks good to be me, but it lacks an unit test. Can you please add a
simple unit test to ensure that it's possible to memoryview to write(), and
that the result is correct? (ex: uncompress and ensure that you get the same
content)
--
nosy:
Changes by STINNER Victor victor.stin...@gmail.com:
--
nosy: +serhiy.storchaka
type: resource usage - performance
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23688
___
Wolfgang Maier added the comment:
Here is a patch with memoryview tests.
Are tests and code patches supposed to go in one file or separate ones ?
--
Added file: http://bugs.python.org/file38526/test_memoryview_write.patch
___
Python tracker
Wolfgang Maier added the comment:
memoryview is converted to bytes because len() for memoryview returns a size
of first dimension (a number of items for one-dimension view), not a number
of bytes.
m = memoryview(array.array('I', [1, 2, 3]))
len(m)
3
len(m.tobytes())
12
Wolfgang Maier added the comment:
@Serhiy:
Why would data = data.cast('B') be required ? When would the memoryview not be
in 'B' format already ?
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23688
Serhiy Storchaka added the comment:
memoryview is converted to bytes because len() for memoryview returns a size of
first dimension (a number of items for one-dimension view), not a number of
bytes.
m = memoryview(array.array('I', [1, 2, 3]))
len(m)
3
len(m.tobytes())
12
len(m.cast('B'))
STINNER Victor added the comment:
Are tests and code patches supposed to go in one file or separate ones ?
It's more convinient to have a single patch with both changes.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23688
35 matches
Mail list logo
