[issue17839] base64 module should use memoryview
Roundup Robot added the comment: New changeset e53984133740 by Nick Coghlan in branch 'default': Issue #17839: mention base64 change in What's New http://hg.python.org/cpython/rev/e53984133740 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17839 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17839] base64 module should use memoryview
Nick Coghlan added the comment: binascii already only supports simple C contiguous buffers, expanding it and the base64 module to handle anything else should be a separate RFE. -- dependencies: -Allow memoryview.cast() for empty views ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17839 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17839] base64 module should use memoryview
Nick Coghlan added the comment: However, _input_type_check should enforce that (as binascii does), so I'll add that before committing. -- assignee: - ncoghlan ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17839 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17839] base64 module should use memoryview
Nick Coghlan added the comment: After working through this, I found that the modern base64 API just relies on the checks in binascii. All that checks for is: 1. Can the data by exported using PyBUF_SIMPLE? 2. Is it C contiguous? It completely ignores the number of dimensions and the format information. I added tests to at least capture this behaviour, even though it seems a little dubious to me. For the legacy API, I didn't relax the input checks that far - the legacy API will still complain if there is more than 1 dimension and if the format code isn't one of 'c', 'b' or 'B'. That's already substantially more permissive than what it supported in previous versions. Just running the full test suite now, will push after that finishes. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17839 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17839] base64 module should use memoryview
Roundup Robot added the comment: New changeset d90f25e1a705 by Nick Coghlan in branch 'default': Close #17839: support bytes-like objects in base64 module http://hg.python.org/cpython/rev/d90f25e1a705 -- nosy: +python-dev resolution: - fixed stage: patch review - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17839 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17839] base64 module should use memoryview
Changes by Serhiy Storchaka storch...@gmail.com: -- dependencies: +Allow memoryview.cast() for empty views ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17839 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17839] base64 module should use memoryview
Serhiy Storchaka added the comment:
It works only if input also supports slicing and this slicing is corresponded
to bytes-slicing. It perhaps doesn't catch a non C-continuous buffers.
Actually we need something like (unlikely cast() doesn't work with empty
buffers):
s = memoryview(s)
if not s:
return b''
s = s.cast('B')
...
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17839
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17839] base64 module should use memoryview
Nick Coghlan added the comment: We now also need to update the new footnote that I added for issue 17844 in http://hg.python.org/cpython/rev/801567d6302c -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17839 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17839] base64 module should use memoryview
Nick Coghlan added the comment: The codec uses the old API that breaks the encoded output up into multiple lines. The new patch: 1. Also changes the behaviour of the older de/encodebytes API 2. Checks that all the defined binary transform codecs actually support memoryview as input for both encoding and decoding (and that the data roundtrips correctly) -- Added file: http://bugs.python.org/file30346/issue17839_base64_buffer_input_3.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17839 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17839] base64 module should use memoryview
Nick Coghlan added the comment: Just adding a note for easier cross-referencing: the old behaviour of these functions was one of the culprits that led to the removal of the codec aliases as discussed in issue 7475 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17839 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17839] base64 module should use memoryview
Serhiy Storchaka added the comment: Thank you Ezio and Nick for your comments. I suggest renaming _bytes_from_decode_data to _bytes_for_decoding and adding _bytes_for_encoding. I rather think a TypeError exception raised by `memoryview(s).tobytes()` is good enough and we don't need a special wrapper. -- Added file: http://bugs.python.org/file30337/base64_buffer_input_2.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17839 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17839] base64 module should use memoryview
Nick Coghlan added the comment: Oops, my review comments don't actually make sense because I looked at the patch in isolation, rather than checking the full context in the module. Sorry about that. We have 2 different cases to deal with, only one of which currently has a helper function. I suggest renaming _bytes_from_decode_data to _bytes_for_decoding and adding _bytes_for_encoding. The difference between them is the implicit encoding of pure ASCII strings to bytes in the decoding case and the details of the error message thrown. The encoding and decoding functions should then use the appropriate coercion helper for both the input data and for altchars. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17839 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17839] base64 module should use memoryview
Serhiy Storchaka added the comment: Nick, what you say about the patch? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17839 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17839] base64 module should use memoryview
Changes by STINNER Victor victor.stin...@gmail.com: -- nosy: +haypo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17839 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17839] base64 module should use memoryview
Serhiy Storchaka added the comment: Here is a patch which allows bytes-like arguments in the base64 module. I just removed type checking if underlying function raises an exception with an appropriate message. I'm not sure about b32encode(), perhaps we can left an exception from memoryview(). -- Added file: http://bugs.python.org/file30044/base64_buffer_input.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17839 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17839] base64 module should use memoryview
Antoine Pitrou added the comment: Yes, this should probably be done the other way around: wrap the base64 argument in a memoryview, and encode/decode from it. -- nosy: +pitrou ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17839 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17839] base64 module should use memoryview
Nick Coghlan added the comment: As Serhiy and Antoine noted, what I had in mind here was to try to wrap the input in a memoryview if it wasn't an instance of str, bytes or bytearray. An existing memoryview will be passed back unmodified, while something like array.array will provide a view into its raw data for encoding or decoding. The tests would then cover both memoryview and array.array to ensure it was all working as expected. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17839 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17839] base64 module should use memoryview
Serhiy Storchaka added the comment: Note that some functions use bytes/bytearray methods and an argument should converted to bytes for them. Other functions use C functions which supports the buffer protocol and do not required any conversion. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17839 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17839] base64 module should use memoryview
Kushal Das added the comment: A patch with tests update to use memoryview in base64 module. -- keywords: +patch Added file: http://bugs.python.org/file30022/issue17839_v1.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17839 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17839] base64 module should use memoryview
Changes by Ezio Melotti ezio.melo...@gmail.com: -- nosy: +ezio.melotti stage: needs patch - patch review ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17839 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17839] base64 module should use memoryview
Serhiy Storchaka added the comment: Not only memoryview should be supported, but any class which supports the buffer protocol and is C-contiguous (i.e. array.array). -- nosy: +serhiy.storchaka ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17839 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17839] base64 module should use memoryview
New submission from Nick Coghlan: The base64 module is currently restricted specifically to bytes and bytearray objects. By using memoryview, it could effectively decode any input type that provides an 8-bit C contiguous view of the underlying data. -- components: Library (Lib) messages: 187760 nosy: ncoghlan priority: normal severity: normal stage: needs patch status: open title: base64 module should use memoryview type: enhancement versions: Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17839 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17839] base64 module should use memoryview
Changes by Serhiy Storchaka storch...@gmail.com: -- dependencies: +Add base64 module tests for a bytearray argument ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17839 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17839] base64 module should use memoryview
Kushal Das added the comment: Working on this. -- nosy: +kushaldas ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17839 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17839] base64 module should use memoryview
Changes by Barry A. Warsaw ba...@python.org: -- nosy: +barry ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17839 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
