[issue21071] struct.Struct.format is bytes, but should be str

2017-06-23 Thread STINNER Victor
STINNER Victor added the comment: Ok, I changed struct.Struct.format type to str (Unicode string). If someone wants to modify the C code to use a PyUnicodeObject rather than a char*, feel free to propose a further change. Since the initial issue is fixed, I now close the issue. Thank you all

[issue21071] struct.Struct.format is bytes, but should be str

2017-06-23 Thread STINNER Victor
STINNER Victor added the comment: New changeset f87b85f80853c580b1c8bf78a51b0e9a25f6e1a7 by Victor Stinner in branch 'master': bpo-21071: struct.Struct.format type is now str (#845) https://github.com/python/cpython/commit/f87b85f80853c580b1c8bf78a51b0e9a25f6e1a7 --

[issue21071] struct.Struct.format is bytes, but should be str

2017-04-28 Thread Martin Panter
Martin Panter added the comment: I don’t think the API should be expanded to accept arbitrary bytes-like objects as format strings. Struct formats are strings of ASCII-compatible characters, but not arbitrary chunks of memory. I think the main question is whether it is okay to break

[issue21071] struct.Struct.format is bytes, but should be str

2017-04-27 Thread Xiang Zhang
Xiang Zhang added the comment: The warnings are possible to remove I think... but deprecate bytes arguments sounds good. -- ___ Python tracker ___

[issue21071] struct.Struct.format is bytes, but should be str

2017-04-27 Thread Serhiy Storchaka
Serhiy Storchaka added the comment: After changing the type of Struct.format to str we perhaps should deprecate accepting bytes as format. Currently this can lead to emitting a BytesWarning. $ ./python -Wa -b >>> import struct >>> struct.pack('I', 12345) b'90\x00\x00' >>> struct.pack(b'I',

[issue21071] struct.Struct.format is bytes, but should be str

2017-04-26 Thread Xiang Zhang
Xiang Zhang added the comment: +1 for change bytes to str. But struct.Struct() accepts both bytes and str, maybe in future buffer objects. When it gets a bytes object, converting it to a str looks unnecessary to me, and as OP said, comparison (a theoretical use case) could still fail. Could

[issue21071] struct.Struct.format is bytes, but should be str

2017-03-27 Thread STINNER Victor
STINNER Victor added the comment: Ok, I opened a thread on python-dev: https://mail.python.org/pipermail/python-dev/2017-March/147688.html Martin: "At a minimum you should acknowledge it in the “porting” section of What’s New." I wasn't sure if the change was worth it to be mentionned in

[issue21071] struct.Struct.format is bytes, but should be str

2017-03-27 Thread Martin Panter
Martin Panter added the comment: Hi Victor, I’m not sure about changing the data type. As Python 3 grows older, there is potentially more code being written that you break by fixing a bug like this. It is incompatible if you used to write >>> print(struct.Struct('hi').format.decode()) hi I

[issue21071] struct.Struct.format is bytes, but should be str

2017-03-27 Thread Serhiy Storchaka
Serhiy Storchaka added the comment: This should be discussed on Python-Dev first. I already raised this issue on Python-Dev, but don't remember what is the result. -- ___ Python tracker

[issue21071] struct.Struct.format is bytes, but should be str

2017-03-27 Thread STINNER Victor
STINNER Victor added the comment: I created https://github.com/python/cpython/pull/845 to change struct.Struct.format type to str (Unicode). struct.Struct() accepts bytes and str format strings, so it's not really a backward incompatible change. It's just a minor enhancement to help

[issue21071] struct.Struct.format is bytes, but should be str

2017-03-27 Thread STINNER Victor
Changes by STINNER Victor : -- pull_requests: +743 ___ Python tracker ___ ___

[issue21071] struct.Struct.format is bytes, but should be str

2017-03-25 Thread Martin Panter
Martin Panter added the comment: A backwards-compatible way forward would be to preserve (and document) the “format” attribute as a byte string, and add a new attribute which is definitely a text string. Not sure of a good name; perhaps “Struct.text_format” or “format_str” is a start.

[issue21071] struct.Struct.format is bytes, but should be str

2014-12-19 Thread Raymond Hettinger
Raymond Hettinger added the comment: I would like to see this and issue 8934 discussed as a usability bug. As far as I can tell, the current state of affairs an unintended by-product of a rushed effort to split the standard library to bytes apis and unicode apis. I don't see any reason that

[issue21071] struct.Struct.format is bytes, but should be str

2014-12-18 Thread STINNER Victor
STINNER Victor added the comment: Is it safe to assume PyUnicode_AsUTF8() is null-terminated? Yes, Python ensures that the string is null terminated. (like PyBytes_AS_STRING() is) Yes, PyBytes_AS_STRING() also ends with a null byte. By the way, Unicode strings internally ends with a null

[issue21071] struct.Struct.format is bytes, but should be str

2014-12-18 Thread Serhiy Storchaka
Serhiy Storchaka added the comment: I think breaking the compatibility should be discussed on Python-Dev. Similar issue (and even worse) is issue8934. -- nosy: +serhiy.storchaka ___ Python tracker rep...@bugs.python.org

[issue21071] struct.Struct.format is bytes, but should be str

2014-12-18 Thread R. David Murray
R. David Murray added the comment: A backward compatibility break would certainly need to be discussed, IMO. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21071 ___

[issue21071] struct.Struct.format is bytes, but should be str

2014-12-18 Thread Arfrever Frehtes Taifersar Arahesis
Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com: -- nosy: +Arfrever ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21071 ___

[issue21071] struct.Struct.format is bytes, but should be str

2014-12-17 Thread STINNER Victor
STINNER Victor added the comment: It would be backwards incompatible to change, though. I'm in favor of breaking the compatibility with Python 3.4 and return the format as an Unicode string. -- nosy: +haypo ___ Python tracker

[issue21071] struct.Struct.format is bytes, but should be str

2014-12-17 Thread Martin Panter
Martin Panter added the comment: I originally assumed it would be a text string from the existing documentation, so changing the behaviour to match also seems reasonable -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21071

[issue21071] struct.Struct.format is bytes, but should be str

2014-12-17 Thread Martin Panter
Martin Panter added the comment: Here is a patch that changes over to a str() type. Is it safe to assume PyUnicode_AsUTF8() is null-terminated (like PyBytes_AS_STRING() is)? My documentation doesn’t say. -- Added file: http://bugs.python.org/file37488/format-str.patch

[issue21071] struct.Struct.format is bytes, but should be str

2014-12-16 Thread Martin Panter
Martin Panter added the comment: It seems to me that the simplest fix is to document: 1. Struct.format attribute is a byte string 2. The input format strings for struct.pack(), Struct class, etc, are also allowed to be byte strings, for consistency (Issue 16349) Here is a patch that does

[issue21071] struct.Struct.format is bytes, but should be str

2014-04-16 Thread Martin Panter
Martin Panter added the comment: This is closely related to Issue 16349. If format strings were explicitly allowed to be byte strings there would be less conflict, but documenting the data type of the “format” attribute is better than nothing. -- nosy: +vadmium

[issue21071] struct.Struct.format is bytes, but should be str

2014-03-26 Thread Zbyszek Jędrzejewski-Szmek
New submission from Zbyszek Jędrzejewski-Szmek: In Python 2, Struct.format used to be a str. In Python 3 it is bytes, which is unexpected. Why do I expect .format to be a string: - This format is pretty much the same as a {}-format - plain text - according to documentation it is composed of

[issue21071] struct.Struct.format is bytes, but should be str

2014-03-26 Thread Benjamin Peterson
Benjamin Peterson added the comment: I agree that's rather unfortunate. It would be backwards incompatible to change, though. -- nosy: +benjamin.peterson ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21071

[issue21071] struct.Struct.format is bytes, but should be str

2014-03-26 Thread Zbyszek Jędrzejewski-Szmek
Zbyszek Jędrzejewski-Szmek added the comment: Maybe a flag param for the constructor? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue21071 ___

[issue21071] struct.Struct.format is bytes, but should be str

2014-03-26 Thread R. David Murray
R. David Murray added the comment: I agree that the implementation does not match the documentation in this case. Especially the part about the format string used to create this Struct object. I don't see what having a flag would buy you: it doesn't help you in writing 2/3 shared code. I