[issue19099] struct.pack fails first time with unicode fmt

Vajrasky Kok added the comment: Nevermind, I already created this issue. http://bugs.python.org/issue19985 -- ___ Python tracker ___ _

[issue19099] struct.pack fails first time with unicode fmt

Vajrasky Kok added the comment: Okay, I think the error message can be improved because in Python 2.7 we differentiate very clearly the string from the unicode. >>> import struct >>> struct.Struct(1) Traceback (most recent call last): File "", line 1, in TypeError: Struct() argument 1 must b

[issue19099] struct.pack fails first time with unicode fmt

Serhiy Storchaka added the comment: Fixed. Thank you Musashi for your report. -- resolution: -> fixed stage: patch review -> committed/rejected status: open -> closed ___ Python tracker ___

[issue19099] struct.pack fails first time with unicode fmt

Roundup Robot added the comment: New changeset 42d3afd29460 by Serhiy Storchaka in branch '2.7': Issue #19099: The struct module now supports Unicode format strings. http://hg.python.org/cpython/rev/42d3afd29460 -- nosy: +python-dev ___ Python tracker

[issue19099] struct.pack fails first time with unicode fmt

Serhiy Storchaka added the comment: Any comments? -- assignee: -> serhiy.storchaka ___ Python tracker ___ ___ Python-bugs-list mailin

[issue19099] struct.pack fails first time with unicode fmt

Serhiy Storchaka added the comment: > Other than that, should we raise struct.error instead of ValueError? Python 3 raises UnicodeEncodeError. And Python 2 raises UnicodeEncodeError when coerce non-ASCII unicode to str. -- ___ Python tracker

[issue19099] struct.pack fails first time with unicode fmt

Vajrasky Kok added the comment: Nevermind about my comment about clearing cache. It only happens if we use struct.pack not struct.Struct. -- ___ Python tracker ___ _

[issue19099] struct.pack fails first time with unicode fmt

Vajrasky Kok added the comment: Serhiy, you don't want to clear the cache in the test to simulate the bug? struct._clearcache() Other than that, should we raise struct.error instead of ValueError? Right now, the current behaviour in python 2.7 is: >>> struct.pack('\x80', 3) Traceback (most re

[issue19099] struct.pack fails first time with unicode fmt

Serhiy Storchaka added the comment: struct.Struct() should be changed instead of struct.pack(). Here is a patch. -- stage: -> patch review type: behavior -> enhancement Added file: http://bugs.python.org/file31931/struct_unicode_fmt.patch ___ Python

[issue19099] struct.pack fails first time with unicode fmt

Vajrasky Kok added the comment: Refactor test to clear the cache before using unicode format. -- Added file: http://bugs.python.org/file31930/handle_ascii_range_unicode_in_struct_pack_v2.patch ___ Python tracker _

[issue19099] struct.pack fails first time with unicode fmt

Musashi Tamura added the comment: Thanks for feedback. I think it should be fixed with allowing unicode. "from __future__ import unicode_literals" may break existing code. -- ___ Python tracker __

[issue19099] struct.pack fails first time with unicode fmt

Antoine Pitrou added the comment: Either way, the runtime inconsistency is a bug. Since we shouldn't break existing code, I would vote for always allowing unicode format strings, rather than always disallowing them. Another argument is that str and unicode are generally substituible in 2.x whe

[issue19099] struct.pack fails first time with unicode fmt

Serhiy Storchaka added the comment: Struct constructor accepts only str and not unicode. But struct.pack() uses caching and it found Struct('B') in the cache (because u'B' and 'B' are equal and have same hash). I doubt we should fix this. Adding support of Unicode argument is new feature. ---

[issue19099] struct.pack fails first time with unicode fmt

Vajrasky Kok added the comment: Here's the preliminary patch. I am assuming that we should accept unicode argument not reject it straight away. Python3 does that. >>> import struct >>> struct.pack('b', 3) b'\x03' >>> struct.pack(b'b', 3) b'\x03' >>> struct.pack(b'\xff', 3) Traceback (most recen

[issue19099] struct.pack fails first time with unicode fmt

New submission from Musashi Tamura: C:\>python Python 2.7.5 (default, May 15 2013, 22:44:16) [MSC v.1500 64 bit AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import struct >>> struct.pack(u'B',1) Traceback (most recent call last): File "", line 1,