[issue19099] struct.pack fails first time with unicode fmt

[Vajrasky Kok]

Vajrasky Kok added the comment:

Nevermind, I already created this issue. http://bugs.python.org/issue19985

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19099] struct.pack fails first time with unicode fmt

[Vajrasky Kok]

Vajrasky Kok added the comment:

Okay, I think the error message can be improved because in Python 2.7 we 
differentiate very clearly the string from the unicode.

>>> import struct
>>> struct.Struct(1)
Traceback (most recent call last):
  File "", line 1, in 
TypeError: Struct() argument 1 must be string, not int

But you can give unicode, right?

>>> struct.Struct(u'b')


This is consistent with other example:

>>> " cutecat ".strip(1)
Traceback (most recent call last):
  File "", line 1, in 
TypeError: strip arg must be None, str or unicode

What do you say, Serhiy?

Here is the patch.

--
Added file: 
http://bugs.python.org/file33126/fix_error_message_struct_Struct.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19099] struct.pack fails first time with unicode fmt

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Fixed. Thank you Musashi for your report.

--
resolution:  -> fixed
stage: patch review -> committed/rejected
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19099] struct.pack fails first time with unicode fmt

[Roundup Robot]

Roundup Robot added the comment:

New changeset 42d3afd29460 by Serhiy Storchaka in branch '2.7':
Issue #19099: The struct module now supports Unicode format strings.
http://hg.python.org/cpython/rev/42d3afd29460

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19099] struct.pack fails first time with unicode fmt

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Any comments?

--
assignee:  -> serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19099] struct.pack fails first time with unicode fmt

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

> Other than that, should we raise struct.error instead of ValueError?

Python 3 raises UnicodeEncodeError. And Python 2 raises UnicodeEncodeError when 
coerce non-ASCII unicode to str.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19099] struct.pack fails first time with unicode fmt

[Vajrasky Kok]

Vajrasky Kok added the comment:

Nevermind about my comment about clearing cache. It only happens if we use 
struct.pack not struct.Struct.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19099] struct.pack fails first time with unicode fmt

[Vajrasky Kok]

Vajrasky Kok added the comment:

Serhiy, you don't want to clear the cache in the test to simulate the bug?

struct._clearcache()

Other than that, should we raise struct.error instead of ValueError? Right now, 
the current behaviour in python 2.7 is:

>>> struct.pack('\x80', 3)
Traceback (most recent call last):
  File "", line 1, in 
struct.error: bad char in struct format

But you are right. Struct() should have been changed instead of struct.pack.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19099] struct.pack fails first time with unicode fmt

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

struct.Struct() should be changed instead of struct.pack(). Here is a patch.

--
stage:  -> patch review
type: behavior -> enhancement
Added file: http://bugs.python.org/file31931/struct_unicode_fmt.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19099] struct.pack fails first time with unicode fmt

[Vajrasky Kok]

Vajrasky Kok added the comment:

Refactor test to clear the cache before using unicode format.

--
Added file: 
http://bugs.python.org/file31930/handle_ascii_range_unicode_in_struct_pack_v2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19099] struct.pack fails first time with unicode fmt

[Musashi Tamura]

Musashi Tamura added the comment:

Thanks for feedback.

I think it should be fixed with allowing unicode. 
"from __future__ import unicode_literals" may break existing code.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19099] struct.pack fails first time with unicode fmt

[Antoine Pitrou]

Antoine Pitrou added the comment:

Either way, the runtime inconsistency is a bug. Since we shouldn't break 
existing code, I would vote for always allowing unicode format strings, rather 
than always disallowing them.

Another argument is that str and unicode are generally substituible in 2.x when 
they are pure ASCII (which they are here).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19099] struct.pack fails first time with unicode fmt

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Struct constructor accepts only str and not unicode. But struct.pack() uses 
caching and it found Struct('B') in the cache (because u'B' and 'B' are equal 
and have same hash).

I doubt we should fix this. Adding support of Unicode argument is new feature.

--
nosy: +mark.dickinson, meador.inge, pitrou, serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19099] struct.pack fails first time with unicode fmt

[Vajrasky Kok]

Vajrasky Kok added the comment:

Here's the preliminary patch. I am assuming that we should accept unicode 
argument not reject it straight away.

Python3 does that.
>>> import struct
>>> struct.pack('b', 3)
b'\x03'
>>> struct.pack(b'b', 3)
b'\x03'
>>> struct.pack(b'\xff', 3)
Traceback (most recent call last):
  File "", line 1, in 
struct.error: bad char in struct format

--
keywords: +patch
nosy: +vajrasky
Added file: 
http://bugs.python.org/file31922/handle_ascii_range_unicode_in_struct_pack.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19099] struct.pack fails first time with unicode fmt

[Musashi Tamura]

New submission from Musashi Tamura:

C:\>python
Python 2.7.5 (default, May 15 2013, 22:44:16) [MSC v.1500 64 bit AMD64)] on 
win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import struct
>>> struct.pack(u'B',1)
Traceback (most recent call last):
  File "", line 1, in 
TypeError: Struct() argument 1 must be string, not unicode
>>> struct.pack(u'B',1)
Traceback (most recent call last):
  File "", line 1, in 
TypeError: Struct() argument 1 must be string, not unicode
>>> struct.pack('B',1) # this is ok
'\x01'
>>> struct.pack(u'B',1)
'\x01'

--
components: Unicode
messages: 198464
nosy: ezio.melotti, miwa
priority: normal
severity: normal
status: open
title: struct.pack fails first time with unicode fmt
type: behavior
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com