[issue16349] Document whether it's safe to use bytes for struct format string

[Martin Panter]

Changes by Martin Panter :


--
dependencies: +struct.Struct.format is bytes, but should be str -Document 
whether it's safe to use bytes for struct format string

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16349] Document whether it's safe to use bytes for struct format string

[Martin Panter]

Martin Panter added the comment:

I think the direction to take for this depends on the outcome of Issue 21071. 
First we have to decide if the “format” attribute is blessed as a byte string 
(and not deprecated), or whether it is deprecated or changed to a text string.

Serhiy pointed out that it is not entirely “safe” because mixing equivalent 
byte and text formats can generate ByteWarning.

--
dependencies: +Document whether it's safe to use bytes for struct format string

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16349] Document whether it's safe to use bytes for struct format string

[Arfrever Frehtes Taifersar Arahesis]

Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com:


--
nosy: +Arfrever

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16349
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16349] Document whether it's safe to use bytes for struct format string

[Martin Panter]

Martin Panter added the comment:

Assuming it is intended to support byte strings, here is a patch that documents 
them being allowed, and adds a test case

--
keywords: +patch
Added file: http://bugs.python.org/file37489/format-bytes.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16349
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16349] Document whether it's safe to use bytes for struct format string

[Martin Panter]

Martin Panter added the comment:

Actually the “struct” module doc string seems to already hint that format 
strings can be byte strings:

“Python bytes objects are used to hold the data representing the C struct and 
also as format strings . . .”

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16349
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16349] Document whether it's safe to use bytes for struct format string

[Martin Panter]

Martin Panter added the comment:

The issue of Struct.format being a byte string has been raised separately in 
Issue 21071.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16349
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16349] Document whether it's safe to use bytes for struct format string

[Thomas Kluyver]

Thomas Kluyver added the comment:

I'm happy to put together a docs patch, but I don't have any indication of the 
right answer (is it a safe feature to use, or an implementation detail?) Is 
there another venue where I should raise the question?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16349
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16349] Document whether it's safe to use bytes for struct format string

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Python 2 supports only str. Support for unicode objects has been added in 
r59687 (merged with other unrelated changes in changeset 13aabc23cf2e). Maybe 
Raymond can explain why the type for the Struct.format was chosen bytes, not 
unicode.

--
components: +Extension Modules -Library (Lib)
nosy: +rhettinger, serhiy.storchaka
versions: +Python 3.2, Python 3.3, Python 3.4

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16349
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16349] Document whether it's safe to use bytes for struct format string

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

No, this is not r59687. I can't found from which revision in 59680-59695 it 
came.

--
nosy: +christian.heimes

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16349
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16349] Document whether it's safe to use bytes for struct format string

[Mark Dickinson]

Mark Dickinson added the comment:

 but also discovered that example outputs are wrong

That's documented to some extent:  there's a line in the docs that says:
All examples assume a native byte order, size, and alignment with a big-endian 
machine.

Given that little-endian machines are much more common that big-endian these 
days, it may be worth rewriting the examples for little-endian machines.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16349
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16349] Document whether it's safe to use bytes for struct format string

[Mark Dickinson]

Mark Dickinson added the comment:

 Also it would be nice to clarify if struct.Struct.format is meant to be  a 
 byte string.

Hmm.  That seems wrong to me.  After all, the format string is supposed to be a 
piece of human-readable text rather than a collection of bytes.  I think it's 
borderline acceptable to allow a bytes instance to be passed in for the format 
(practicality beats purity and all that), but I'd say that the output format 
should definitely be unicode.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16349
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16349] Document whether it's safe to use bytes for struct format string

[Terry J. Reedy]

Terry J. Reedy added the comment:

I think the example should be switched *and* the formats should specify the 
endianess so the examples work on all systems.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16349
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16349] Document whether it's safe to use bytes for struct format string

[Terry J. Reedy]

Terry J. Reedy added the comment:

For 3.3, I verified that adding b prefix to first three doc examples gives same 
output as without, but also discovered that example outputs are wrong, at least 
on windows, because of byte ordering issues.

 pack('hhl', 1, 2, 3)
b'\x01\x00\x02\x00\x03\x00\x00\x00'
 pack(b'hhl', 1, 2, 3)
b'\x01\x00\x02\x00\x03\x00\x00\x00'
 unpack(b'hhl', b'\x00\x01\x00\x02\x00\x00\x00\x03')
(256, 512, 50331648)
 unpack('hhl', b'\x00\x01\x00\x02\x00\x00\x00\x03')
(256, 512, 50331648)

--
nosy: +mark.dickinson, meador.inge, terry.reedy

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16349
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16349] Document whether it's safe to use bytes for struct format string

[Thomas Kluyver]

New submission from Thomas Kluyver:

At least in CPython, format strings can be given as bytes, as an alternative to 
str. E.g.

 struct.unpack(b'hhl', b'\x00\x01\x00\x02\x00\x00\x00\x03')
(1, 2, 3)

Looking at the source code [1], this appears to be consciously accounted for. 
But it doesn't seem to be mentioned in the documentation. I think the docs 
should either say it's a possibility, or warn that it's an implementation 
detail.

[1] http://hg.python.org/cpython/file/cde4b66699fe/Modules/_struct.c#l1340

--
components: Library (Lib)
messages: 174042
nosy: takluyver
priority: normal
severity: normal
status: open
title: Document whether it's safe to use bytes for struct format string

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16349
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16349] Document whether it's safe to use bytes for struct format string

[Martin Panter]

Martin Panter added the comment:

Also it would be nice to clarify if struct.Struct.format is meant to be a byte 
string. Reading the documentation and examples I expected a character string. 
It was an issue for me when embedding one structure within another:

HSF_VOL_DESC = Struct( B 5s B)

# Python 3.2.3's Struct.format is actually a byte string
NSR_DESC = Struct(HSF_VOL_DESC.format.decode() + B)

--
nosy: +vadmium

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16349
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com