[issue35714] Document that the null character '\0' terminates a struct format spec

[Mark Dickinson]

Mark Dickinson  added the comment:

Yes, this looks closeable. Thank you!

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35714] Document that the null character '\0' terminates a struct format spec

[Irit Katriel]

Irit Katriel  added the comment:

This seems resolved, can it be closed?

--
nosy: +iritkatriel

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35714] Document that the null character '\0' terminates a struct format spec

[miss-islington]

miss-islington  added the comment:


New changeset 4ea802868460fad54e40cb99eb0ca283b3b293f0 by Miss Islington (bot) 
in branch '3.7':
[3.8] bpo-35714: Reject null characters in struct format strings (GH-16928) 
(GH-20419)
https://github.com/python/cpython/commit/4ea802868460fad54e40cb99eb0ca283b3b293f0


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35714] Document that the null character '\0' terminates a struct format spec

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset 5ff5edfef63b3dbc1abb004b3fa4b3db87e79ff9 by Zackery Spytz in 
branch '3.8':
[3.8] bpo-35714: Reject null characters in struct format strings (GH-16928) 
(GH-20419)
https://github.com/python/cpython/commit/5ff5edfef63b3dbc1abb004b3fa4b3db87e79ff9


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35714] Document that the null character '\0' terminates a struct format spec

[miss-islington]

Change by miss-islington :


--
pull_requests: +19679
pull_request: https://github.com/python/cpython/pull/20420

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35714] Document that the null character '\0' terminates a struct format spec

[Zackery Spytz]

Change by Zackery Spytz :


--
pull_requests: +19678
pull_request: https://github.com/python/cpython/pull/20419

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35714] Document that the null character '\0' terminates a struct format spec

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Zackery, do you mind to create a backport to 3.8?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35714] Document that the null character '\0' terminates a struct format spec

[miss-islington]

miss-islington  added the comment:


New changeset 5221a10dde4a3853fe7ace316d95767648055109 by Miss Islington (bot) 
in branch '3.9':
bpo-35714: Reject null characters in struct format strings (GH-16928)
https://github.com/python/cpython/commit/5221a10dde4a3853fe7ace316d95767648055109


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35714] Document that the null character '\0' terminates a struct format spec

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset 3f59b55316f4c6ab451997902579aa69020b537c by Zackery Spytz in 
branch 'master':
bpo-35714: Reject null characters in struct format strings (GH-16928)
https://github.com/python/cpython/commit/3f59b55316f4c6ab451997902579aa69020b537c


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35714] Document that the null character '\0' terminates a struct format spec

[miss-islington]

Change by miss-islington :


--
pull_requests: +19638
pull_request: https://github.com/python/cpython/pull/20375

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35714] Document that the null character '\0' terminates a struct format spec

[miss-islington]

Change by miss-islington :


--
pull_requests: +19639
pull_request: https://github.com/python/cpython/pull/20376

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35714] Document that the null character '\0' terminates a struct format spec

[miss-islington]

Change by miss-islington :


--
pull_requests: +19637
pull_request: https://github.com/python/cpython/pull/20374

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35714] Document that the null character '\0' terminates a struct format spec

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 6.0 -> 7.0
pull_requests: +19636
pull_request: https://github.com/python/cpython/pull/20373

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35714] Document that the null character '\0' terminates a struct format spec

[Zackery Spytz]

Zackery Spytz  added the comment:

I've created a patch to reject null characters in the format string.

--
nosy: +ZackerySpytz

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35714] Document that the null character '\0' terminates a struct format spec

[Mark Dickinson]

Mark Dickinson  added the comment:

I agree with Serhiy. Any other unrecognised character would raise an error. The 
null character should do the same.

--
nosy: +mark.dickinson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35714] Document that the null character '\0' terminates a struct format spec

[Zackery Spytz]

Change by Zackery Spytz :


--
keywords: +patch
pull_requests: +16457
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/16928

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35714] Document that the null character '\0' terminates a struct format spec

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I think the null character is illegal character in the format string, and 
struct functions should raise a struct.error for it.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35714] Document that the null character '\0' terminates a struct format spec

[Steven D'Aprano]

Steven D'Aprano  added the comment:

I'm not sure whether having NULLs terminate a struct format string is a feature 
or a bug.

Given that nearly every other string in Python treat NULLs as ordinary 
characters, I'm inclined to say this is a bug. Or at least an unnecessary 
restriction that ought to be lifted.

--
nosy: +steven.daprano
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35714] Document that the null character '\0' terminates a struct format spec

[Dan Snider]

New submission from Dan Snider :

ie.:
>>> from struct import calcsize
>>> calcsize('\144\u0064\000xf\U0031000\60d\121\U0051')
16

I'm sure some people think it's obvious or even expect the null character to 
signal EOF but it probably isn't obvious at all to those without experience in 
lower level languages. It actually seems like Python goes out of its way to 
make sure everything treats the null character no more special than the letter 
"H", which is good.

At first glance I'd think something like this was just another trivial quirk of 
the language and not bring it up, but because the documentation doesn't mention 
it I actually got stuck on something related for half an hour when unit testing 
some dynamically generated format specs. 

Without going into unnecessary detail, what happened was that a typo in another 
tangentially related part of the test was enabling the generation of a rogue 
null byte. I'm bad at those "find face in the crowd" puzzles and this was 
hardly different, being literally camouflaged within a 300 character format 
spec containing a random mixture of escaped and non-escaped source characters 
in the forms: \U, \u, \777, \xff, \x00, + latin/ascii.

If I'm not the only one who sees this as a slightly bigger deal than poor 
documentation, the fix is trivial with an extra call to PyBytes_GET_SIZE when 
null is found. But just because I can't think of a use case in allowing the 
null character to precede other characters in the format string doesn't mean 
there isn't one, which is why only documentation is currently selected.

--
assignee: docs@python
components: Documentation, Library (Lib)
messages: 333424
nosy: bup, docs@python
priority: normal
severity: normal
status: open
title: Document that the null character '\0' terminates a struct format spec
versions: Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com