[Python-ideas] Re: Allow more flexibility for describing bytes objects.

2022-11-23 Thread Eric V. Smith via Python-ideas 

On 11/23/2022 3:02 PM, Chris Angelico wrote:

On Thu, 24 Nov 2022 at 06:46, Barry Scott  wrote:

I have written a lot of low level protocol and IOCTL calls that cannot think of 
a time this would have helped me.
struct is often a mean to create blocks of binary data.

Long strings of binary data would be a maintenance issue. What do all the bits 
mean?

Sounds like a system of compile-time evaluation - as a way to extend
constant folding to more forms - might be the way to do this, then. I
don't think the compiler can optimize this:

x = bytes.fromhex("0123456789abcdef")

and while it's theoretically possible to optimize this, it falls down
badly on the "why was this written like this?" test:

x = b"".fromhex("0123456789abcdef")

But suppose there were some way to say "this is intended to be a
compile-time constant, please evaluate it and retain the result as a
literal". Obviously this is only valid if the result is a type that
can be saved, but it would allow some quite nice optimizations that
might even include struct calls.


PEP 638 – Syntactic Macros would be perfect for this. It gives this 
example for f-strings: "The f-string |f"..."| could be implemented as 
macro as |f!("...")|. Not quite as nice to read, but would still be 
useful for experimenting with."



On the flip side, the performance gain probably wouldn't be all that
much compared to doing the work at import time. For literals used in
tight loops, there'd still be SOME benefit (if the work is done at
import time, it would have to be stored as a global, and that means
the inner loop is looking up a global rather than loading a constant),
but it would have to be a fairly hot piece of code to be worth the
effort.


That would be the beauty of 638: if it's important to one piece of code, 
it could be just enabled there. Of course the implementing macro would 
need to be either part of your application or become a dependency.


But I'm under no delusions that this PEP will ever get accepted, 
unfortunately.


Eric
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/3UANICJKEZ7FMNAXCVU7PVBLATJAFOBD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Allow more flexibility for describing bytes objects.

2022-11-23 Thread Chris Angelico 
On Thu, 24 Nov 2022 at 06:46, Barry Scott  wrote:
> I have written a lot of low level protocol and IOCTL calls that cannot think 
> of a time this would have helped me.
> struct is often a mean to create blocks of binary data.
>
> Long strings of binary data would be a maintenance issue. What do all the 
> bits mean?

Sounds like a system of compile-time evaluation - as a way to extend
constant folding to more forms - might be the way to do this, then. I
don't think the compiler can optimize this:

x = bytes.fromhex("0123456789abcdef")

and while it's theoretically possible to optimize this, it falls down
badly on the "why was this written like this?" test:

x = b"".fromhex("0123456789abcdef")

But suppose there were some way to say "this is intended to be a
compile-time constant, please evaluate it and retain the result as a
literal". Obviously this is only valid if the result is a type that
can be saved, but it would allow some quite nice optimizations that
might even include struct calls.

On the flip side, the performance gain probably wouldn't be all that
much compared to doing the work at import time. For literals used in
tight loops, there'd still be SOME benefit (if the work is done at
import time, it would have to be stored as a global, and that means
the inner loop is looking up a global rather than loading a constant),
but it would have to be a fairly hot piece of code to be worth the
effort.

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/RLW7D73GCDGFCL4UQXNOI2MAK7AWQ6QQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Allow more flexibility for describing bytes objects.

2022-11-23 Thread Barry Scott 


> On 23 Nov 2022, at 12:32, Ronald Hoogenboom via Python-ideas 
>  wrote:
> 
> Refer to PEP 3137 and PEP 358.
>  
> Bytes objects are for conveying binary data (or encoded strings). Such binary 
> data is customary specified in hex-dump or base64 format in source files.
> It would be nice to introduce a way in python to do that ‘natively’ (at 
> lexical analysis time) using x-strings and y-strings (in lieu of f-strings).
>  
> x-strings:
>  
> x”0123456789abcdef”  (x-prefixed quoted string of 
> even-numbered amount of characters of the set [0-9a-fA-F], whitespace ignored)
> equivalent to:
> b”\x01\x23\x45\x67\x89\xab\xcd\xef”  (but more readable)
>  
> y-strings:
>  
> y”ASNFZ4mrze8=” (y-prefixed quoted string of valid base64, 
> whitespace ignored)
> equivalent to:
> b”\x01\x23\x45\x67\x89\xab\xcd\xef”  (but shorter)
>  
> This is not a replacement of the hex/base64 encoding, binascii packages etc. 
> It just gives the programmer more freedom to specify literal bytes objects in 
> the source code.

How often would this be useful? If its rare and only nice to have I would doubt 
it would be added to python.

I have written a lot of low level protocol and IOCTL calls that cannot think of 
a time this would have helped me.
struct is often a mean to create blocks of binary data.

Long strings of binary data would be a maintenance issue. What do all the bits 
mean?

Barry



>  
> ___
> Python-ideas mailing list -- python-ideas@python.org 
> 
> To unsubscribe send an email to python-ideas-le...@python.org 
> 
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/VGXARYOPWYXUVGF6FA4DPMHCKIQVQF6L/
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/XOZQHUZGRVH62HHRGMJU2ZOHWD5LGJBX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Allow more flexibility for describing bytes objects.

2022-11-23 Thread Joao S. O. Bueno 
Well - one can already do:

x = bytes.fromhex
x("01 23 45 67 89")

As for base64, I really, really, don't see a use case where allowing base64
native strings as
part oif the syntax could be useful - less so when decoding is one call
away.
Base64 was already created to allow one to convey arbitrary data as strings
- let them be strings,

As for the "x" mode: I think the burden - including cognitive when teaching
Python - to include
new prefix strings easily overcome any gains there might be - again, more
so once that
a call like `bytes.frohex` is readily available.


   js
 -><-


On Wed, Nov 23, 2022 at 9:33 AM Ronald Hoogenboom via Python-ideas <
python-ideas@python.org> wrote:

> Refer to PEP 3137 and PEP 358.
>
>
>
> Bytes objects are for conveying binary data (or encoded strings). Such
> binary data is customary specified in hex-dump or base64 format in source
> files.
>
> It would be nice to introduce a way in python to do that ‘natively’ (at
> lexical analysis time) using x-strings and y-strings (in lieu of f-strings).
>
>
>
> x-strings:
>
>
>
> x”0123456789abcdef”  (x-prefixed quoted string of
> even-numbered amount of characters of the set [0-9a-fA-F], whitespace
> ignored)
>
> equivalent to:
>
> b”\x01\x23\x45\x67\x89\xab\xcd\xef”  (but more readable)
>
>
>
> y-strings:
>
>
>
> y”ASNFZ4mrze8=” (y-prefixed quoted string of valid base64,
> whitespace ignored)
>
> equivalent to:
>
> b”\x01\x23\x45\x67\x89\xab\xcd\xef”  (but shorter)
>
>
>
> This is not a replacement of the hex/base64 encoding, binascii packages
> etc. It just gives the programmer more freedom to specify literal bytes
> objects in the source code.
>
>
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/VGXARYOPWYXUVGF6FA4DPMHCKIQVQF6L/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/E7ZTS5OGHRT42TIP3WBSYTLZ6L6MPMK7/
Code of Conduct: http://python.org/psf/codeofconduct/