[Python-Dev] Re: f-strings in the grammar

On Mon, 20 Sep 2021, 9:19 pm Pablo Galindo Salgado, wrote: > Hi, > > I have started a project to move the parsing off-strings to the parser and > the grammar. Appart > from some maintenance improvements (we can drop a considerable amount of > hand-written code), > there are some interesting

[Python-Dev] Re: f-strings in the grammar

On 9/21/2021 7:15 PM, Guido van Rossum wrote: On Tue, Sep 21, 2021 at 4:08 PM Steve Dower > wrote: On 9/21/2021 7:42 PM, Eric V. Smith wrote: > I don't recall exactly why, but I disallowed backslashes inside > expressions at the last minute before 3.6

[Python-Dev] Re: f-strings in the grammar

On Tue, Sep 21, 2021 at 4:08 PM Steve Dower wrote: > On 9/21/2021 7:42 PM, Eric V. Smith wrote: > > I don't recall exactly why, but I disallowed backslashes inside > > expressions at the last minute before 3.6 was released. It might have > > been because I was interpreting them in a way that

[Python-Dev] Re: f-strings in the grammar

On 9/21/2021 7:42 PM, Eric V. Smith wrote: I don't recall exactly why, but I disallowed backslashes inside expressions at the last minute before 3.6 was released. It might have been because I was interpreting them in a way that didn't make sense if a "real" parser were inspecting f-strings.

[Python-Dev] Re: f-strings in the grammar

On Sep 21, 2021, at 15:03, Terry Reedy wrote: > > If same-quote nesting were limited to 1 deep, REs could handle it. Since > nesting is not, and same-quote nesting would not be, they cannot in general. > > f'''length is {len(3*f"{f'{a}'}")}''' I tried this in the latest python-mode.el for

[Python-Dev] Re: f-strings in the grammar

On 9/21/2021 3:29 PM, Guido van Rossum wrote: On Tue, Sep 21, 2021 at 11:49 AM Eric V. Smith > wrote: [Pablo] * The parser will allow nesting quote characters. This means that we **could** allow reusing the same quote type in nested expressions like

[Python-Dev] Re: f-strings in the grammar

On Tue, Sep 21, 2021 at 11:49 AM Eric V. Smith wrote: > [Pablo] > > * The parser will allow nesting quote characters. This means that we > **could** allow reusing the same quote type in nested expressions > like this: > > f"some text { my_dict["string1"] } more text" > > I'm okay with this, with

[Python-Dev] Re: f-strings in the grammar

To bring this back on track, I'll try and answer the questions from your original email. On 9/20/2021 7:18 AM, Pablo Galindo Salgado wrote: I have started a project to move the parsing off-strings to the parser and the grammar. Appart from some maintenance improvements (we can drop a

[Python-Dev] Re: f-strings in the grammar

I didn't meant to bring back backticks, but to use the semantics they have in shell languages of using backslashes to escape nested substitutions, like this: f"string {code f\"string2 \{code2\} string2\" code} string" Upon reflection though, I agree that since we already use brackets which lend

[Python-Dev] Re: f-strings in the grammar

>> What do you envision tokenize.py will do with f-strings after this? It will emit new tokens: FSTRING_START FSTRING_MIDDLE '{' NAME FSTRING_FORMAT '}' FSTRING_END On Tue, 21 Sept 2021 at 12:50, Anders Munch wrote: > Pablo Galindo Salgado [mailto:pablog...@gmail.com] wrote: > > We already

[Python-Dev] Re: f-strings in the grammar

Pablo Galindo Salgado [mailto:pablog...@gmail.com] wrote: > We already expose APIs that return AST objects that can be used for all sort > of things and a tokenizer module that exposes some form of lexing that is > relatively close to the one that CPython uses internally. What do you envision

[Python-Dev] Re: f-strings in the grammar

Guido van Rossum writes: > I don't know about the line breaks, but in recent weeks I've found myself > more than once having to remind myself that inside interpolations, you must > use the other type of quote. My earlier remarks were specifically directed to line breaks. I see the point, but

[Python-Dev] Re: f-strings in the grammar

[Stephen J. Turnbull] > Is this syntax useful? Or is it just a variant of purity trying to > escape Pandora's virtualbox? I mean, am I going to see it often > enough to get used to it? Or am I going to WTF at it for the rest of > my life? > I don't know about the line breaks, but in recent

[Python-Dev] Re: f-strings in the grammar

I know I'm strongly -1 on allowing much more than currently exists for f-strings. For basically the same reason Stephen explains. Newlines inside braces, for example, go way too far away from readability. Nested expressions also feel like an attractive nuisance. I use f-strings all the time, but

[Python-Dev] Re: f-strings in the grammar

Eric V. Smith writes: > >> But this does not: > >> > >> f'{1 + > >> 2}' > > > > The later is an error with or without the 'f' prefix and I think that > > this should continue to be the case. > > > The thought is that anything that's within braces {} and is a valid > expression should

[Python-Dev] Re: f-strings in the grammar

>> But I also think this means we definitely have to get a parser module What is in this context a "parse" module? Because that will massively change depending who you ask. We already expose APIs that return AST objects that can be used for all sort of things and a tokenizer module that exposes

[Python-Dev] Re: f-strings in the grammar

On Mon, Sep 20, 2021 at 8:58 AM Thomas Grainger wrote: > I don't think the python syntax should be beholden to syntax highlighting > tools, eventually some syntax feature that PEG enables will require every > parser or highlighter to switch to a similar or more powerful parse tool > But that's

[Python-Dev] Re: f-strings in the grammar

On Mon, Sep 20, 2021 at 1:07 PM Patrick Reader <_...@pxeger.com> wrote: > > The current restrictions will also confuse some users (e.g. those used > to bash, and IIRC JS, where the rules are similar as what Pablo is > proposing). > > -- > > --Guido van Rossum (python.org/~guido

[Python-Dev] Re: f-strings in the grammar

On 20 Sep 2021, at 13:18, Pablo Galindo Salgado mailto:pablog...@gmail.com>> wrote: We are doing this work in this branch: https://github.com/we-like-parsers/cpython/blob/fstring-grammar That link is broken. Assuming you mean https://github.com/we-like-parsers/cpython/tree/fstring-grammar?

[Python-Dev] Re: f-strings in the grammar

> The current restrictions will also confuse some users (e.g. those used to > bash, and IIRC JS, where the rules are similar as what Pablo is proposing). > -- > --Guido van Rossum (python.org/~guido ) WRT the similar syntax in bash (and similar shells), there are two

[Python-Dev] Re: f-strings in the grammar

On 9/20/2021 11:48 AM, Eric V. Smith wrote: When I initially wrote f-strings, it was an explicit design goal to be just like existing strings, but with a new prefix. That's why there are all of the machinations in the parser for scanning within f-strings: the parser had already done its

[Python-Dev] Re: f-strings in the grammar

I just want to say that I am very excited to see where this goes. As an author of a package that tries to recreate compiled f-strings at runtime, they are a hard thing to generate given the current tools within Python. On Mon, Sep 20, 2021 at 4:23 AM Pablo Galindo Salgado wrote: > > Tell me

[Python-Dev] Re: f-strings in the grammar

Thanks a lot, Eric for your message! I actually share some of these worries myself and that's why I wanted to have a bigger conversation. I wanted to also make clear that the change doesn't force us to do *everything*. This means that we can absolutely have some of the improvements but not others

[Python-Dev] Re: f-strings in the grammar

The current restrictions will also confuse some users (e.g. those used to bash, and IIRC JS, where the rules are similar as what Pablo is proposing). On Mon, Sep 20, 2021 at 8:24 AM Terry Reedy wrote: > On 9/20/2021 7:18 AM, Pablo Galindo Salgado wrote: > > > there are some interesting things

[Python-Dev] Re: f-strings in the grammar

I don't think the python syntax should be beholden to syntax highlighting tools, eventually some syntax feature that PEG enables will require every parser or highlighter to switch to a similar or more powerful parse tool ___ Python-Dev mailing list --

[Python-Dev] Re: f-strings in the grammar

On 9/20/2021 11:19 AM, Terry Reedy wrote: On 9/20/2021 7:18 AM, Pablo Galindo Salgado wrote: there are some interesting things we **could** (emphasis on could) get out of this and I wanted to discuss what people think about them. * The parser will allow nesting quote characters. This means

[Python-Dev] Re: f-strings in the grammar

On 9/20/2021 11:21 AM, Terry Reedy wrote: On 9/20/2021 8:46 AM, Serhiy Storchaka wrote: 20.09.21 14:18, Pablo Galindo Salgado пише: * The parser will likely have "\n" characters and backslashes in f-strings expressions, which currently is impossible: What about characters "\x7b", "\x7d",

[Python-Dev] Re: f-strings in the grammar

On 9/20/2021 8:46 AM, Serhiy Storchaka wrote: 20.09.21 14:18, Pablo Galindo Salgado пише: * The parser will likely have "\n" characters and backslashes in f-strings expressions, which currently is impossible: What about characters "\x7b", "\x7d", "\x5c", etc? What about newlines in single

[Python-Dev] Re: f-strings in the grammar

On 9/20/2021 7:18 AM, Pablo Galindo Salgado wrote: there are some interesting things we **could** (emphasis on could) get out of this and I wanted to discuss what people think about them. * The parser will allow nesting quote characters. This means that we **could** allow reusing the same

[Python-Dev] Re: f-strings in the grammar

> > What about characters "\x7b", "\x7d", "\x5c", etc? > What about newlines in single quotes? Currently this works: This is from the current branch: >>> f"ble { '\x7b' }" 'ble {' >>> f"{1 + ... 2}" '3' >>> f'{1 + ... 2}' '3' On Mon, 20 Sept 2021 at 13:52, Serhiy Storchaka wrote: >

[Python-Dev] Re: f-strings in the grammar

20.09.21 14:18, Pablo Galindo Salgado пише: > * The parser will likely have "\n" characters and backslashes in > f-strings expressions, which currently is impossible: What about characters "\x7b", "\x7d", "\x5c", etc? What about newlines in single quotes? Currently this works: f'''{1 + 2}'''