Re: [Python-Dev] PEP 498: Literal String Interpolation is ready for pronouncement

[Eric V. Smith]

On 9/8/2015 8:27 PM, Guido van Rossum wrote:
> I'm accepting PEP 498. Congratulations Eric! And thanks to everyone who
> contributed. A lot of thought and discussion went into this -- Eric
> himself was against the idea when it first came up!

Thanks, Guido. In the next few days I'll update the implementation in
issue 24965 to reflect some changes in the PEP.

Thanks to Mike, Nick, Guido, and everyone else who helped improve the PEP.

Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 498: Literal String Interpolation is ready for pronouncement

[Guido van Rossum]

I'm accepting PEP 498. Congratulations Eric! And thanks to everyone who
contributed. A lot of thought and discussion went into this -- Eric himself
was against the idea when it first came up!

For those wondering about the fate of PEPs 501 and 502: 501 will be
deferred, and 502 will be turned into an informational PEP with background
information on string interpolation across the ages (so to speak :-).
Thanks to Nick Coghlan and Mike Miller for being good sports about it, and
for contributing valuable tire-kicking and devil's advocacy of PEP 498.

Now if only PEP 495 could be as easy... :-)

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 498: Literal String Interpolation is ready for pronouncement

[Mike Miller]

Yeah, can't wait to use it!

-Mike


On 09/08/2015 05:37 PM, Eric V. Smith wrote:

On 9/8/2015 8:27 PM, Guido van Rossum wrote:

I'm accepting PEP 498. Congratulations Eric! And thanks to everyone who
contributed. A lot of thought and discussion went into this -- Eric
himself was against the idea when it first came up!


Thanks, Guido. In the next few days I'll update the implementation in
issue 24965 to reflect some changes in the PEP.

Thanks to Mike, Nick, Guido, and everyone else who helped improve the PEP.

Eric.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 498: Literal String Interpolation is ready for pronouncement

[Guido van Rossum]

The byte code for the embedded expressions is all produced during the
regular byte code generation phase.

On Tue, Sep 8, 2015 at 3:30 AM, Erik  wrote:

> Hi.
>
> I realise I'm coming late to the party on this, but I have a question
> about something I don't think the PEP makes clear.
>
>
>
> When is the bytecode for the embedded expressions created?
>
>
>
> Is it when the source code is parsed, or at runtime when the f-string is
> evaluated?
>
> If the latter, then any process that works on bytecode (introspection
> tools, peep-hole optimisers etc) will need work.
>
> I understand that colorising editors and analysis tools which operate at
> the source code level will need work, that's fair enough.
>
>
> The PEP hints at things in this area in the "Specification", "Code
> equivalence" and "Expression evaluation" sections, and my guess is that it
> will (and should) happen during parsing, but I can't see where it comes
> right out and says it.
>
> Thanks, E.
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 498: Literal String Interpolation is ready for pronouncement

[Brett Cannon]

On Tue, 8 Sep 2015 at 10:26 Erik  wrote:

> Hi.
>
> I realise I'm coming late to the party on this, but I have a question
> about something I don't think the PEP makes clear.
>
>
>
> When is the bytecode for the embedded expressions created?


>
>
> Is it when the source code is parsed, or at runtime when the f-string is
> evaluated?
>

It's at compile time.


>
> If the latter, then any process that works on bytecode (introspection
> tools, peep-hole optimisers etc) will need work.
>
> I understand that colorising editors and analysis tools which operate at
> the source code level will need work, that's fair enough.
>
>
> The PEP hints at things in this area in the "Specification", "Code
> equivalence" and "Expression evaluation" sections, and my guess is that
> it will (and should) happen during parsing, but I can't see where it
> comes right out and says it.
>

It doesn't explicitly state it because it's inherent in the fact that it is
implemented in bytecode which is created at compile time for Python source
(typically when a module is imported, but can be when you call compile()).
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 498: Literal String Interpolation is ready for pronouncement

[Erik]

Hi.

I realise I'm coming late to the party on this, but I have a question 
about something I don't think the PEP makes clear.




When is the bytecode for the embedded expressions created?



Is it when the source code is parsed, or at runtime when the f-string is 
evaluated?


If the latter, then any process that works on bytecode (introspection 
tools, peep-hole optimisers etc) will need work.


I understand that colorising editors and analysis tools which operate at 
the source code level will need work, that's fair enough.



The PEP hints at things in this area in the "Specification", "Code 
equivalence" and "Expression evaluation" sections, and my guess is that 
it will (and should) happen during parsing, but I can't see where it 
comes right out and says it.


Thanks, E.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 498: Literal String Interpolation is ready for pronouncement

[Mike Miller]

And thanks for making it a reality.  ;)

-Mike

On 09/08/2015 05:37 PM, Eric V. Smith wrote:
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 498: Literal String Interpolation is ready for pronouncement

[Sven R. Kunze]

On 05.09.2015 12:58, Chris Angelico wrote:

On Sat, Sep 5, 2015 at 8:57 PM, Chris Angelico  wrote:

IMHO the main advantage of allowing expressions inside f-string is
that it adds something really new compared to the current str.format()
method. Without it, f-string are less interesting.

I agree with your conclusion paragraph,
and feel that they *are* interesting and useful in their
currently-described state.

By the way, I've been using this PEP to drum up some excitement about
Python 3. Still trying to get some people to make the jump from 2.7,
but if this lands, it'll be cool and interesting and fun, and maybe
some more weight on the Py3 side of the scale :)


Do you really think that matters?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 498: Literal String Interpolation is ready for pronouncement

[haypo s]

2015-09-05 10:44 GMT+02:00 haypo s :
> PEP 498 allows to write >'abc' f'string'< which is replaced with
>>'abc' 'string'.__format__()< whereas str+str is a bad practice.

Oops, you should read which is replaced with >'abc' +
'string'.__format__()< with a '+' between the two strings.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 498: Literal String Interpolation is ready for pronouncement

[Gustavo Carneiro]

On 5 September 2015 at 09:44, haypo s  wrote:

> 2015-09-05 5:01 GMT+02:00 Guido van Rossum :
> > And I'm ready to accept it. I'll wait until Tuesday night (Monday's a
> > holiday in the US) in case anything unforeseen comes up, but this is
> really
> > the Last Call for this PEP.
>
> String concatenation is inefficient in Python because strings are
> immutable. There is a micro-optimization which tried to reduce the bad
> performances of a+b, but it's better to avoid it.
>
> Python replaces >'abc' 'def'< with a single string >'abcdef'<. It's
> done by the parser, there is no overhead at runtime.
>
> PEP 498 allows to write >'abc' f'string'< which is replaced with
> >'abc' 'string'.__format__()< whereas str+str is a bad practice.
>
> I would prefer to force users to write an explicit '+' to remind them
> that there are more efficient ways to concatenate strings like
> ''.join((str1, str2)) or putting the first string in the f-string:
> >f'abcstring'<.
>
> Section in the PEP:
> https://www.python.org/dev/pepsstring/pep-0498/#concatenating-strings
>
>
That's a very good point!


> Victor
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/gjcarneiro%40gmail.com
>



-- 
Gustavo J. A. M. Carneiro
Gambit Research
"The universe is always one step beyond logic." -- Frank Herbert
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 498: Literal String Interpolation is ready for pronouncement

[haypo s]

2015-09-05 5:01 GMT+02:00 Guido van Rossum :
> And I'm ready to accept it. I'll wait until Tuesday night (Monday's a
> holiday in the US) in case anything unforeseen comes up, but this is really
> the Last Call for this PEP.

Would it be possible to specify a subset of the Python language
allowed in f-string? For example, __import__ or lambda should not be
used in a f-string. I'm not convinced that a loop or
list/dict/set-comprehension is a good idea neither.

I would prefer to keep as much code as possible *outside* f-string because:
- text editor knows well how to color it
- static analyzers know how to analyze it
- it encourage developers to indent and comment their code correctly,
whereas f-string has more restrictons on indentation (is it possible
to indent and comment code inside a f-string?)

For example, for me it's a common practice to write a complex
list-comprehension on two lines for readability:

newlist = [very_complex_expression(item)
for item in oldlist]
# sorry, it's hard to indent correctly in a mail client, especially Gmail

Well, I'm not convinced that we need a larger subset than what is
allowed currently in str.format(), simple expressions like: obj.attr,
obj[index], etc.

I recall horrible examples in the previous mail threads showing how
much complex code you can put inside f-string.

Even the following example from the PEP seems too complex to me:
print(f"Usage: {sys.argv[0]} [{'|'.join('--'+opt for opt in
valid_opts)}]", file=sys.stderr)

Oh, first I read [...] as a list-comprehension :-p But it's part of
the output string, not of the Python code...

I prefer to build the second parameter outside the f-string:
opts = '|'.join('--'+opt for opt in valid_opts)
print(f"Usage: {sys.argv[0]} [{opts}]", file=sys.stderr)

f-string with complex code remembers me PHP where it was possible to
mix PHP and HTML. I have bad memories such... code? template? (I don't
know how to cal them). Text editors and me were unable to identify
quickly the beginning and end of the code. We have a similar issue
with Python unit test embedding Python code to run subprocesses. My
text editor vim is unable to identify if the current code is the
outter code or the code embedded in a long triple-quoted string
(especially when you open the file at the embedded code).

Maybe we should start with a simple f-string, play with it during one
cycle (Python 3.6), and then discuss again if it's necessary to extend
the allowed Python expresions inside f-string?

If you really want to allow *any* Python inside a f-string, can you
please at least explain in th PEP why you consider that it's a good
thing?

IMHO the main advantage of allowing expressions inside f-string is
that it adds something really new compared to the current str.format()
method. Without it, f-string are less interesting.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 498: Literal String Interpolation is ready for pronouncement

[Chris Angelico]

On Sat, Sep 5, 2015 at 7:10 PM, haypo s  wrote:
> 2015-09-05 5:01 GMT+02:00 Guido van Rossum :
>> And I'm ready to accept it. I'll wait until Tuesday night (Monday's a
>> holiday in the US) in case anything unforeseen comes up, but this is really
>> the Last Call for this PEP.
>
> Would it be possible to specify a subset of the Python language
> allowed in f-string? For example, __import__ or lambda should not be
> used in a f-string. I'm not convinced that a loop or
> list/dict/set-comprehension is a good idea neither.

PEP 8 and other style guides will quite possibly pick up some sort of
guidelines on the subject. The way I see it, though, it's like the
difference between lambda and def - there's a technical difference
(lambda and f-strings are restricted to expressions, so if you need
statements, pull it out of line), but for the rest, it's a judgement
call  as to how big your expression should be. The interpreter doesn't
need to be involved in that decision.

> I would prefer to keep as much code as possible *outside* f-string because:
> - text editor knows well how to color it
> - static analyzers know how to analyze it
> - it encourage developers to indent and comment their code correctly,
> whereas f-string has more restrictons on indentation (is it possible
> to indent and comment code inside a f-string?)

If it can be done inside parentheses, it can be done inside an
f-string. Though I suspect multi-line expressions will generally be a
bad idea - it'll be harder to see how much text is literal and how
much is the expression. But again, question for the style guides not
the interpreter.

Text editors and static analyzers will simply need to be taught that
f-strings are special forms of expression.

> Well, I'm not convinced that we need a larger subset than what is
> allowed currently in str.format(), simple expressions like: obj.attr,
> obj[index], etc.
> ...
> IMHO the main advantage of allowing expressions inside f-string is
> that it adds something really new compared to the current str.format()
> method. Without it, f-string are less interesting.

Putting these two paragraphs together: you're not convinced we need
f-strings, is that correct? I agree with your conclusion paragraph,
and feel that they *are* interesting and useful in their
currently-described state. Having full expression syntax available is
definitely worth while; it's exactly the same benefit as being able to
use lambda inside a function call, or anything else that we can do
inline.

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 498: Literal String Interpolation is ready for pronouncement

[Chris Angelico]

On Sat, Sep 5, 2015 at 8:57 PM, Chris Angelico  wrote:
>> IMHO the main advantage of allowing expressions inside f-string is
>> that it adds something really new compared to the current str.format()
>> method. Without it, f-string are less interesting.
>
> I agree with your conclusion paragraph,
> and feel that they *are* interesting and useful in their
> currently-described state.

By the way, I've been using this PEP to drum up some excitement about
Python 3. Still trying to get some people to make the jump from 2.7,
but if this lands, it'll be cool and interesting and fun, and maybe
some more weight on the Py3 side of the scale :)

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 498: Literal String Interpolation is ready for pronouncement

[haypo s]

2015-09-05 5:01 GMT+02:00 Guido van Rossum :
> And I'm ready to accept it. I'll wait until Tuesday night (Monday's a
> holiday in the US) in case anything unforeseen comes up, but this is really
> the Last Call for this PEP.

String concatenation is inefficient in Python because strings are
immutable. There is a micro-optimization which tried to reduce the bad
performances of a+b, but it's better to avoid it.

Python replaces >'abc' 'def'< with a single string >'abcdef'<. It's
done by the parser, there is no overhead at runtime.

PEP 498 allows to write >'abc' f'string'< which is replaced with
>'abc' 'string'.__format__()< whereas str+str is a bad practice.

I would prefer to force users to write an explicit '+' to remind them
that there are more efficient ways to concatenate strings like
''.join((str1, str2)) or putting the first string in the f-string:
>f'abcstring'<.

Section in the PEP:
https://www.python.org/dev/pepsstring/pep-0498/#concatenating-strings

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 498: Literal String Interpolation is ready for pronouncement

[Nathaniel Smith]

On Sep 5, 2015 11:32 AM, "Eric V. Smith"  wrote:
>
> > Actually, my current implementation doesn't use the lexer, although I
> > suppose it could. I'm currently manually scanning the string, keeping
> > track of strings and parens. To find the end of an expression, it looks
> > for a '!', ':', or non-doubled '}', not inside of a string or (), [], or
> > {}. There's a special case for '!=' so the bang isn't seen as ending the
> > expression.
>
> Ignore the part about non-doubled '}'. The actual description is:
>
> To find the end of an expression, it looks for a '!', ':', or '}', not
> inside of a string or (), [], or {}. There's a special case for '!=' so
> the bang isn't seen as ending the expression.

Sounds like you're reimplementing a lot of the lexer... I guess that's
doable, but how confident are you that your definition of "inside a string"
matches the original in all corner cases?

In any case the abstract language definition part should be phrased in
terms of the python lexer -- the expression ends when you encounter the
first } *token* that is not nested inside () [] {} *tokens*, and then you
can implement it however makes sense...

(This is then the same rule that patsy uses to find the end of python
expressions embedded inside patsy formula strings: patsy.readthedocs.org)

-n
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 498: Literal String Interpolation is ready for pronouncement

[Victor Stinner]

> If you really want to allow *any* Python inside a f-string, can you
>> please at least explain in the PEP why you consider that it's a good
>> thing?
>>
>
> Sigh. We've gone over this on python-ideas. Your objection is not new.
>

I'm sure that later others will have exactly the same question than me.
Is''t the purpose of the PEP to summarize a discussion to not have to
repeat it again and again?

Please, can anyone summerize the discussion (on allowing or not complex
expressions in f-string) into a short paragraph in the PEP?

I don't follow python-ideas and I don't want to subscribe.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 498: Literal String Interpolation is ready for pronouncement

[Brett Cannon]

On Sat, 5 Sep 2015 at 09:19 Guido van Rossum  wrote:

> On Sat, Sep 5, 2015 at 2:10 AM, haypo s  wrote:
>
>> 2015-09-05 5:01 GMT+02:00 Guido van Rossum :
>> > And I'm ready to accept it. I'll wait until Tuesday night (Monday's a
>> > holiday in the US) in case anything unforeseen comes up, but this is
>> really
>> > the Last Call for this PEP.
>>
>> Would it be possible to specify a subset of the Python language
>> allowed in f-string? For example, __import__ or lambda should not be
>> used in a f-string. I'm not convinced that a loop or
>> list/dict/set-comprehension is a good idea neither.
>>
>
> We already went over this. You might as well argue that __import__ or
> lambda should not be used as arguments to print(). It's an expression, and
> it must allow exactly everything that is allowed in other places where
> expressions are allowed.
>
> Of course you don't *have* to write those things in f-string
> interpolations. But that's just a style guide; the language should not try
> to second-guess the user.
>

Right, it's the whole "consenting adults" bit along with trying to minimize
surprise by placing restrictions that are hard to remember. Having the
restriction of "it must be an expression" is pretty minor thing to remember
vs. "it must a specific f-string subset of Python as defined in section
N.N.N of the language spec". One of those fits in my brain and one of them
does not. ;)


>
>
>> I would prefer to keep as much code as possible *outside* f-string
>> because:
>> - text editor knows well how to color it
>> - static analyzers know how to analyze it
>> - it encourage developers to indent and comment their code correctly,
>> whereas f-string has more restrictons on indentation
>
>
> Really, we already went over most of this. You can put whitespace (even
> newlines) exactly where they are allowed in other expressions, as long as
> they don't terminate the string. You probably shouldn't do any of those
> things regularly, but there are lots of other things you that you can do in
> Python that you shouldn't.
>

As Guido said, we're consenting adults and if it makes the code unreadable
then don't do it. Just like we promote refactoring code to have more
functions to be more readable even though it adds overhead, I suspect we
will end up updating PEP 8 to say that f-strings should tend towards simple
expressions and should never compromise the readability of the code, in
which case the expressions should be hoisted out of the f-string and into
the surrounding code.

I also don't think that we should tie our hands in the design of the
language because it makes some work on behalf of the tool providers. If it
made it impossible then I think we should discuss it, but simply requiring
some more work from tools isn't a sufficient condition to restrict what we
do. That's like saying Guido shouldn't have gone with whitespace-delimited
formatting because most editors didn't support it back in the day and it
took some effort to support it since it wasn't such a simple "find the
matching curly brace" algorithm that applied to most languages.

I appreciate wanting to *promote* making code be readable, I don't think it
should come at the *cost* of the simplicity of the solution when coding
practices can easily make up for any ugliness that is possible.

-Brett


>
>>
>> (is it possible to indent and comment code inside a f-string?)
>>
>
> Now that's an interesting question. I think the answer must be No, because
> we don't want to deal with ambiguities like whether a closing curly bracket
> or string quote should be ignored inside such comments. The processing of
> f-strings described by the PEP uses several phases:
>
> - find the end of the string (the final quote[s]) using the same algorithm
> used for all string literals
> - expand \ escapes (e.g. \u)
> - look for single {, then scan ahead to a matching } -- this skips
> matching () [] {}
> - look for optional !a,!r,!s and ! inside each {...} pair
> - take what's left, enclose it in (), parse as expression
>
> The presence of comments would impede several of these stages.
>
>
>> For example, for me it's a common practice to write a complex
>> list-comprehension on two lines for readability:
>>
>> newlist = [very_complex_expression(item)
>> for item in oldlist]
>> # sorry, it's hard to indent correctly in a mail client, especially Gmail
>>
>
> The list comprehension across two lines without the comment would be fine
> in a triple-quoted f-string (or perhaps even in a single-quoted f-string if
> you put a \ before the line break).
>
>
>> Well, I'm not convinced that we need a larger subset than what is
>> allowed currently in str.format(), simple expressions like: obj.attr,
>> obj[index], etc.
>>
>
> The PEP explains why actually.
>
>
>> I recall horrible examples in the previous mail threads showing how
>> much complex code you can put inside f-string.
>>
>
> Yes, that was a very 

Re: [Python-Dev] PEP 498: Literal String Interpolation is ready for pronouncement

[Nathaniel Smith]

On Sep 5, 2015 9:20 AM, "Guido van Rossum"  wrote:
>
> The processing of f-strings described by the PEP uses several phases:
>
> - find the end of the string (the final quote[s]) using the same
algorithm used for all string literals
> - expand \ escapes (e.g. \u)

It might be worth calling out explicitly in the text how this works for
what I suspect will be the most common case: embedded quoting embedded
quotes, like

f"{ table[\"foo\"] }"

> - look for single {, then scan ahead to a matching } -- this skips
matching () [] {}

I assume this part of the algorithm uses the lexer and scans for matching
tokens rather than just scanning for characters? I tried looking at the PEP
to double check but couldn't find the relevant part. It's important for
things like

 f"{ '}' }"

> - look for optional !a,!r,!s and ! inside each {...} pair
> - take what's left, enclose it in (), parse as expression

In fact maybe this whole list that Guido wrote should be copied into the
PEP; it's a good summary :-).

-n
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 498: Literal String Interpolation is ready for pronouncement

[Terry Reedy]

On 9/5/2015 12:18 PM, Guido van Rossum wrote:

On Sat, Sep 5, 2015 at 2:10 AM, haypo s > wrote:



We already went over this. You might as well argue that __import__ or
lambda should not be used as arguments to print(). It's an expression,
and it must allow exactly everything that is allowed in other places
where expressions are allowed.



(is it possible to indent and comment code inside a f-string?)


I cannot think of any expression where indents on continuation lines 
have any significance other than being extra whitespace.  Comments are 
allowed, but cannot cause line continuation by ending with \.



(1 + #\

... 2)
3

1 + #\

  File "", line 1
1 + #\
 ^
SyntaxError: invalid syntax


Now that's an interesting question. I think the answer must be No,


This is a restriction on 'exactly everything'.  The doc should then say 
'any expression without embedded comments' (plus the usual restriction 
on not having embedded strings using the same quotes as the string itself).



because we don't want to deal with ambiguities like whether a closing
curly bracket or string quote should be ignored inside such comments.
The processing of f-strings described by the PEP uses several phases:

- find the end of the string (the final quote[s]) using the same
algorithm used for all string literals
- expand \ escapes (e.g. \u)
- look for single {, then scan ahead to a matching } -- this skips
matching () [] {}
- look for optional !a,!r,!s and ! inside each {...} pair
- take what's left, enclose it in (), parse as expression

The presence of comments would impede several of these stages.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 498: Literal String Interpolation is ready for pronouncement

[Eric V. Smith]

> Actually, my current implementation doesn't use the lexer, although I
> suppose it could. I'm currently manually scanning the string, keeping
> track of strings and parens. To find the end of an expression, it looks
> for a '!', ':', or non-doubled '}', not inside of a string or (), [], or
> {}. There's a special case for '!=' so the bang isn't seen as ending the
> expression.

Ignore the part about non-doubled '}'. The actual description is:

To find the end of an expression, it looks for a '!', ':', or '}', not
inside of a string or (), [], or {}. There's a special case for '!=' so
the bang isn't seen as ending the expression.

Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 498: Literal String Interpolation is ready for pronouncement

[Eric V. Smith]

On 9/5/2015 3:23 PM, Nathaniel Smith wrote:
> On Sep 5, 2015 11:32 AM, "Eric V. Smith"  > wrote:
>> Ignore the part about non-doubled '}'. The actual description is:
>>
>> To find the end of an expression, it looks for a '!', ':', or '}', not
>> inside of a string or (), [], or {}. There's a special case for '!=' so
>> the bang isn't seen as ending the expression.
> 
> Sounds like you're reimplementing a lot of the lexer... I guess that's
> doable, but how confident are you that your definition of "inside a
> string" matches the original in all corner cases?

Well, this is 35 lines of code (including comments), and it's much
simpler than a lexer (in the sense of "something that generates
tokens"). So I don't think I'm reimplementing a lot of the lexer.

However, your point is valid: if I don't do the same thing the lexer
would do, I could either prematurely find the end of an expression, or
look too far. In either case, when I call ast.parse() I'll get a syntax
error, and/or I'll get an error when parsing/lexing the remainder of the
string.

But it's not like I have to agree with the lexer: no larger error will
occur if I get it wrong. Everything is confined to a single f-string,
since I've already used the lexer to find the f-string in its entirety.
I only need to make sure the users understand how expressions are
extracted from f-strings.

I did look at using the actual lexer (Parser/tokenizer.c) to do this,
but it would require a large amount of surgery. I think it's overkill
for this task.

So far, I've tested it enough to have reasonable confidence that it's
correct. But the implementation could always be swapped out for an
improved version. I'm certainly open to that, if we find cases that the
simple scanner can't deal with.

> In any case the abstract language definition part should be phrased in
> terms of the python lexer -- the expression ends when you encounter the
> first } *token* that is not nested inside () [] {} *tokens*, and then
> you can implement it however makes sense...

I'm not sure that's an improvement on Guido's description when you're
trying to explain it to a user. But when time comes to write the
documentation, we can discuss it then.

> (This is then the same rule that patsy uses to find the end of python
> expressions embedded inside patsy formula strings: patsy.readthedocs.org
> )

I don't see where patsy looks for expressions in parts of strings. Let
me know if I'm missing it.

Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 498: Literal String Interpolation is ready for pronouncement

[Eric V. Smith]

On 9/5/2015 1:28 PM, Nathaniel Smith wrote:
> On Sep 5, 2015 9:20 AM, "Guido van Rossum"  > wrote:
>>
>> The processing of f-strings described by the PEP uses several phases:
>>
>> - find the end of the string (the final quote[s]) using the same
> algorithm used for all string literals
>> - expand \ escapes (e.g. \u)
> 
> It might be worth calling out explicitly in the text how this works for
> what I suspect will be the most common case: embedded quoting embedded
> quotes, like
> 
> f"{ table[\"foo\"] }"
> 
>> - look for single {, then scan ahead to a matching } -- this skips
> matching () [] {}
> 
> I assume this part of the algorithm uses the lexer and scans for
> matching tokens rather than just scanning for characters? I tried
> looking at the PEP to double check but couldn't find the relevant part.
> It's important for things like
> 
>  f"{ '}' }"

Actually, my current implementation doesn't use the lexer, although I
suppose it could. I'm currently manually scanning the string, keeping
track of strings and parens. To find the end of an expression, it looks
for a '!', ':', or non-doubled '}', not inside of a string or (), [], or
{}. There's a special case for '!=' so the bang isn't seen as ending the
expression.

So it does work on this f-string:

>>> f"{ '}' }"
'}'

Although by using the lexer, I'd be worried about multiple levels of
escaping. I'd have to give it some thought.


>> - look for optional !a,!r,!s and ! inside each {...} pair
>> - take what's left, enclose it in (), parse as expression
> 
> In fact maybe this whole list that Guido wrote should be copied into the
> PEP; it's a good summary :-).

I think the PEP does say these things, but maybe it could be tightened up.

Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 498: Literal String Interpolation is ready for pronouncement

[Eric V. Smith]

On 9/5/2015 1:14 PM, Gustavo Carneiro wrote:
> Why not allow string concatenation without plus sign only if/when the
> implementation becomes optimised to allow compile time concatenation? 
> This makes it crystal clear whether the concatenation is compile time or
> runtime. If we allow it now, it's hard to tell without looking at the
> CPython source code...

Although you can make it faster than '+', it's not possible for it to be
a compile-time concatenation. The PEP calls out that it's an expression
that's evaluated at runtime.

I think having strings concatenated with adjacent f-strings follows the
principle of least surprise. I also think it won't get used all that
often, just as compile-time string concatenation isn't seen with great
frequency.

Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 498: Literal String Interpolation is ready for pronouncement

[Eric V. Smith]

On 9/5/2015 12:18 PM, Guido van Rossum wrote:
> On Sat, Sep 5, 2015 at 2:10 AM, haypo s  > wrote:
>
> (is it possible to indent and comment code inside a f-string?)
> 
> 
> Now that's an interesting question. I think the answer must be No,
> because we don't want to deal with ambiguities like whether a closing
> curly bracket or string quote should be ignored inside such comments.
> The processing of f-strings described by the PEP uses several phases:

I'll update the PEP to say comments aren't allowed. It gets especially
wacky when adding () around the expression: f'{2#}' would become the
expression "(2#)", which is a syntax error.

Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 498: Literal String Interpolation is ready for pronouncement

[Guido van Rossum]

On Sat, Sep 5, 2015 at 2:10 AM, haypo s  wrote:

> 2015-09-05 5:01 GMT+02:00 Guido van Rossum :
> > And I'm ready to accept it. I'll wait until Tuesday night (Monday's a
> > holiday in the US) in case anything unforeseen comes up, but this is
> really
> > the Last Call for this PEP.
>
> Would it be possible to specify a subset of the Python language
> allowed in f-string? For example, __import__ or lambda should not be
> used in a f-string. I'm not convinced that a loop or
> list/dict/set-comprehension is a good idea neither.
>

We already went over this. You might as well argue that __import__ or
lambda should not be used as arguments to print(). It's an expression, and
it must allow exactly everything that is allowed in other places where
expressions are allowed.

Of course you don't *have* to write those things in f-string
interpolations. But that's just a style guide; the language should not try
to second-guess the user.


> I would prefer to keep as much code as possible *outside* f-string because:
> - text editor knows well how to color it
> - static analyzers know how to analyze it
> - it encourage developers to indent and comment their code correctly,
> whereas f-string has more restrictons on indentation


Really, we already went over most of this. You can put whitespace (even
newlines) exactly where they are allowed in other expressions, as long as
they don't terminate the string. You probably shouldn't do any of those
things regularly, but there are lots of other things you that you can do in
Python that you shouldn't.

>
> (is it possible to indent and comment code inside a f-string?)
>

Now that's an interesting question. I think the answer must be No, because
we don't want to deal with ambiguities like whether a closing curly bracket
or string quote should be ignored inside such comments. The processing of
f-strings described by the PEP uses several phases:

- find the end of the string (the final quote[s]) using the same algorithm
used for all string literals
- expand \ escapes (e.g. \u)
- look for single {, then scan ahead to a matching } -- this skips matching
() [] {}
- look for optional !a,!r,!s and ! inside each {...} pair
- take what's left, enclose it in (), parse as expression

The presence of comments would impede several of these stages.


> For example, for me it's a common practice to write a complex
> list-comprehension on two lines for readability:
>
> newlist = [very_complex_expression(item)
> for item in oldlist]
> # sorry, it's hard to indent correctly in a mail client, especially Gmail
>

The list comprehension across two lines without the comment would be fine
in a triple-quoted f-string (or perhaps even in a single-quoted f-string if
you put a \ before the line break).


> Well, I'm not convinced that we need a larger subset than what is
> allowed currently in str.format(), simple expressions like: obj.attr,
> obj[index], etc.
>

The PEP explains why actually.


> I recall horrible examples in the previous mail threads showing how
> much complex code you can put inside f-string.
>

Yes, that was a very poorly thought out argument. All of Python should be
condemned if you took it seriously.


> Even the following example from the PEP seems too complex to me:
> print(f"Usage: {sys.argv[0]} [{'|'.join('--'+opt for opt in
> valid_opts)}]", file=sys.stderr)
>
> Oh, first I read [...] as a list-comprehension :-p But it's part of
> the output string, not of the Python code...
>
> I prefer to build the second parameter outside the f-string:
> opts = '|'.join('--'+opt for opt in valid_opts)
> print(f"Usage: {sys.argv[0]} [{opts}]", file=sys.stderr)
>

The same criticism can be made for the original version (using .format()),
which was lifted from real code.


> f-string with complex code remembers me PHP where it was possible to
> mix PHP and HTML. I have bad memories such... code? template? (I don't
> know how to cal them). Text editors and me were unable to identify
> quickly the beginning and end of the code. We have a similar issue
> with Python unit test embedding Python code to run subprocesses. My
> text editor vim is unable to identify if the current code is the
> outter code or the code embedded in a long triple-quoted string
> (especially when you open the file at the embedded code).
>

Yeah, tools will need to be updated. There are several other languages
(e.g. Ruby) that allow arbitrary expressions in strings and at least some
editors have no problem with them, even vim.


> Maybe we should start with a simple f-string, play with it during one
> cycle (Python 3.6), and then discuss again if it's necessary to extend
> the allowed Python expresions inside f-string?
>
> If you really want to allow *any* Python inside a f-string, can you
> please at least explain in the PEP why you consider that it's a good
> thing?
>

Sigh. We've gone over this on python-ideas. Your objection is not new.


> IMHO the main 

Re: [Python-Dev] PEP 498: Literal String Interpolation is ready for pronouncement

[Guido van Rossum]

On Sat, Sep 5, 2015 at 1:44 AM, haypo s  wrote:

> 2015-09-05 5:01 GMT+02:00 Guido van Rossum :
> > And I'm ready to accept it. I'll wait until Tuesday night (Monday's a
> > holiday in the US) in case anything unforeseen comes up, but this is
> really
> > the Last Call for this PEP.
>
> String concatenation is inefficient in Python because strings are
> immutable. There is a micro-optimization which tried to reduce the bad
> performances of a+b, but it's better to avoid it.
>
> Python replaces >'abc' 'def'< with a single string >'abcdef'<. It's
> done by the parser, there is no overhead at runtime.
>
> PEP 498 allows to write >'abc' f'string'< which is replaced with
> >'abc' 'string'.__format__()< whereas str+str is a bad practice.
>
> I would prefer to force users to write an explicit '+' to remind them
> that there are more efficient ways to concatenate strings like
> ''.join((str1, str2)) or putting the first string in the f-string:
> >f'abcstring'<.
>
> Section in the PEP:
> https://www.python.org/dev/pepsstring/pep-0498/#concatenating-strings
>
> Victor
>

What does the extra + buy users? It's just more punctuation noise. The
implementation is intentionally left unspecified by the PEP; it should be
possible to design an implementation that combines all the parts together
more efficiently than the use of + (e.g. using a "string builder" object
internally).

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 498: Literal String Interpolation is ready for pronouncement

[Barry Warsaw]

On Sep 04, 2015, at 08:01 PM, Guido van Rossum wrote:

>And I'm ready to accept it.

As I've mentioned to Eric, I think it's a great PEP, so congratulations on
getting one through The Gauntlet.  I know that in my own code, I'm looking
forward to using it in however many years it takes for 3.6 to be the base
compatible version of course. :)

I'm okay with it not being appropriate for i18n (IMHO) but no worries,
string.Template and libraries like mine built on top of it all do the trick
nicely and are quite stable.  No reason for them to go away, and PEP 498 is
orthogonal to that.

Still, I read *a lot* of Python code written by other people.  Most of it is
really good (it's nice that there are so many excellent Python developers out
there :).  I also know that the syntax is going to be abused and I rather
dread having to read, comprehend, and debug it.  But oh well.

In another thread it was suggested that a PEP 8 -type of best practices be
included in new PEPs.  I definitely think that any documentation written for
f-strings should include such best practices.  Maybe PEP 8 should also include
some style suggestions or admonitions for f-strings, but I'll let others
propose language for that.

Cheers,
-Barry
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 498: Literal String Interpolation is ready for pronouncement

[Eric V. Smith]

On 9/5/2015 7:12 PM, Nathaniel Smith wrote:
> On Sat, Sep 5, 2015 at 1:00 PM, Eric V. Smith  wrote:
>> On 9/5/2015 3:23 PM, Nathaniel Smith wrote:
>>> On Sep 5, 2015 11:32 AM, "Eric V. Smith" >> > wrote:
 Ignore the part about non-doubled '}'. The actual description is:

 To find the end of an expression, it looks for a '!', ':', or '}', not
 inside of a string or (), [], or {}. There's a special case for '!=' so
 the bang isn't seen as ending the expression.
>>>
>>> Sounds like you're reimplementing a lot of the lexer... I guess that's
>>> doable, but how confident are you that your definition of "inside a
>>> string" matches the original in all corner cases?
>>
>> Well, this is 35 lines of code (including comments), and it's much
>> simpler than a lexer (in the sense of "something that generates
>> tokens"). So I don't think I'm reimplementing a lot of the lexer.
>>
>> However, your point is valid: if I don't do the same thing the lexer
>> would do, I could either prematurely find the end of an expression, or
>> look too far. In either case, when I call ast.parse() I'll get a syntax
>> error, and/or I'll get an error when parsing/lexing the remainder of the
>> string.
>>
>> But it's not like I have to agree with the lexer: no larger error will
>> occur if I get it wrong. Everything is confined to a single f-string,
>> since I've already used the lexer to find the f-string in its entirety.
>> I only need to make sure the users understand how expressions are
>> extracted from f-strings.
>>
>> I did look at using the actual lexer (Parser/tokenizer.c) to do this,
>> but it would require a large amount of surgery. I think it's overkill
>> for this task.
>>
>> So far, I've tested it enough to have reasonable confidence that it's
>> correct. But the implementation could always be swapped out for an
>> improved version. I'm certainly open to that, if we find cases that the
>> simple scanner can't deal with.
>>
>>> In any case the abstract language definition part should be phrased in
>>> terms of the python lexer -- the expression ends when you encounter the
>>> first } *token* that is not nested inside () [] {} *tokens*, and then
>>> you can implement it however makes sense...
>>
>> I'm not sure that's an improvement on Guido's description when you're
>> trying to explain it to a user. But when time comes to write the
>> documentation, we can discuss it then.
> 
> I'm not talking about end-user documentation, I'm talking about the
> formal specification, like in the Python Language Reference.

I think the formal specification can talk about scanning a string
looking for the end of an expression without discussing tokens. Just as
https://docs.python.org/3/reference/lexical_analysis.html#string-and-bytes-literals
uses an EBNF sort of language.

The implementation is not tokenizing anything: it's just trying to find
the end of the string in order to pass it to ast.parse(). That task is
significantly easier than tokenizing.

> I'm pretty sure that just calling the tokenizer will be easier for
> Cython or PyPy than implementing a special purpose scanner :-)

I sincerely doubt that, but I'd be curious how they implemented
_string.formatter_parser(), which is extremely close in functionality.
Sadly, just not close enough to be reusable for this.

But I'm not going to argue about implementation when the PEP hasn't been
accepted yet. Especially for code I'm not going to write in Cython and PyPy!

>>> (This is then the same rule that patsy uses to find the end of python
>>> expressions embedded inside patsy formula strings: patsy.readthedocs.org
>>> )
>>
>> I don't see where patsy looks for expressions in parts of strings. Let
>> me know if I'm missing it.
> 
> Patsy parses strings like
> 
>"np.sin(a + b) + c"
> 
> using a grammar that supports some basic arithmetic-like infix
> operations (+, *, parentheses, etc.), and in which the atoms are
> arbitrary Python expressions. So the above string is parsed into a
> patsy-AST that looks something like:
> 
>   Add(PyExpr("np.sin(a + b)"), PyExpr("c"))
> 
> The rule it uses to do this is that it uses the Python tokenizer,
> counts nesting of () [] {}, and when it sees a valid unnested patsy
> operator, then that's the end of the embedded expression:
> 
>   https://github.com/pydata/patsy/blob/master/patsy/parse_formula.py#L37
> 
> Not tremendously relevant, but that's why I've thought this through before :-)

For the actual parsing of the expressions, I use the equivalent of
ast.parse(). So that phase of the processing I'm not worried about.

Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 498: Literal String Interpolation is ready for pronouncement

[Guido van Rossum]

On Sat, Sep 5, 2015 at 2:22 PM, Victor Stinner 
wrote:

>
> If you really want to allow *any* Python inside a f-string, can you
>>> please at least explain in the PEP why you consider that it's a good
>>> thing?
>>>
>>
>> Sigh. We've gone over this on python-ideas. Your objection is not new.
>>
>
> I'm sure that later others will have exactly the same question than me.
> Is''t the purpose of the PEP to summarize a discussion to not have to
> repeat it again and again?
>
> Please, can anyone summerize the discussion (on allowing or not complex
> expressions in f-string) into a short paragraph in the PEP?
>
> I don't follow python-ideas and I don't want to subscribe.
>

But you can at least read the PEP. There is a whole section "Supporting
full Python expressions" which explains this.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 498: Literal String Interpolation is ready for pronouncement

[Nathaniel Smith]

On Sat, Sep 5, 2015 at 1:00 PM, Eric V. Smith  wrote:
> On 9/5/2015 3:23 PM, Nathaniel Smith wrote:
>> On Sep 5, 2015 11:32 AM, "Eric V. Smith" > > wrote:
>>> Ignore the part about non-doubled '}'. The actual description is:
>>>
>>> To find the end of an expression, it looks for a '!', ':', or '}', not
>>> inside of a string or (), [], or {}. There's a special case for '!=' so
>>> the bang isn't seen as ending the expression.
>>
>> Sounds like you're reimplementing a lot of the lexer... I guess that's
>> doable, but how confident are you that your definition of "inside a
>> string" matches the original in all corner cases?
>
> Well, this is 35 lines of code (including comments), and it's much
> simpler than a lexer (in the sense of "something that generates
> tokens"). So I don't think I'm reimplementing a lot of the lexer.
>
> However, your point is valid: if I don't do the same thing the lexer
> would do, I could either prematurely find the end of an expression, or
> look too far. In either case, when I call ast.parse() I'll get a syntax
> error, and/or I'll get an error when parsing/lexing the remainder of the
> string.
>
> But it's not like I have to agree with the lexer: no larger error will
> occur if I get it wrong. Everything is confined to a single f-string,
> since I've already used the lexer to find the f-string in its entirety.
> I only need to make sure the users understand how expressions are
> extracted from f-strings.
>
> I did look at using the actual lexer (Parser/tokenizer.c) to do this,
> but it would require a large amount of surgery. I think it's overkill
> for this task.
>
> So far, I've tested it enough to have reasonable confidence that it's
> correct. But the implementation could always be swapped out for an
> improved version. I'm certainly open to that, if we find cases that the
> simple scanner can't deal with.
>
>> In any case the abstract language definition part should be phrased in
>> terms of the python lexer -- the expression ends when you encounter the
>> first } *token* that is not nested inside () [] {} *tokens*, and then
>> you can implement it however makes sense...
>
> I'm not sure that's an improvement on Guido's description when you're
> trying to explain it to a user. But when time comes to write the
> documentation, we can discuss it then.

I'm not talking about end-user documentation, I'm talking about the
formal specification, like in the Python Language Reference.

I'm pretty sure that just calling the tokenizer will be easier for
Cython or PyPy than implementing a special purpose scanner :-)

>> (This is then the same rule that patsy uses to find the end of python
>> expressions embedded inside patsy formula strings: patsy.readthedocs.org
>> )
>
> I don't see where patsy looks for expressions in parts of strings. Let
> me know if I'm missing it.

Patsy parses strings like

   "np.sin(a + b) + c"

using a grammar that supports some basic arithmetic-like infix
operations (+, *, parentheses, etc.), and in which the atoms are
arbitrary Python expressions. So the above string is parsed into a
patsy-AST that looks something like:

  Add(PyExpr("np.sin(a + b)"), PyExpr("c"))

The rule it uses to do this is that it uses the Python tokenizer,
counts nesting of () [] {}, and when it sees a valid unnested patsy
operator, then that's the end of the embedded expression:

  https://github.com/pydata/patsy/blob/master/patsy/parse_formula.py#L37

Not tremendously relevant, but that's why I've thought this through before :-)

-n

-- 
Nathaniel J. Smith -- http://vorpus.org
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 498: Literal String Interpolation is ready for pronouncement

[Gustavo Carneiro]

Why not allow string concatenation without plus sign only if/when the
implementation becomes optimised to allow compile time concatenation?  This
makes it crystal clear whether the concatenation is compile time or
runtime. If we allow it now, it's hard to tell without looking at the
CPython source code...

On Sat, 5 Sep 2015 16:57 Guido van Rossum  wrote:

On Sat, Sep 5, 2015 at 1:44 AM, haypo s  wrote:

2015-09-05 5:01 GMT+02:00 Guido van Rossum :
> And I'm ready to accept it. I'll wait until Tuesday night (Monday's a
> holiday in the US) in case anything unforeseen comes up, but this is
really
> the Last Call for this PEP.

String concatenation is inefficient in Python because strings are
immutable. There is a micro-optimization which tried to reduce the bad
performances of a+b, but it's better to avoid it.

Python replaces >'abc' 'def'< with a single string >'abcdef'<. It's
done by the parser, there is no overhead at runtime.

PEP 498 allows to write >'abc' f'string'< which is replaced with
>'abc' 'string'.__format__()< whereas str+str is a bad practice.

I would prefer to force users to write an explicit '+' to remind them
that there are more efficient ways to concatenate strings like
''.join((str1, str2)) or putting the first string in the f-string:
>f'abcstring'<.

Section in the PEP:
https
://

www.python.org
/dev/

pepsstring

/pep-0498/#concatenating-strings


Victor

What does the extra + buy users? It's just more punctuation noise. The
implementation is intentionally left unspecified by the PEP; it should be
possible to design an implementation that combines all the parts together
more efficiently than the use of + (e.g. using a "string builder" object
internally).


-- 

--Guido van Rossum (python.org /~
guido )

___
Python-Dev mailing list
Python-Dev@python.org
https ://
mail.python.org
/mailman/
listinfo
/
python-dev

Unsubscribe: https

://

mail.python.org

/mailman/options/

python-dev
/

gjcarneiro%40gmail.com

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PEP 498: Literal String Interpolation is ready for pronouncement

[Eric V. Smith]

I've made a number of small changes to PEP 498. I don't think any of the
changes I've made in the last week are substantive. Mostly I've
clarified how it works and removing some limitations. The only
meaningful change is that expressions are now surrounded by parens
before they're evaluated. This allows for newlines inside of the
expressions to be ignored.

I think it's now ready for Guido to pronounce on it.

The code in issue 24965 is slightly out of date, in that it has some
limitations that are no longer mentioned in the PEP and doesn't enclose
the expression in parens before evaluation. If the PEP is accepted, I'll
fix that and update the issue.

-- 
Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 498: Literal String Interpolation is ready for pronouncement

[Nikolaus Rath]

On Sep 04 2015, "Eric V. Smith"  wrote:
> I've made a number of small changes to PEP 498. I don't think any of the
> changes I've made in the last week are substantive. Mostly I've
> clarified how it works and removing some limitations. The only
> meaningful change is that expressions are now surrounded by parens
> before they're evaluated. This allows for newlines inside of the
> expressions to be ignored.
>
> I think it's now ready for Guido to pronounce on it.

I just wanted to say that I think this is fantastic work. If the PEP
gets accepted, I expect that in my code f-strings will replace the other
interpolation mechanism in almost all cases.


Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

 »Time flies like an arrow, fruit flies like a Banana.«
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 498: Literal String Interpolation is ready for pronouncement

[Guido van Rossum]

On Fri, Sep 4, 2015 at 6:45 PM, Eric V. Smith  wrote:

> I've made a number of small changes to PEP 498. I don't think any of the
> changes I've made in the last week are substantive. Mostly I've
> clarified how it works and removing some limitations. The only
> meaningful change is that expressions are now surrounded by parens
> before they're evaluated. This allows for newlines inside of the
> expressions to be ignored.
>
> I think it's now ready for Guido to pronounce on it.
>

And I'm ready to accept it. I'll wait until Tuesday night (Monday's a
holiday in the US) in case anything unforeseen comes up, but this is really
the Last Call for this PEP.


> The code in issue 24965 is slightly out of date, in that it has some
> limitations that are no longer mentioned in the PEP and doesn't enclose
> the expression in parens before evaluation. If the PEP is accepted, I'll
> fix that and update the issue.


No worries. (The parens thing was something I suggested to Eric this
afternoon. :-)

I'll say a few words about PEPs 501 and 502, but I'll start separate
threads.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com