Re: [Python-ideas] Let’s make escaping in f-literals impossible
On Tue, Aug 30, 2016 at 10:30 AM, Steven D'Aprano wrote:
[...]
> That looks like you are doing a key lookup on bar:
>
> bar = {'\n': 'something'}
> f'foo{ bar['\n'] }baz'
>
> looks like it will return 'foosomethingbaz'. I expect that syntax will
> confuse an awful lot of people.
Can we please stop debating this? This observation has been made about
100 times by now.
--
--Guido van Rossum (python.org/~guido)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On Tue, Aug 30, 2016 at 11:02:25AM +, Philipp A. wrote:
> Hi Eric,
>
> Very cool of you to get this going!
>
> I hope the outcome is to ban escapes within braced code parts of f-literals
> for 3.6 and add them “the right way” in 3.7: f'foo{ bar['\n'] }baz'
That looks like you are doing a key lookup on bar:
bar = {'\n': 'something'}
f'foo{ bar['\n'] }baz'
looks like it will return 'foosomethingbaz'. I expect that syntax will
confuse an awful lot of people.
--
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
You need to write a new PEP. And it's PEP, not RFC. On Tue, Aug 30, 2016 at 9:56 AM, Philipp A. wrote: > Hi Guido, thanks for calling me out. > > Yikes, I'm terribly sorry that it came over that way! > > I'll write the RFC. Should I expand the existing one (this would need Chris’ > pending changes though) or write a new one? > > My goals were to sound factual and terse, not to insult anyone. And I don't > see the flaws in my phrasing, so it seems I'm still sometimes bad at written > communication. > > @everyone who perceived it as Guido did: It would be really nice if you > could pinpoint the phrases and reasons that make it seem that I mean it that > way. (In a private mail to me) > > Best, Philipp > > > Guido van Rossum schrieb am Di., 30. Aug. 2016, 18:43: >> >> Philipp, you need to stop debating this issue *now*. >> >> You need to write a PEP that can go into Python 3.7. Further debate at >> the current level (a hair-width close to name-calling) is not going to >> sway anyone. >> >> (This actually goes for Chris too -- nothing is obviously going to >> change Philipp's mind, so you might as well stop debating and save all >> the onlookers the embarrassment.) >> >> -- >> --Guido van Rossum (python.org/~guido) -- --Guido van Rossum (python.org/~guido) ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
Hi Guido, thanks for calling me out. Yikes, I'm terribly sorry that it came over that way! I'll write the RFC. Should I expand the existing one (this would need Chris’ pending changes though) or write a new one? My goals were to sound factual and terse, not to insult anyone. And I don't see the flaws in my phrasing, so it seems I'm still sometimes bad at written communication. @everyone who perceived it as Guido did: It would be really nice if you could pinpoint the phrases and reasons that make it seem that I mean it that way. (In a private mail to me) Best, Philipp Guido van Rossum schrieb am Di., 30. Aug. 2016, 18:43: > Philipp, you need to stop debating this issue *now*. > > You need to write a PEP that can go into Python 3.7. Further debate at > the current level (a hair-width close to name-calling) is not going to > sway anyone. > > (This actually goes for Chris too -- nothing is obviously going to > change Philipp's mind, so you might as well stop debating and save all > the onlookers the embarrassment.) > > -- > --Guido van Rossum (python.org/~guido) > ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
Philipp, you need to stop debating this issue *now*. You need to write a PEP that can go into Python 3.7. Further debate at the current level (a hair-width close to name-calling) is not going to sway anyone. (This actually goes for Chris too -- nothing is obviously going to change Philipp's mind, so you might as well stop debating and save all the onlookers the embarrassment.) -- --Guido van Rossum (python.org/~guido) ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On Tue, Aug 30, 2016 at 11:43:03PM +1000, Chris Angelico wrote:
> On Tue, Aug 30, 2016 at 10:56 PM, Philipp A. wrote:
> > My issue is just that it’s as much of a string as a call of a (string
> > returning) function/method or an expression concatenating strings:
> >
> > ''.join(things) # would you call this a string?
> > '{!r}'.format(x) # or this? it’s basically the same as this “f-string”:
> > f'{x!r}'
> > 'start' + 'end' # or this? it’s a concatenation of two strings, just like
> > f'start{ "end" }'
>
> Yes, an f-string is really a form of expression, not a literal. But
> prior art has generally been to have similar constructs referred to as
> "interpolated strings" or similar terms:
>
> https://en.wikipedia.org/wiki/String_interpolation
*shrug* A misleading name is misleading no matter how many people use
it. If started calling
function calls:
func(arg, x, y, z+1)
"s-strings" for "source code strings", because you write them as source
code, and changed the syntax to
func"arg, x, y, z+1"
we'd all recognise what a mistake it is to call this a string, string
delimiters or not. The *result* of calling it may be a string, but the
expression itself is a kind of function call.
[nasty thought]
Why don't we allow func"*args" as syntactic sugar for str(func(*args))?
[/remove tongue from cheek]
> Plenty of languages have some such feature, and it's usually
> considered a type of string.
The result is a type of string. The expression is not.
> Notice the close parallels between actual
> string literals used as format strings ("I have %d apples" % apples)
> and f-strings (f"I have {apples} apples"),
That's a misleading comparison. The *template string* is "I have %d
apples" and that is just a string. The *format operator* is % and the
formatting operation or expression is the entire expression.
Since f-strings can contain arbitrary expressions, the analogy is not
with the template string, but a function (method or operator) call:
f"I have {fruit - bananas - oranges} apples"
is not equivalent to "I have {} apples", but to
"I have {} apples".format(fruit - bananas - oranges)
which is clearly a method call that merely returns a string, not a
string itself.
Consequently, there's no way to delay evaluation of such an f-string,
any more than you can delay evaluation of the expression[1]:
func(fruit, 2*bananas, 3/oranges)
You can't generate a template, and pass it to different environments to
be formatted. If you need to evaluate f"I have {apples} apples" in six
different contexts, you have to repeat yourself: it is equivalent to a
function call, complete with implicit arguments, not a function object,
and certainly not equivalent to a template string.
> and how this same parallel
> can be seen in many other languages. Yes, it may be a join expression
> to the compiler, but it's a string to the programmer.
Only if the programmer is fooled by the mere use of string delimiters.
It really isn't a string. It is executable code that returns a string.
It is basically a distant cousin to eval(), in disguise.
I really wish we had a better name for these things than f-strings :-(
[1] Tricks with eval() excluded.
--
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
Please just call it f-string and move on, we've had the naming debate
previously, it's no longer productive.
Regarding eventually supporting f'{'x'}', that will have to be a new
PEP to extend PEP 498. (I previously thought it would be an
incompatibility, but since f'{' is currently invalid, it's not.
However it's a huge change conceptually and implementation-wise, and I
don't blame Eric if he doesn't want to be responsible for it. So it
has to be a new PEP, to be introduced in 3.7 at the earliest.
--
--Guido van Rossum (python.org/~guido)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
Sorry, but I'm afraid you are projecting your thinking onto others.
The syntactical constructs are called “string interpolations”, not
“interpolated strings”. I.e. they're interpolations (a certain type of
action) on strings.
Strings are the objects, not the subjects. Strings are data, we have
code/expressions that look like strings with holes, but in reality, only
the parts outside of the braces are strings.
I hope I explained my semantics here adequately. Even if they're internally
post-processed strings in the CPython code: that's an implementation
detail, not a description of the way they work for Python users.
Best, Philipp
Chris Angelico schrieb am Di., 30. Aug. 2016, 15:43:
> On Tue, Aug 30, 2016 at 10:56 PM, Philipp A. wrote:
> > My issue is just that it’s as much of a string as a call of a (string
> > returning) function/method or an expression concatenating strings:
> >
> > ''.join(things) # would you call this a string?
> > '{!r}'.format(x) # or this? it’s basically the same as this “f-string”:
> > f'{x!r}'
> > 'start' + 'end' # or this? it’s a concatenation of two strings, just
> like
> > f'start{ "end" }'
>
> Yes, an f-string is really a form of expression, not a literal. But
> prior art has generally been to have similar constructs referred to as
> "interpolated strings" or similar terms:
>
> https://en.wikipedia.org/wiki/String_interpolation
>
> Plenty of languages have some such feature, and it's usually
> considered a type of string. Notice the close parallels between actual
> string literals used as format strings ("I have %d apples" % apples)
> and f-strings (f"I have {apples} apples"), and how this same parallel
> can be seen in many other languages. Yes, it may be a join expression
> to the compiler, but it's a string to the programmer.
>
> ChrisA
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On Tue, Aug 30, 2016 at 10:56 PM, Philipp A. wrote:
> My issue is just that it’s as much of a string as a call of a (string
> returning) function/method or an expression concatenating strings:
>
> ''.join(things) # would you call this a string?
> '{!r}'.format(x) # or this? it’s basically the same as this “f-string”:
> f'{x!r}'
> 'start' + 'end' # or this? it’s a concatenation of two strings, just like
> f'start{ "end" }'
Yes, an f-string is really a form of expression, not a literal. But
prior art has generally been to have similar constructs referred to as
"interpolated strings" or similar terms:
https://en.wikipedia.org/wiki/String_interpolation
Plenty of languages have some such feature, and it's usually
considered a type of string. Notice the close parallels between actual
string literals used as format strings ("I have %d apples" % apples)
and f-strings (f"I have {apples} apples"), and how this same parallel
can be seen in many other languages. Yes, it may be a join expression
to the compiler, but it's a string to the programmer.
ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
Hi Eric,
Very cool of you to get this going!
I hope the outcome is to ban escapes within braced code parts of f-literals
for 3.6 and add them “the right way” in 3.7: f'foo{ bar['\n'] }baz'
It really is how things work in every single language that i ever
encountered that has template literals / string interpolation / f-literals
/ whatchacallit, so in order to be logical and non-surprising, we should do
it that way (eventually).
Also the name “f-strings” is really misleading: They’re composite
expressions that evaluate to strings. They can only be considered
strings if you have no braced code parts in them. So I’m also still in
favor of renaming them (e.g. to “f-literals”).
Best, Philipp
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
Eric V. Smith schrieb am Di., 30. Aug. 2016 um
14:13 Uhr:
> There's debate on if that's the right way, and I personally think it's
> probably not. Personally, I'd be happy with the only change being to not
> allow backslashes inside braces. But that's not an argument that I'm
> willing to get into now, since I need to get this rolling for beta 1.
And exactly that’s why i’m very happy with the way things are going:
Banning escapes is the right short term solution, and possibly the best
compromise if we can’t find consensus on how escapes should behave inside
of those braces.
אלעזר schrieb am Di., 30. Aug. 2016 um 14:37 Uhr:
> The problem is that "literal" is a technical term in the domain of
> compilers, and not a well known or self-explanatory term. Especially for
> beginners. I'd suggest something like "Format expression".
>
That sounds fine as well!
My issue is just that it’s as much of a string as a call of a (string
returning) function/method or an expression concatenating strings:
''.join(things) # would you call this a string?
'{!r}'.format(x) # or this? it’s basically the same as this “f-string”:
f'{x!r}'
'start' + 'end' # or this? it’s a concatenation of two strings, just like
f'start{ "end" }'
Best, Philipp
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
> On Tue, Aug 30, 2016 at 3:13 PM Eric V. Smith wrote: > >On 08/30/2016 07:02 AM, Philipp A. wrote: > > Also the name “f-strings” is really misleading: They’re composite > > expressions that evaluate to strings. They can only be considered > > strings if you have no braced code parts in them. So I’m also still in > > favor of renaming them (e.g. to “f-literals”). > I don't have much of an opinion here. I think there's not a lot of > confusion to be had by calling them f-strings, but I think someone who > works with teaching python might have a better handle on that. The problem is that "literal" is a technical term in the domain of compilers, and not a well known or self-explanatory term. Especially for beginners. I'd suggest something like "Format expression". ~Elazar ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On 08/30/2016 07:02 AM, Philipp A. wrote:
> Very cool of you to get this going!
Thanks for raising the issue.
> I hope the outcome is to ban escapes within braced code parts of
> f-literals for 3.6 and add them “the right way” in 3.7: f'foo{ bar['\n']
> }baz'
There's debate on if that's the right way, and I personally think it's
probably not. Personally, I'd be happy with the only change being to not
allow backslashes inside braces. But that's not an argument that I'm
willing to get into now, since I need to get this rolling for beta 1.
> Also the name “f-strings” is really misleading: They’re composite
> expressions that evaluate to strings. They can only be considered
> strings if you have no braced code parts in them. So I’m also still in
> favor of renaming them (e.g. to “f-literals”).
I don't have much of an opinion here. I think there's not a lot of
confusion to be had by calling them f-strings, but I think someone who
works with teaching python might have a better handle on that.
Eric.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On 8/29/2016 5:33 PM, Eric V. Smith wrote: On 8/29/2016 5:26 PM, Ethan Furman wrote: On 08/29/2016 02:16 PM, Eric V. Smith wrote: I've been looking at this, and I agree it's the best thing to do, for now (and possibly forever). I'm just not convinced I can get it done before alpha 1. Isn't the f-string feature already in place? Yes. It's been in 3.6 for quite a while (maybe a year?). Update the PEP, then it's a bugfix. ;) You can do bug fixes during the beta series. Heh. I guess that's true. But it's sort of a big change, so shipping beta 1 with the code not agreeing with the PEP rubs me the wrong way. Or, I could stop worrying and typing emails, and instead just get on with it! Eric. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/ -- Terry Jan Reedy ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On 30 August 2016 at 07:40, Steve Dower wrote: > On 29Aug2016 1433, Eric V. Smith wrote: >> >> On 8/29/2016 5:26 PM, Ethan Furman wrote: >>> >>> Update the PEP, then it's a bugfix. ;) >> >> >> Heh. I guess that's true. But it's sort of a big change, so shipping >> beta 1 with the code not agreeing with the PEP rubs me the wrong way. >> >> Or, I could stop worrying and typing emails, and instead just get on >> with it! > > > I like this approach :) It would be good to update the PEP to say "No backslash escapes allowed inside braces" and file a bug against 3.6 for allowing it, though :) > But I agree. Release Manager Ned has the final say, but I think this change > can comfortably go in during the beta period. (I also disagree that it's a > big change - nobody could agree on the 'obvious' behaviour of backslashes > anyway, so chances are people would avoid them anyway, and there was strong > consensus on advising people to avoid them.) +1 - the beta deadline is "no new features", rather than "no further changes to features we already added". The beta period wouldn't be very useful if we couldn't make changes to new features based on user feedback :) (e.g. PEP 492's native coroutines needed some fairly major surgery during the 3.5 beta, as the Cython and Tornado folks found that some of the design decisions we'd made in the initial object model were major barriers to interoperability with other event loops and third party coroutine implementations) Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On 29Aug2016 1433, Eric V. Smith wrote: On 8/29/2016 5:26 PM, Ethan Furman wrote: Update the PEP, then it's a bugfix. ;) Heh. I guess that's true. But it's sort of a big change, so shipping beta 1 with the code not agreeing with the PEP rubs me the wrong way. Or, I could stop worrying and typing emails, and instead just get on with it! I like this approach :) But I agree. Release Manager Ned has the final say, but I think this change can comfortably go in during the beta period. (I also disagree that it's a big change - nobody could agree on the 'obvious' behaviour of backslashes anyway, so chances are people would avoid them anyway, and there was strong consensus on advising people to avoid them.) Cheers, Steve ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On 8/29/2016 5:40 PM, Steve Dower wrote: On 29Aug2016 1433, Eric V. Smith wrote: On 8/29/2016 5:26 PM, Ethan Furman wrote: Update the PEP, then it's a bugfix. ;) Heh. I guess that's true. But it's sort of a big change, so shipping beta 1 with the code not agreeing with the PEP rubs me the wrong way. Or, I could stop worrying and typing emails, and instead just get on with it! I like this approach :) But I agree. Release Manager Ned has the final say, but I think this change can comfortably go in during the beta period. (I also disagree that it's a big change - nobody could agree on the 'obvious' behaviour of backslashes anyway, so chances are people would avoid them anyway, and there was strong consensus on advising people to avoid them.) By "big", I meant "a lot of C code changes". And you'd be surprised by the percentage of the tests that are devoted to backslashes inside braces! Eric. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On 8/29/2016 5:26 PM, Ethan Furman wrote: On 08/29/2016 02:16 PM, Eric V. Smith wrote: I've been looking at this, and I agree it's the best thing to do, for now (and possibly forever). I'm just not convinced I can get it done before alpha 1. Isn't the f-string feature already in place? Yes. It's been in 3.6 for quite a while (maybe a year?). Update the PEP, then it's a bugfix. ;) Heh. I guess that's true. But it's sort of a big change, so shipping beta 1 with the code not agreeing with the PEP rubs me the wrong way. Or, I could stop worrying and typing emails, and instead just get on with it! Eric. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On 08/29/2016 02:16 PM, Eric V. Smith wrote: I've been looking at this, and I agree it's the best thing to do, for now (and possibly forever). I'm just not convinced I can get it done before alpha 1. Isn't the f-string feature already in place? Update the PEP, then it's a bugfix. ;) -- ~Ethan~ ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
Oops, I meant beta 1 where I said alpha 1. Eric. On 8/29/2016 5:12 PM, Eric V. Smith wrote: On 8/23/2016 8:18 AM, Nick Coghlan wrote: On 21 August 2016 at 03:32, Eric V. Smith wrote: If anything, I'd make it an error to have any backslashes inside the brackets of an f-string for 3.6. We could always remove this restriction at a later date. +1 for this if you can find a way to do it - it eliminates the problematic cases where the order of evaluation makes a difference, and ensures the parts within the braces can be reliably processed as normal Python code. I've been looking at this, and I agree it's the best thing to do, for now (and possibly forever). I'm just not convinced I can get it done before alpha 1. Assuming I can get the coding done, I think I should update PEP 498 to say there can be no backslashes inside the curly braces. That's my preferred outcome. If I can't get it done by alpha 1, then I think the options are: 1. Leave f-strings as they are now, and that's how they'll always be. 2. Leave f-strings as they are now, but mark them as provisional and warn people that the backslash restrictions will show up in an upcoming release. 3. Disallow any backslashes anywhere in f-strings for 3.6, and relax the restriction in 3.7 to make it only inside braces where the restriction is enforced. 4. Remove f-strings from 3.6, and add them in 3.7 with the "no backslash inside braces" restriction. I'm not wild about 2: people will ignore this and will write code that will break in 3.7. I'm also not wild about 3, since it's too restrictive. I'm open to suggestions. Eric. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On 8/23/2016 8:18 AM, Nick Coghlan wrote: On 21 August 2016 at 03:32, Eric V. Smith wrote: If anything, I'd make it an error to have any backslashes inside the brackets of an f-string for 3.6. We could always remove this restriction at a later date. +1 for this if you can find a way to do it - it eliminates the problematic cases where the order of evaluation makes a difference, and ensures the parts within the braces can be reliably processed as normal Python code. I've been looking at this, and I agree it's the best thing to do, for now (and possibly forever). I'm just not convinced I can get it done before alpha 1. Assuming I can get the coding done, I think I should update PEP 498 to say there can be no backslashes inside the curly braces. That's my preferred outcome. If I can't get it done by alpha 1, then I think the options are: 1. Leave f-strings as they are now, and that's how they'll always be. 2. Leave f-strings as they are now, but mark them as provisional and warn people that the backslash restrictions will show up in an upcoming release. 3. Disallow any backslashes anywhere in f-strings for 3.6, and relax the restriction in 3.7 to make it only inside braces where the restriction is enforced. 4. Remove f-strings from 3.6, and add them in 3.7 with the "no backslash inside braces" restriction. I'm not wild about 2: people will ignore this and will write code that will break in 3.7. I'm also not wild about 3, since it's too restrictive. I'm open to suggestions. Eric. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On 21 August 2016 at 03:32, Eric V. Smith wrote: > If anything, I'd make it an error to have any backslashes inside the > brackets of an f-string for 3.6. We could always remove this restriction at > a later date. +1 for this if you can find a way to do it - it eliminates the problematic cases where the order of evaluation makes a difference, and ensures the parts within the braces can be reliably processed as normal Python code. > In any event, I'll take a look at adding this restriction, just to get an > estimate of the magnitude of work involved. The easiest thing to do might be > to disallow backslashes in any part of an f-string for 3.6, although that > seems to be going too far. Disallowing \t, \n, etc even in the plain text parts of the f-string would indeed be problematic. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
Sorry for replying to late, i had an email issue.
First two important things: 1. mental model and intuition and 2.
precendence.
About how to think of them: I’m strongly of the opinion that the mental
models of either an alternating sequence of strings and formatted
expressions, or a string with “holes” for expressions, are better than “a
string where parts are magically evaluated after it’s created”. That smells
like “eval” instead of the usual separation of data and code. That’s the
same reason I dislike calling them “f-strings”: they’re undoubtedly
**expressions
evaluating to strings**, not string literals.
Precedence exists in ruby’s and CoffeeScript’s string interpolation, Bash’s
$(), JavaScript’s template literals, and many more:
https://en.wikipedia.org/wiki/String_interpolation
!!! *All* of them that support arbitrary code (And that I ever encountered)
work the way I propose for python to work
@Brett Cannon: Pragmatism is only good as long as it only compromises on
elegance, not usability. I think “partly evauable f-strings” are harder to
use than “f-literals with string and expression parts”
@Chris Angelo: All those things being illegal is surprising, which is
another argument in favor of my proposal.
@Guido van Rossum I’d rather not like them to be preliminarily in the
language in this form, considering Python’s track record of not changing
preliminary things anymore…but: @Eriv V. Smith: Great idea with banning all
backslashes for now. This is so close to release, so we could ban escape
sequences and use all of the existing code, then write a new RFC to make
sure things are optimal (which in my eyes means the holes/alternating
sequence model instead of the thing we have now)
Thank you all for your contributions to the discussion and again sorry for
messing up and only now posting this correctly.
Best, Philipp
Chris Angelico schrieb am So., 21. Aug. 2016 um
09:57 Uhr:
> On Sun, Aug 21, 2016 at 5:51 PM, Franklin? Lee
> wrote:
> > Speaking of which, how is this parsed?
> > f"{'\n'}"
> > If escape-handling is done first, the expression is a string literal
> holding
> > an actual newline character (normally illegal), rather than an escape
> > sequence which resolves to a newline character.
>
> It's illegal.
>
> > If that one somehow works, how about this?
> > f"{r'\n'}"
>
> Also illegal.
>
> > I guess you'd have to write one of these:
> > f"{'\\n'}"
> > f"{'''\n''')"
> > rf"{'\n'}"
>
> Modulo the typo in the second one, these all result in the same code:
>
> >>> dis.dis(lambda: f"{'\\n'}")
> 1 0 LOAD_CONST 1 ('\n')
> 2 FORMAT_VALUE 0
> 4 RETURN_VALUE
> >>> f"{'\\n'}"
> '\n'
>
> ChrisA
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On Sun, Aug 21, 2016 at 5:51 PM, Franklin? Lee
wrote:
> Speaking of which, how is this parsed?
> f"{'\n'}"
> If escape-handling is done first, the expression is a string literal holding
> an actual newline character (normally illegal), rather than an escape
> sequence which resolves to a newline character.
It's illegal.
> If that one somehow works, how about this?
> f"{r'\n'}"
Also illegal.
> I guess you'd have to write one of these:
> f"{'\\n'}"
> f"{'''\n''')"
> rf"{'\n'}"
Modulo the typo in the second one, these all result in the same code:
>>> dis.dis(lambda: f"{'\\n'}")
1 0 LOAD_CONST 1 ('\n')
2 FORMAT_VALUE 0
4 RETURN_VALUE
>>> f"{'\\n'}"
'\n'
ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On Aug 20, 2016 1:32 PM, "Eric V. Smith" wrote:
>
> On 8/19/2016 2:57 PM, Guido van Rossum wrote:
>>
>> I don't think we should take action now.
>>
>> Would it make sense, as a precaution, to declare the PEP provisional for
>> one release? Then we can develop a sense of whether the current approach
>> causes real problems.
>>
>> We could also emit some kind of warning if the expression part contains
>> an escaped quote, since that's where a potential change would cause
>> breakage. (Or we could leave that to the linters.)
>
>
> If anything, I'd make it an error to have any backslashes inside the
brackets of an f-string for 3.6. We could always remove this restriction at
a later date.
>
> I say this because as soon as f-strings make it into the wild, we're
going to have a hard time breaking code in say 3.7 by saying "well, we told
you that f-strings might change".
>
> Although frankly, other than be executive fiat (which I'm okay with), I
don't see us ever resolving the issue if f-strings are strings first, or if
the brackets put you into "non-string mode". There are good arguments on
both sides.
>
> Moving to the implementation details, I'm not sure how easy it would be
to even find backslashes, though. IIRC, backslashes are replaced early,
before the f-string parser really gets to process the string. It might
require a new implementation of the f-string parser independent of regular
strings, which I likely would not have time for before beta 1. Although
since this would be a reduction in functionality, maybe it doesn't have to
get done by then.
>
> I also haven't thought of how this would affect raw f-strings.
>
> In any event, I'll take a look at adding this restriction, just to get an
estimate of the magnitude of work involved. The easiest thing to do might
be to disallow backslashes in any part of an f-string for 3.6, although
that seems to be going too far.
>
>
> Eric.
Speaking of which, how is this parsed?
f"{'\n'}"
If escape-handling is done first, the expression is a string literal
holding an actual newline character (normally illegal), rather than an
escape sequence which resolves to a newline character.
If that one somehow works, how about this?
f"{r'\n'}"
I guess you'd have to write one of these:
f"{'\\n'}"
f"{'''\n''')"
rf"{'\n'}"
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On 8/19/2016 2:57 PM, Guido van Rossum wrote: I don't think we should take action now. Would it make sense, as a precaution, to declare the PEP provisional for one release? Then we can develop a sense of whether the current approach causes real problems. We could also emit some kind of warning if the expression part contains an escaped quote, since that's where a potential change would cause breakage. (Or we could leave that to the linters.) If anything, I'd make it an error to have any backslashes inside the brackets of an f-string for 3.6. We could always remove this restriction at a later date. I say this because as soon as f-strings make it into the wild, we're going to have a hard time breaking code in say 3.7 by saying "well, we told you that f-strings might change". Although frankly, other than be executive fiat (which I'm okay with), I don't see us ever resolving the issue if f-strings are strings first, or if the brackets put you into "non-string mode". There are good arguments on both sides. Moving to the implementation details, I'm not sure how easy it would be to even find backslashes, though. IIRC, backslashes are replaced early, before the f-string parser really gets to process the string. It might require a new implementation of the f-string parser independent of regular strings, which I likely would not have time for before beta 1. Although since this would be a reduction in functionality, maybe it doesn't have to get done by then. I also haven't thought of how this would affect raw f-strings. In any event, I'll take a look at adding this restriction, just to get an estimate of the magnitude of work involved. The easiest thing to do might be to disallow backslashes in any part of an f-string for 3.6, although that seems to be going too far. Eric. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On 20 August 2016 at 05:02, Random832 wrote:
> On Fri, Aug 19, 2016, at 19:09, Paul Moore wrote:
>> So, to me
>>
>> f'{x.partition(' + ')[0]}'
>>
>> reads as a string concatenation. I'm not sure how you'd expect a
>> syntax highlighter to make it look like anything else, to be honest
>
> One possible syntax highlighting scheme:
Thanks for the detailed explanation and example. Yes, that may well be
a reasonable highlighting scheme.
I'd still object to reusing single quotes in the example given,
though, as it would be confusing if printed, or in email, etc. And as
a general principle, "needs syntax highlighting to be readable" is a
problem to me. So I stand by my statement that as a style rule,
f-strings should be written to work identically regardless of whether
this proposal is implemented or not.
Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On 20 August 2016 at 04:57, C Anthony Risinger wrote: > The two string parts are string-colored and the x.partition bits would look > like > any other code in the file. It won't look like concatenation at that point. That's entirely subjective and theoretical (unless you've implemented it and reviewed the resulting look of the code). In my (equally subjective and theoretical) opinion it would still look like concatenation, and would confuse me. I made a deliberate point of saying that *to me* it looked like concatenation. YMMV - remember this tangent was started by people stating their opinions. Saying that your opinion differs doesn't invalidate their (my) view. > tl;dr, UX is weaker when the editor implies a free-form expression in every > possible way, but the writer can't use the quote they always use, and I > think strings will be common in f-string expression sections. FWIW I would instantly reject any code passed to me for review which used the same quote within an f-string as was used to delimit it, should this proposal be accepted. Also, a lot of code is read on media that doesn't do syntax highlighting - email, books, etc. A construct that needs syntax highlighting to be readable is problematic because of this. Paul ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
C Anthony Risinger writes: > The only time I personally use a different quote is when it somehow > makes the data more amenable to the task at hand. The data! The > literal data! Not the expressions I'm conveniently inlining with > the help of f-strings. You do *not* know that yet! *Nobody* does. Nobody has yet written an f-string in production code, let alone read thousands and written hundreds. Can you be sure that after you write a couple dozen f-strings you won't find that such "quote context" is carried over naturally from the way you write other strings? (Eg, because "I'm still in a string" is signaled by the highlighting of the surrounding stringish text.) I think the proposed changes to the PEP fall into the "Sometimes never is better than *right* now" category. The arguments I've seen so far are plausible but not founded in experience: it could easily go the other way, and I don't see potential for true disaster. > If I have to water it down for people to find it acceptable (such > as creating simpler variables ahead-of-time) I'd probably just keep > using .format(...). Because what I have gained with an f-string? I don't see a problem if you choose not to write f-strings. Would other people using that convention be hard for you to *read*? > Not just because it's at odds with other languages, but because > it's at odds with what the editor is telling the user (free-form > expression). There are no editors that will tell you such a thing yet. And if you trust an editor that *does* tell you that it's a free-form expression and use the same kind of quote that delimits the f-string, you won't actually create a syntax error. You're merely subject to the same kind of "problem" that you have if you don't write PEP8 conforming code. Regards, ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On 2016-08-19 20:57, C Anthony Risinger wrote:
In a quality editor, everything about the {...} will tell me I'm writing a
Python expression. It'll be colored like an expression. It'll do fancy
completion like an expression. Aw shucks, it*IS* a Python expression!
Except for one tiny detail: I'm not allowed to use the quote I use in 95%
of all my Python code--without thinking--because I already used it at the
string start :-( It's like this weird invisible ruh-roh-still-in-a-string
state hangs over you despite everything else suggesting otherwise
But it IS inside a string. That's why it's an f-string. The essence
of your argument seems to be that you want expressions inside f-strings
to act just like expressions outside of f-strings. But there's already
a way to do that: just write the expression outside of the f-string.
Then you can assign it to a variable, and refer to the variable in the
f-string. The whole point of f-strings is that they allow expressions
*inside strings*. It doesn't make sense to pretend those expressions
are not inside strings. It's true that the string itself "isn't really
a string" in the sense that it's put together at runtime rather than
being a constant, but again, the point of f-strings is to make things
like that writable as strings in source code. If you don't want to
write them as strings, you can still concatenate separate string values
or use various other solutions.
--
Brendan Barnwell
"Do not follow where the path may lead. Go, instead, where there is no
path, and leave a trail."
--author unknown
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On 2016-08-19 13:11, C Anthony Risinger wrote: It might be harder to find the end of an f-string in one shot, but I think that's the crux of the issue: to a reader/developer, is an f-string conceptually one thing or a compound thing? To me (someone who would like to see f-string expressions appear like normal expressions, without extra quoting, and proper syntax highlighting *always*, just like shell), this argument is essentially the same as trying to use a regex to find a closing bracket or brace or parse HTML. It's only hard (disregarding any underlying impl details) because that view regards f-strings as singular things with only one "end", when in reality an f-string is much much more like a compound expression that just happens to look like a string. Personally I think that is a dangerous road to go down. It seems it would lead to the practice of doing all sorts of complicated things inside f-strings, which seems like a bad idea to me. In principle you could write your entire program in an f-string, but that doesn't mean we need to accommodate the sort of syntax highlighting that would facilitate that. To me it seems more prudent to just say that f-strings are (as the name implies) strings, and leave it at that. If I ever get to the point where what I'm doing in the f-string is so complicated that I really need syntax highlighting for it to look good, I'd take that as a sign that I should move some of that code out of the f-string into ordinary expressions. -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On 08/19/2016 08:57 PM, C Anthony Risinger wrote: [...] The appeal of f-strings is the rapid inlining of whatever plus string data. "Whatever" is typically more complex than a simple attribute access or variable reference, though not much more complex eg. `object.method(key, "default")`. If I have to water it down for people to find it acceptable (such as creating simpler variables ahead-of-time) I'd probably just keep using .format(...). Because what I have gained with an f-string? I suspect f-strings are in the same category as lambda -- if it's that complex, use the other tools instead. At this point I don't see this changing. If you want to make any headway you're going to have to do it with a complete alternate implementation, and even then I don't think you have good odds. -- ~Ethan~ ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On Fri, Aug 19, 2016, at 19:09, Paul Moore wrote:
> So, to me
>
> f'{x.partition(' + ')[0]}'
>
> reads as a string concatenation. I'm not sure how you'd expect a
> syntax highlighter to make it look like anything else, to be honest
One possible syntax highlighting scheme:
- f' and ' are hilighted in pink, along with any plain text content of
the string.
- - Incidentally, any backslash escapes, not shown here, are highlighted
in orange.
- { and } are hilighted in blue; along with format specifiers, maybe, or
maybe they get another color.
- The code inside the expression is highlighted in orange.
- Any keywords, builtins, constants, etc, within the expression are
highlighted in their usual colors.
- - In this example in particular, ' + ' and 0 are highlighted in pink.
A pink + is a character within a string, a gray or orange + is an
operator.
In terms of Vim's basic syntax groups:
- C = Constant [pink]
- P = PreProc [blue], by precedent as use for the $(...)
delimiters in sh)
- S = Special [orange], by precedent as use for the content within
$(...) in sh, and longstanding near-universal precedent, including in
python, for backslash escapes. These would, naturally, have separate
non-basic highlight groups, in case a particular user wanted to change
one of them.
f'foo {x.partition(' + ')[0]:aaa} bar\n'
CCPCSSCSPSSC
> (given that you're arguing *not* to highlight the whole of the content
> of the f-string the same way).
I'm not sure what you mean by "the same way" here, I haven't followed
the discussion closely enough to know what statement by whom you're
referring to here.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On Fri, Aug 19, 2016 at 6:09 PM, Paul Moore wrote:
> On 19 August 2016 at 21:50, C Anthony Risinger wrote:
> > The only real point I'm trying to make is that expressions within an
> > f-string are an *escape*. They escape the normal semantics of a string
> > literal and instead do something else for a while. Therefore, the escaped
> > sections should not look like (or need to conform to) the rest of the
> string
> > and they should not require quoting as if it were still within the
> string,
> > because I escaped it already!
>
> So, to me
>
> f'{x.partition(' + ')[0]}'
>
> reads as a string concatenation. I'm not sure how you'd expect a
> syntax highlighter to make it look like anything else, to be honest
> (given that you're arguing *not* to highlight the whole of the content
> of the f-string the same way).
>
The two string parts are string-colored and the x.partition bits would look
like any other code in the file. It won't look like concatenation at that
point. Examples referencing f'{one_expr_and_no_real_string_in_here}' feel
somewhat crafted to confuse because the start and end quotes are directly
adjacent to the expression. str(...) is the same complexity. Usage in the
wild will have plenty of string-stuff on one or both sides, otherwise, why?
Shell or Ruby code is probably more representative of how f-strings will be
used.
I know a couple people have mentioned they won't/don't care about
highlighting in an f-string, but I honestly don't know a single person that
would prefer this, except maybe one devops guy I know that does everything
on old-school green text because why not. I've also spent hours and hours
staring at--and sometimes editing--code on barebones servers/containers and
I've come to respect the role colors play in my ability to quickly scan and
read code.
> The *real* solution is not to write something like this, instead write
>
> f"{x.partition(' + ')[0]}"
>
Why? Why should I have to care what kind of quote I used at the start of
the string? I thought I "escaped" the string at the `{` and now my brain
has moved on to the expression? Am I still "inside" the string? etc...
It's not the highlighting I care about per se, I think we have a small UX
failure here.
In a quality editor, everything about the {...} will tell me I'm writing a
Python expression. It'll be colored like an expression. It'll do fancy
completion like an expression. Aw shucks, it *IS* a Python expression!
Except for one tiny detail: I'm not allowed to use the quote I use in 95%
of all my Python code--without thinking--because I already used it at the
string start :-( It's like this weird invisible ruh-roh-still-in-a-string
state hangs over you despite everything else suggesting otherwise
(highlighting and whatever fanciness helps people output code).
The only time I personally use a different quote is when it somehow makes
the data more amenable to the task at hand. The data! The literal data! Not
the expressions I'm conveniently inlining with the help of f-strings. When
I do it's a conscious decision and comes with a reason. Otherwise I'll use
one type of quote exclusively (which depends on the lang, but more and
more, it's simply doubles).
The appeal of f-strings is the rapid inlining of whatever plus string data.
"Whatever" is typically more complex than a simple attribute access or
variable reference, though not much more complex eg. `object.method(key,
"default")`. If I have to water it down for people to find it acceptable
(such as creating simpler variables ahead-of-time) I'd probably just keep
using .format(...). Because what I have gained with an f-string?
The problem I have is the very idea that while inlining expressions I'm
still somehow inside the string, and I have to think about that. It's not a
*huge* overhead for an experienced, most-handsome developer such as myself,
but still falls in that 5% territory (using a quote because I must vs. the
one used 95%). Since f-string are fabulous, I want to use them all the
time! Alas, now I have to think about flip-flopping quotes. I don't know
what it's like to be taught programming but this seems like a possible
negative interaction for new people (learning and using [one] quote
combined with easy string building).
I know it's not *that* big of deal to switch quotes. I believe this simpler
implementation out the gate (not a dig! still great!) will come at the cost
of introducing a small Python oddity requiring explanation. Not just
because it's at odds with other languages, but because it's at odds with
what the editor is telling the user (free-form expression).
tl;dr, UX is weaker when the editor implies a free-form expression in every
possible way, but the writer can't use the quote they always use, and I
think strings will be common in f-string expression sections.
--
C Anthony
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Condu
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On 19 August 2016 at 21:50, C Anthony Risinger wrote:
> The only real point I'm trying to make is that expressions within an
> f-string are an *escape*. They escape the normal semantics of a string
> literal and instead do something else for a while. Therefore, the escaped
> sections should not look like (or need to conform to) the rest of the string
> and they should not require quoting as if it were still within the string,
> because I escaped it already!
So, to me
f'{x.partition(' + ')[0]}'
reads as a string concatenation. I'm not sure how you'd expect a
syntax highlighter to make it look like anything else, to be honest
(given that you're arguing *not* to highlight the whole of the content
of the f-string the same way).
The *real* solution is not to write something like this, instead write
f"{x.partition(' + ')[0]}"
That makes it easy for *humans* to read. Computers parsing it is
irrelevant. Once you do that, the proposal here (that unescaped quotes
can be used in an f-string) also becomes irrelevant - this expression
parses exactly the same way under both the current code and the
proposed approach. And that's generally true - code that is clearly
written should, in my mind, work the same way regardless. So the
proposal ends up being merely "choose your preference as to which form
of badly-written code is a syntax error".
So the only relevance of syntax highlighting is how badly it fails
when handling badly-written or syntactically incorrect code. And
detecting an f-string just like you detect any other string, is *much*
better behaved in that situation. Detecting a closing quote is simple,
and isn't thrown off by incorrect nesting. If you want the *content*
of an f-string to be highlighted as an expression, Vim can do that, it
can apply specific syntax when in another syntax group such as an
f-string, and I'm sure other editors can do this as well - but you
should do that once you've determined where the f-string starts and
ends.
Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
Ok. My .vimrc is probably different from yours. I'm sure you are right I *could* make that happen. But I haven't so far. On Aug 19, 2016 1:50 PM, "C Anthony Risinger" wrote: > On Fri, Aug 19, 2016 at 3:39 PM, David Mertz wrote: > >> I don't think I've ever used a syntax highlighter than changed color of >> \n in a string. I get the concept, but I haven't suffered for the absence >> of that. >> >> Moreover, although I haven't yet used them, I really doubt I want extra >> syntax highlighting in f-strings beyond simply the color strings appear as. >> Well, maybe a uniform distinction for f-string vs. some other kind of >> string, but nothing internal to the string. >> >> YMMV, but that would be my preference in my text editor. Curly braces are >> perfectly good visual distinction to me. >> > At least vim does this and so does Sublime Text IIRC. Maybe I spend a lot > of time writing shell code too, but I very much appreciate the extra visual > cue. > > The only real point I'm trying to make is that expressions within an > f-string are an *escape*. They escape the normal semantics of a string > literal and instead do something else for a while. Therefore, the escaped > sections should not look like (or need to conform to) the rest of the > string and they should not require quoting as if it were still within the > string, because I escaped it already! > ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On Fri, Aug 19, 2016 at 3:39 PM, David Mertz wrote: > I don't think I've ever used a syntax highlighter than changed color of \n > in a string. I get the concept, but I haven't suffered for the absence of > that. > > Moreover, although I haven't yet used them, I really doubt I want extra > syntax highlighting in f-strings beyond simply the color strings appear as. > Well, maybe a uniform distinction for f-string vs. some other kind of > string, but nothing internal to the string. > > YMMV, but that would be my preference in my text editor. Curly braces are > perfectly good visual distinction to me. > At least vim does this and so does Sublime Text IIRC. Maybe I spend a lot of time writing shell code too, but I very much appreciate the extra visual cue. The only real point I'm trying to make is that expressions within an f-string are an *escape*. They escape the normal semantics of a string literal and instead do something else for a while. Therefore, the escaped sections should not look like (or need to conform to) the rest of the string and they should not require quoting as if it were still within the string, because I escaped it already! ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
I don't think I've ever used a syntax highlighter than changed color of \n in a string. I get the concept, but I haven't suffered for the absence of that. Moreover, although I haven't yet used them, I really doubt I want extra syntax highlighting in f-strings beyond simply the color strings appear as. Well, maybe a uniform distinction for f-string vs. some other kind of string, but nothing internal to the string. YMMV, but that would be my preference in my text editor. Curly braces are perfectly good visual distinction to me. On Aug 19, 2016 1:25 PM, "C Anthony Risinger" wrote: > On Fri, Aug 19, 2016 at 3:11 PM, C Anthony Risinger > wrote: >> >> >> When I look at a string I want to immediately know just how literal it >> really is. >> > > To further this point, editors today show me \n and \t and friends in a > different color, because they are escapes, and this visually tells me the > thing going into the string at that point is not what is literally in the > code. A raw string does not highlight these because they are no longer > escapes, and what you see is what you get. > > Probably f-strings will be used most in short strings, but they'll also be > used for long, heredoc-like triple-quoted strings. It's not going to be fun > picking expressions out of that when the wall-of-text contains no visual > cues. > > ___ > Python-ideas mailing list > Python-ideas@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On Fri, Aug 19, 2016 at 3:11 PM, C Anthony Risinger wrote: > > > When I look at a string I want to immediately know just how literal it > really is. > To further this point, editors today show me \n and \t and friends in a different color, because they are escapes, and this visually tells me the thing going into the string at that point is not what is literally in the code. A raw string does not highlight these because they are no longer escapes, and what you see is what you get. Probably f-strings will be used most in short strings, but they'll also be used for long, heredoc-like triple-quoted strings. It's not going to be fun picking expressions out of that when the wall-of-text contains no visual cues. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On Fri, Aug 19, 2016 at 1:43 PM, Eric V. Smith wrote:
>
>
> With your proposal, it's much more difficult to find the end of an
> f-string. I do not think this is a reasonable requirement.
>
> For example, consider the following:
> f'a{func({'a{':1,'3}':[{}, ')', '{']})}b'
>
> A regex will not be able to deal with the matching braces needed to
> find the end of the expression. You need to keep track of the nesting
> level of parens, braces, brackets, and quotes (at least those, I might
> have left something off).
>
> The way this currently is written in Python 3.6a4:
> f"a{func({'a{':1,'3}':[{}, ')', '{']})}b"
>
> It's trivially easy to find the end of the string. It's easy for both
> humans and the parsers.
>
It might be harder to find the end of an f-string in one shot, but I think
that's the crux of the issue: to a reader/developer, is an f-string
conceptually one thing or a compound thing?
To me (someone who would like to see f-string expressions appear like
normal expressions, without extra quoting, and proper syntax highlighting
*always*, just like shell), this argument is essentially the same as trying
to use a regex to find a closing bracket or brace or parse HTML. It's only
hard (disregarding any underlying impl details) because that view regards
f-strings as singular things with only one "end", when in reality an
f-string is much much more like a compound expression that just happens to
look like a string.
If one rejects the notion that an f-string is "one thing", the boundaries
can then be defined as either an unescaped closing quote an unescaped
opening curly brace. When that boundary is met, the highlighter switches to
normal python syntax parsing just like it would have at the real end of the
string. It also starts looking for a closing curly brace to denote the
start of "another" string.
There is a difference however in that f-string "expressions" also support
format specifications. These are not proper Python expressions to begin
with so they don't have any existing highlights. Not sure what they should
look like
I really think it should. Please look at python code with f-literals. if
>> they’re highlighted as strings throughout, you won’t be able to spot
>> which parts are code. if they’re highlighted as code, the escaping rules
>> guarantee that most highlighters can’t correctly highlight python
>> anymore. i think that’s a big issue for readability.
>>
>
> Maybe I'm the odd man out, but I really don't care if my editor ever
> syntax highlights within f-strings. I don't plan on putting anything more
> complicated than variable names in my f-strings, and I think PEP 8 should
> recommend something similar.
>
If things aren't highlighted properly I can't see them very well. If
f-strings look like other strings in my editor I will absolutely gloss over
them as normal strings, expecting no magic, until I later realize some
other way. Since I spend significant amounts of time reading foreign code I
could see this being quite obnoxious.
It might be sufficient to highlight the entire f-string a different color,
but honestly I don't think an f-string should ever ever... ever
misrepresent itself as a "string", because it's not. It's code that outputs
a string, in a compact and concise way.
Proper syntax highlighting is one of the most important things in my
day-to-day development. I loved the idea of f-strings when they were
discussed previously, but I haven't used them yet. If they hide what they
really doing under the guise of being a string, I personally will use them
much less, despite wanting to, and understanding their value.
When I look at a string I want to immediately know just how literal it
really is.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On 19 August 2016 at 19:43, Eric V. Smith wrote: > Maybe I'm the odd man out, but I really don't care if my editor ever syntax > highlights within f-strings. I don't plan on putting anything more > complicated than variable names in my f-strings, and I think PEP 8 should > recommend something similar. I agree entirely with this. You're definitely not alone. Paul ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On 19Aug2016 1157, Guido van Rossum wrote:
I don't think we should take action now.
Would it make sense, as a precaution, to declare the PEP provisional for
one release? Then we can develop a sense of whether the current approach
causes real problems.
We could also emit some kind of warning if the expression part contains
an escaped quote, since that's where a potential change would cause
breakage. (Or we could leave that to the linters.)
After reading the responses, I like the idea of explicitly discouraging
any sort of string escapes within the expression (whether quotes or
special characters), but think it's best left for the linters and style
guides.
Basically, avoid writing a literal f'{expr}' where you'd need to modify
expr at all to rewrite it as:
>>> x = expr
>>> f'{x}'
We will almost certainly be looking to enable code completions and
syntax highlighting in Visual Studio within expressions, and we can't
easily process the string and then parse it for this purpose, but I
think we'll be able to gracefully degrade in cases where escapes that
are valid in strings are not valid in code.
Cheers,
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On Sat, Aug 20, 2016 at 4:43 AM, Eric V. Smith wrote:
> Maybe I'm the odd man out, but I really don't care if my editor ever syntax
> highlights within f-strings. I don't plan on putting anything more
> complicated than variable names in my f-strings, and I think PEP 8 should
> recommend something similar.
I'd go further than "variable names", and happily include attribute
access, subscripting (item access), etc, including a couple of levels
of same:
def __repr__(self):
return f"{self.__class__.__name__}(foo={self.foo!r}, spam={self.spam!r})"
But yes, I wouldn't put arbitrarily complex expressions into
f-strings. Whether PEP 8 needs to explicitly say so or not, I would
agree with you that it's a bad idea.
ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
I don't think we should take action now. Would it make sense, as a precaution, to declare the PEP provisional for one release? Then we can develop a sense of whether the current approach causes real problems. We could also emit some kind of warning if the expression part contains an escaped quote, since that's where a potential change would cause breakage. (Or we could leave that to the linters.) -- --Guido van Rossum (python.org/~guido) ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On 8/18/2016 11:05 AM, Philipp A. wrote:
Hi, I originially posted this via google groups, which didn’t make it
through to the list proper, sorry! Read it here please:
https://groups.google.com/forum/#!topic/python-ideas/V1U6DGL5J1s
Hi, Philipp.
I'm including your original proposal here, so that it's archived
properly. Here's what you'd like to have work:
f'foo{repr('bar\n') * 3}baz' == 'foo"bar\n""bar\n""bar\n"baz'
We've had this discussion before, but I can't find it in the
archives. It's possible it happened off-list.
The problem I have with your proposal is that it greatly complicates
the implementation and it makes it harder for humans to reason about
the code (for the same reasons). This is not just an ease of
implementation issue: it's a cognitive burden issue.
As it currently stands, Python strings, of all 'flavors' (raw, unicode,
binary, and f-), are parsed the same way: an optional prefix, one or
three quote characters, the body of the string, then matching quote
characters. This is sufficiently simple that it can be (and often is)
implemented with regular expressions. You also need to support
combinations of prefixes, like raw f-strings (fr or rf).
With your proposal, it's much more difficult to find the end of an
f-string. I do not think this is a reasonable requirement.
For example, consider the following:
f'a{func({'a{':1,'3}':[{}, ')', '{']})}b'
A regex will not be able to deal with the matching braces needed to
find the end of the expression. You need to keep track of the nesting
level of parens, braces, brackets, and quotes (at least those, I might
have left something off).
The way this currently is written in Python 3.6a4:
f"a{func({'a{':1,'3}':[{}, ')', '{']})}b"
It's trivially easy to find the end of the string. It's easy for both
humans and the parsers.
Now admittedly in order to execute or syntax highlight the existing
f-strings, you need to perform this parsing. But of the 3 parsers that
ship with Python (ast.c, tokenize.py, IDLE), only ast.c needs to do
that currently. I don't think tokenize.py ever will, and IDLE might
(but could use the ast module).
I think many parsers (e.g. python-mode.el, etc.) will just be able to
simply consume the f-strings without looking inside them and move one,
and we shouldn't unnecessarily complicate them.
My arguments are basically:
1. f-literals are semantically not strings, but expressions.
2. Their escape sequences in the code parts are fundamentally both
detrimental and superfluous (they’re only in for convenience, as
confirmed by Guido in the quote below)
I disagree that they're detrimental and superfluous. I'd say they're
consistent with all other strings.
3. They’re detrimental because Syntax highlighters are (by design)
unable to handle this part of Python 3.6a4’s grammar. This will
cause code to be highlighted as parts of a string and therefore
overlooked. i’m very sure this will cause bugs.
I disagree.
4. The fact that people see the embedded expressions as somehow “part
of the string” is confusing.
My poposal is to redo their grammar:
They shouldn’t be parsed as strings and post-processed, but be their own
thing. This also opens the door to potentially extend to with something
like JavaScript’s tagged templates)
Without the limitations of the string tokenization code/rules, only the
string parts would have escape sequences, and the expression parts would
be regular python code (“holes” in the literal).
Below the mentioned quote and some replies to the original thread:
Guido van Rossum mailto:gu...@python.org>> schrieb am
Mi., 17. Aug. 2016 um 20:11 Uhr:
The explanation is honestly that the current approach is the most
straightforward for the implementation (it's pretty hard to
intercept the string literal before escapes have been processed) and
nobody cares enough about the edge cases to force the implementation
to jump through more hoops.
I really don't think this discussion should be reopened. If you
disagree, please start a new thread on python-ideas.
I really think it should. Please look at python code with f-literals. if
they’re highlighted as strings throughout, you won’t be able to spot
which parts are code. if they’re highlighted as code, the escaping rules
guarantee that most highlighters can’t correctly highlight python
anymore. i think that’s a big issue for readability.
Maybe I'm the odd man out, but I really don't care if my editor ever
syntax highlights within f-strings. I don't plan on putting anything
more complicated than variable names in my f-strings, and I think PEP 8
should recommend something similar.
Brett Cannon mailto:br...@python.org>> schrieb am
Mi., 17. Aug. 2016 um 20:28 Uhr:
They are still strings, there is just post-processing on the string
itself to do the interpolation.
Sounds hacky to me. I’d rather see a proper parser for them, which of
course would make my vision easy.
You're saying i
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On 8/19/2016 2:07 AM, אלעזר wrote: בתאריך יום ו׳, 19 באוג' 2016, 08:29, מאת Terry Reedy mailto:tjre...@udel.edu>>: On 8/18/2016 8:18 PM, Steven D'Aprano wrote: > On Fri, Aug 19, 2016 at 02:17:29AM +1000, Chris Angelico wrote: > >> Format codes are just text, > > I really think that is wrong. They're more like executable code. > > https://www.python.org/dev/peps/pep-0498/#expression-evaluation I agree with you here. I just note that the strings passed to exec, eval, and compile are also executable code strings (and nothing but!). But I don't remember a suggestion that *they* should by colored as anything other than a string. But these are objects of type str, not string literals. If they were, I guess someone would have suggested such coloring. I was referring to strings created by string literals in the code sitting in an editor. -- Terry Jan Reedy ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On 19 August 2016 at 18:27, Eric V. Smith wrote:
> For something else that would become significantly more complicated to
> implement, you need look no further than the stdlib's own tokenizer module.
> So Python itself would require changes to parsers/lexers in Python/ast.c,
> IDLE, and Lib/tokenizer.py. In addition it would require adding tokens to
> Include/tokens.h and the generated Lib/token.py, and everyone using those
> files would need to adapt.
>
> Not that it's impossible, of course. But don't underestimate the amount of
> work this proposal would cause to the many places in and outside of Python
> that examine Python code.
And if folks want to do something more clever than regex based single
colour string highlighting, Python's own AST module is available to
help them out:
>>> tree = ast.parse("f'example{parsing:formatting}and trailer'")
>>> ast.dump(tree)
"Module(body=[Expr(value=JoinedStr(values=[Str(s='example'),
FormattedValue(value=Name(id='parsing', ctx=Load()), conversion=-1,
format_spec=Str(s='formatting')), Str(s='and trailer')]))])"
Extracting the location of the field expression for syntax highlighting:
>>> ast.dump(tree.body[0].value.values[1].value)
"Name(id='parsing', ctx=Load())"
(I haven't shown it in the example, but AST nodes have lineno and
col_offset fields so you can relocate the original source code for
processing)
Cheers,
Nick.
--
Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On 8/19/2016 1:16 AM, Terry Reedy wrote: On 8/18/2016 8:27 PM, Eric V. Smith wrote: So something that parses or scans a Python file and currently understands u, b, and r to be string prefixes, just needs to add f to the prefixes it uses, and it can now at least understand f-strings (and fr-strings). It doesn't need to implement a full-blown expression parser just to find out where the end of a f-string is. Indeed, IDLE has one prefix re, which has changed occasionally and which I need to change for 3.6, and 4 res for the 4 unprefixed strings, which have been the same, AFAIK, for decades. It that prefixes all 4 string res with the prefix re and o or's the results together to get the 'string' re. For something else that would become significantly more complicated to implement, you need look no further than the stdlib's own tokenizer module. So Python itself would require changes to parsers/lexers in Python/ast.c, IDLE, and Lib/tokenizer.py. In addition it would require adding tokens to Include/tokens.h and the generated Lib/token.py, and everyone using those files would need to adapt. Not that it's impossible, of course. But don't underestimate the amount of work this proposal would cause to the many places in and outside of Python that examine Python code. Eric. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On 19 August 2016 at 06:07, Terry Reedy wrote:
> It seems to me that that this is at least somewhat a strawman issue.
>
> If you want to prohibit backslashed quote reuse in expressions, as in
> f'{x.partition(\'-\')[0]}', that is okay with me, as this is unnecessary*
> and arguably bad. The third alternative above is better. What breaks
> colorizers, and what I therefore object to, is the innovation of adding
> magical escaping of ' or " without \.
>
> Or add a new style rule to PEP 8.
>
> F-strings: avoid unnecessary escaping in the expression part of f-strings.
> Good: f"{x.partition('-')[0]}"
> Bad: f'{x.partition(\'-\')[0]}'
>
> Then PEP-8 checkers will flag such usage.
+1.
While substantial IDEs like PyCharm or PTVS may use a full-scale
parser to do syntax highlighting, I suspect that many tools just use
relatively basic regex parsing (Vim certainly does). For those tools,
unescaped nested quotes will likely be extremely difficult, if not
impossible, to parse correctly. Whereas the current behaviour is
"just" standard string highlighting.
So if the Python parser were to change as proposed, I'd still argue
strongly for a coding style that never uses any construct that would
be interpreted differently from current behaviour (i.e., the changed
behaviour should essentially be irrelevant). Given this, I thing the
argument to change, whether it's theoretically an improvement or not,
is irrelevant, and practicality says there's no point in bothering.
(Python's parser is intentionally simple, to make it easy for humans
and tools to parse Python code - I'm not sure the proposed change to
syntax meets that guideline for simple syntax).
Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On 19.08.16 08:07, Terry Reedy wrote:
On 8/18/2016 3:30 PM, Steve Dower wrote:
On 18Aug2016 1215, Terry Reedy wrote:
On 8/18/2016 12:50 PM, Steve Dower wrote:
I don't think f'{x.partition('-')[0]}' is any less readable as a result
of the reused quotes,
Why are you reusing the single quote', which needs the escaping that you
don't like, instead of any of at least 6 alternatives that do not need
any escaping?
f'{x.partition("-")[0]}'
f'{x.partition("""-""")[0]}'
f"{x.partition('-')[0]}"
f'''{x.partition('-')[0]}'''
f"""{x.partition('-')[0]}"""
f"""{x.partition('''-''')[0]}"""
It seems to me that that this is at least somewhat a strawman issue.
If you want to prohibit backslashed quote reuse in expressions, as in
f'{x.partition(\'-\')[0]}', that is okay with me, as this is
unnecessary* and arguably bad. The third alternative above is better.
What breaks colorizers, and what I therefore object to, is the
innovation of adding magical escaping of ' or " without \.
Or add a new style rule to PEP 8.
+1. It is even possible to add a SyntaxWarning in future.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
בתאריך יום ו׳, 19 באוג' 2016, 08:29, מאת Terry Reedy : > On 8/18/2016 8:18 PM, Steven D'Aprano wrote: > > On Fri, Aug 19, 2016 at 02:17:29AM +1000, Chris Angelico wrote: > > > >> Format codes are just text, > > > > I really think that is wrong. They're more like executable code. > > > > https://www.python.org/dev/peps/pep-0498/#expression-evaluation > > I agree with you here. I just note that the strings passed to exec, > eval, and compile are also executable code strings (and nothing but!). > But I don't remember a suggestion that *they* should by colored as > anything other than a string. But these are objects of type str, not string literals. If they were, I guess someone would have suggested such coloring. ~Elazar ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On Thu, Aug 18, 2016 at 08:27:50PM -0400, Eric V. Smith wrote: > Right. Because all strings (regardless of prefixes) are first parsed as > strings, and then have their prefix "operator" applied, it's easy for a > parser to ignore any sting prefix character. Is that why raw strings can't end with a backspace? If so, that's the first time I've seen an explanation of that fact which makes sense! -- Steve ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On 8/18/2016 8:18 PM, Steven D'Aprano wrote: On Fri, Aug 19, 2016 at 02:17:29AM +1000, Chris Angelico wrote: Format codes are just text, I really think that is wrong. They're more like executable code. https://www.python.org/dev/peps/pep-0498/#expression-evaluation I agree with you here. I just note that the strings passed to exec, eval, and compile are also executable code strings (and nothing but!). But I don't remember a suggestion that *they* should by colored as anything other than a string. However, this thread has suggested to me that perhaps there *should* be a way to syntax check such strings in the editor rather than waiting for the runtime call. -- Terry Jan Reedy ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On 8/18/2016 8:27 PM, Eric V. Smith wrote:
On 8/18/2016 3:15 PM, Terry Reedy wrote:
Without the escapes, existing f-unaware highlighters like IDLE's will be
broken in that they will highlight the single f-string as two strings
with differently highlighted content in the middle. For
f'{x.partition('if')[0]}', the 'if' is and will be erroneously
highlighted as a keyword. I consider this breakage unacceptible.
Right. Because all strings (regardless of prefixes) are first parsed as
strings, and then have their prefix "operator" applied, it's easy for a
parser to ignore any sting prefix character.
So something that parses or scans a Python file and currently
understands u, b, and r to be string prefixes, just needs to add f to
the prefixes it uses, and it can now at least understand f-strings (and
fr-strings). It doesn't need to implement a full-blown expression parser
just to find out where the end of a f-string is.
Indeed, IDLE has one prefix re, which has changed occasionally and which
I need to change for 3.6, and 4 res for the 4 unprefixed strings, which
have been the same, AFAIK, for decades. It that prefixes all 4 string
res with the prefix re and o or's the results together to get the
'string' re.
--
Terry Jan Reedy
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On 8/18/2016 3:30 PM, Steve Dower wrote:
On 18Aug2016 1215, Terry Reedy wrote:
On 8/18/2016 12:50 PM, Steve Dower wrote:
I don't think f'{x.partition('-')[0]}' is any less readable as a result
of the reused quotes,
Why are you reusing the single quote', which needs the escaping that you
don't like, instead of any of at least 6 alternatives that do not need
any escaping?
f'{x.partition("-")[0]}'
f'{x.partition("""-""")[0]}'
f"{x.partition('-')[0]}"
f'''{x.partition('-')[0]}'''
f"""{x.partition('-')[0]}"""
f"""{x.partition('''-''')[0]}"""
It seems to me that that this is at least somewhat a strawman issue.
If you want to prohibit backslashed quote reuse in expressions, as in
f'{x.partition(\'-\')[0]}', that is okay with me, as this is
unnecessary* and arguably bad. The third alternative above is better.
What breaks colorizers, and what I therefore object to, is the
innovation of adding magical escaping of ' or " without \.
Or add a new style rule to PEP 8.
F-strings: avoid unnecessary escaping in the expression part of f-strings.
Good: f"{x.partition('-')[0]}"
Bad: f'{x.partition(\'-\')[0]}'
Then PEP-8 checkers will flag such usage.
*I am sure that there are possible complex expressions that would be
prohibited by the rule that would be otherwise possible. But they
should be extremely rare and possibly not the best solution anyway.
I find it hard to not read f'{x.partition(' + ')[0]}' as string
concatenation.
That's a fair counter-example. Though f'{x.partition(\' + \')[0]}' still
reads like string concatenation to me at first glance. YMMV.
When the outer and inner quotes are no longer the same, the effect is
greatly diminished if not eliminated.
and it will certainly be easier for highlighters
to handle (assuming they're doing anything more complicated than simply
displaying the entire expression in a different colour).
Without the escapes, existing f-unaware highlighters like IDLE's will be
broken in that they will highlight the single f-string as two strings
with differently highlighted content in the middle. For
f'{x.partition('if')[0]}', the 'if' is and will be erroneously
highlighted as a keyword. I consider this breakage unacceptible.
Won't it be broken anyway because of the new prefix?
No. IDLE currently handles f-strings just fine other than not coloring
the 'f'. This is a minor issue and easily fixed by adding '|f' and if
allowed, '|F' at the end of the current stringprefix re.
I'm sure there's a fairly straightforward way for a regex to say that a
closing quote must not be preceded immediately by a backslash or by an
open brace at all without a closing brace in between.
I do not know that this is possible.
Here is IDLE's current re for an unprefixed single quoted string.
r"'[^'\\\n]*(\\.[^'\\\n]*)*'?"
The close quote is optional because it must match a string that is in
the process of being typed and is not yet closed. I consider providing
a tested augmented re to be required for this proposal.
Even then, making the parsing out of strings in Python code for
colorizing version dependent is a problem in itself for colorizers not
tied to a particular x.y version. Leaving prefixes aside, I can't
remember string delimiter syntax changing since I learned it in 1.3.
Not having escapes within the expression makes it harder for everyone
except the Python developer, in my opinion, and the rest of us ought to
go out of our way for them.
I am not sure that this says what you mean.
--
Terry Jan Reedy
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On Fri, Aug 19, 2016 at 10:18 AM, Steven D'Aprano wrote:
> On Fri, Aug 19, 2016 at 02:17:29AM +1000, Chris Angelico wrote:
>
>> Format codes are just text,
>
> I really think that is wrong. They're more like executable code.
>
> https://www.python.org/dev/peps/pep-0498/#expression-evaluation
>
> "Just text" implies it is data:
>
> result = "function(arg)"
>
> like the string on the right hand side of the = is data. You wouldn't
> say that a function call was data (although it may *return* data):
>
> result = function(arg)
>
> or that it was "just text", and you shouldn't say the same about:
>
> result = f"{function(arg)}"
>
> either since they are functionally equivalent. Format codes are "just
> text" only in the sense that source code is "just text". Its technically
> correct and horribly misleading.
>
By "format code", I'm talking about the bit after the colon, which
isn't executable code, but is a directive that says how the result is
to be formatted. These have existed since str.format() was introduced,
and have always been text, not code.
ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On 8/18/2016 3:15 PM, Terry Reedy wrote:
On 8/18/2016 12:50 PM, Steve Dower wrote:
I find it hard to not read f'{x.partition(' + ')[0]}' as string
concatenation.
and it will certainly be easier for highlighters
to handle (assuming they're doing anything more complicated than simply
displaying the entire expression in a different colour).
Without the escapes, existing f-unaware highlighters like IDLE's will be
broken in that they will highlight the single f-string as two strings
with differently highlighted content in the middle. For
f'{x.partition('if')[0]}', the 'if' is and will be erroneously
highlighted as a keyword. I consider this breakage unacceptible.
Right. Because all strings (regardless of prefixes) are first parsed as
strings, and then have their prefix "operator" applied, it's easy for a
parser to ignore any sting prefix character.
So something that parses or scans a Python file and currently
understands u, b, and r to be string prefixes, just needs to add f to
the prefixes it uses, and it can now at least understand f-strings (and
fr-strings). It doesn't need to implement a full-blown expression parser
just to find out where the end of a f-string is.
Eric.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On Thu, Aug 18, 2016 at 12:26:26PM -0400, Random832 wrote: > There's a precedent. "$()" works this way in bash - call it a recursive > parser context or whatever you like, but the point is that "$(command > "argument with spaces")" works fine, and humans don't seem to have any > trouble with it. This is the first time I've ever seen anyone claim that humans don't have any trouble with bash escaping and evaluation rules. -- Steve ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On Fri, Aug 19, 2016 at 02:17:29AM +1000, Chris Angelico wrote:
> Format codes are just text,
I really think that is wrong. They're more like executable code.
https://www.python.org/dev/peps/pep-0498/#expression-evaluation
"Just text" implies it is data:
result = "function(arg)"
like the string on the right hand side of the = is data. You wouldn't
say that a function call was data (although it may *return* data):
result = function(arg)
or that it was "just text", and you shouldn't say the same about:
result = f"{function(arg)}"
either since they are functionally equivalent. Format codes are "just
text" only in the sense that source code is "just text". Its technically
correct and horribly misleading.
> so I should be able to use Unicode
> escapes. Okay. Now let's make that an F-string.
>
> >>> f"This is a number: {13:0\u07c4}"
> 'This is a number: 0013'
If your aim is to write obfuscated code, then, yes, you should be able
to write something like that.
*wink*
I seem to recall that Java allows string escapes in ordinary
expressions, so that instead of writing:
result = function(arg)
you could write:
result = \x66\x75\x6e\x63\x74\x69\x6f\x6e\x28\x61\x72\x67\x29
instead. We can't, and shouldn't, allow anything like this in Python
code. Should we allow it inside f-strings?
result = f"{\x66\x75\x6e\x63\x74\x69\x6f\x6e\x28\x61\x72\x67\x29}"
--
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
Chris Angelico wrote:
f"This is a number: {13:0\u07c4}"
If I understand correctly, the proposal intends to make
it easier for a syntax hightlighter to treat
f"This is a number: {foo[42]:0\u07c4}"
as
f"This is a number: {foo[42] :0\u07c4}"
---- --
highlight as string hightlight highlight as string
as
code
I'm not sure an RE-based syntax hightlighter would
have any easier a time with that, because for the
second part it would need to recognise ':' as starting
a string, but only if it followed some stuff that was
preceded by the beginning of an f-string.
I'm not very familiar with syntax higlighters, so I
don't know if they're typically smart enought to cope
with things like that.
--
Greg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On 18Aug2016 1215, Terry Reedy wrote:
On 8/18/2016 12:50 PM, Steve Dower wrote:
I don't think f'{x.partition('-')[0]}' is any less readable as a result
of the reused quotes,
I find it hard to not read f'{x.partition(' + ')[0]}' as string
concatenation.
That's a fair counter-example. Though f'{x.partition(\' + \')[0]}' still
reads like string concatenation to me at first glance. YMMV.
and it will certainly be easier for highlighters
to handle (assuming they're doing anything more complicated than simply
displaying the entire expression in a different colour).
Without the escapes, existing f-unaware highlighters like IDLE's will be
broken in that they will highlight the single f-string as two strings
with differently highlighted content in the middle. For
f'{x.partition('if')[0]}', the 'if' is and will be erroneously
highlighted as a keyword. I consider this breakage unacceptible.
Won't it be broken anyway because of the new prefix?
I'm sure there's a fairly straightforward way for a regex to say that a
closing quote must not be preceded immediately by a backslash or by an
open brace at all without a closing brace in between.
Not having escapes within the expression makes it harder for everyone
except the Python developer, in my opinion, and the rest of us ought to
go out of our way for them.
Cheers,
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On Thu, Aug 18, 2016, at 15:15, Terry Reedy wrote:
> This is the crux of this thread. Is an f-string a single string that
> contains magically handled code, or interleaved strings using { and } as
> closing and opening quotes (which is backwards from their normal
> function of being opener and closer)
I'd rather conceptualize it as a sequence of two* kinds of thing:
literal character sequences [as sequences of characters other than {]
and expressions [started with {, and ended with a } that is not
otherwise part of the expression] rather than treating { as a closing
quote.
In particular, treating } as an opening quote doesn't really work, since
expressions can contain both strings (which may contain an unbalanced })
and dictionary/set literals (which contain balanced }'s which are not in
quotes) - what ends the expression is a } at the top level.
*or three, considering that escapes are used in the non-expression
parts.
> and expressions? The latter view
> makes the grammar context sensitive, I believe, as } could only open a
> string if there is a previous f-tagged string an indefinite number of
> alternations back.
} at the top level is otherwise a syntax error. I don't know enough
about the theoretical constructs involved to know if this makes it
formally 'context sensitive' or not - I don't know that it's any more
context sensitive than ) being valid if there is a matching (. Honestly,
I'd be more worried about : than }.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On 8/18/2016 12:50 PM, Steve Dower wrote:
> I'm generally inclined to agree, especially as someone who is very
> likely to be implementing syntax highlighting and completion support
> within f-literals.
I consider these separate issues. IDLE currently provides filename
completion support within strings while still highlighting the string
one color. Even if it were enhanced to do name completion within an
f-string, I am not sure I would want to put a mixture of colors within
the string rather than leave if all one color.
> I stepped out of the original discussion near the start as it looked
> like we were going to end up with interleaved strings and normal
> expressions, but if that's not the case then it is going to make it
> very difficult to provide a nice coding experience for them.
This is the crux of this thread. Is an f-string a single string that
contains magically handled code, or interleaved strings using { and } as
closing and opening quotes (which is backwards from their normal
function of being opener and closer) and expressions? The latter view
makes the grammar context sensitive, I believe, as } could only open a
string if there is a previous f-tagged string an indefinite number of
alternations back.
It is not uncommon to write strings that consist completely of code.
"for i in iterable: a.append(f(i))"
to be written out or eval()ed or exec()ed.
Does your environment have a mode to provide syntax highlighting and
completion support for such things?
What I think would be more useful would be the ability to syntax check
such code strings while editing. A python-coded editor could just pass
the extracted string to compile().
I don't think f'{x.partition('-')[0]}' is any less readable as a result
of the reused quotes,
I find it hard to not read f'{x.partition(' + ')[0]}' as string
concatenation.
and it will certainly be easier for highlighters
to handle (assuming they're doing anything more complicated than simply
displaying the entire expression in a different colour).
Without the escapes, existing f-unaware highlighters like IDLE's will be
broken in that they will highlight the single f-string as two strings
with differently highlighted content in the middle. For
f'{x.partition('if')[0]}', the 'if' is and will be erroneously
highlighted as a keyword. I consider this breakage unacceptible.
--
Terry Jan Reedy
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On Thu, 18 Aug 2016 at 08:32 Philipp A. wrote: > [SNIP] > Brett Cannon schrieb am Mi., 17. Aug. 2016 um > 20:28 Uhr: > >> They are still strings, there is just post-processing on the string >> itself to do the interpolation. >> > > Sounds hacky to me. I’d rather see a proper parser for them, which of > course would make my vision easy. > You say "hacky", I say "pragmatic". And Python's code base is actually rather top-notch and so it isn't bad code, but simply a design decision you are disagreeing with. Please remember that you're essentially asking people to spend their personal time to remove working code and re-implement something that you have not volunteered to actually code up yourself. Don't forget that none of us get paid to work on Python full-time; a lucky couple of us get to spend one day a week on Python and we all take time away from our family to work on things when we can. Insulting someone's hard work that they did for free to try and improve Python is not going to motivate people to want to help out with this idea. And considering Eric Smith who originally implemented all of this is possibly the person in the best position to implement your idea just had his work called "hacky" by you is not really a great motivator for him. IOW you really need to be mindful of the tone of your emails (as does anyone else who ever asks for something to change while not being willing to put in the time and effort to actually produce the code to facilitate the change). You have now had both Steve and me point out your tone and so you're quickly approaching a threshold where people will stop pointing this out and simply ignore your emails, so please be mindful of how you phrase things. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On 18Aug2016 0950, Steve Dower wrote: I'm generally inclined to agree, especially as someone who is very likely to be implementing syntax highlighting and completion support within f-literals. I also really don't like the subject line. "Do not require string escapes within expressions in f-literals" more accurately represents the topic and the suggestion. "Let's make impossible" is just asking for a highly emotionally-charged discussion, which is best avoided in basically all circumstances, especially for less-frequent contributors to a community, and extra-especially when you haven't met most of the other contributors in person. Cheers, Steve ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
I'm generally inclined to agree, especially as someone who is very
likely to be implementing syntax highlighting and completion support
within f-literals.
I stepped out of the original discussion near the start as it looked
like we were going to end up with interleaved strings and normal
expressions, but if that's not the case then it is going to make it very
difficult to provide a nice coding experience for them.
On 18Aug2016 0805, Philipp A. wrote:
My poposal is to redo their grammar:
They shouldn’t be parsed as strings and post-processed, but be their own
thing. This also opens the door to potentially extend to with something
like JavaScript’s tagged templates)
Without the limitations of the string tokenization code/rules, only the
string parts would have escape sequences, and the expression parts would
be regular python code (“holes” in the literal).
This is where I thought we'd end up - the '{' character (unless escaped
by, e.g. \N, which addresses a concern below) would terminate the string
literal and start an expression, which may be followed by a ':' and a
format code literal. The '}' character would open the next string
literal, and this continues until the closing quote.
They are still strings, there is just post-processing on the string
itself to do the interpolation.
Sounds hacky to me. I’d rather see a proper parser for them
I believe the proper parser is already used, but the issue is that
escapes have already been dealt with. Of course, it shouldn't be too
difficult for the tokenizer to recognize {} quoted expressions within an
f-literal and not modify escapes. There are multiple ways to handle this.
Or another reason is you can explain f-strings as "basically
str.format_map(**locals(), **globals()), but without having to make
the actual method call" (and worrying about clashing keys but I
couldn't think of a way of using dict.update() in a single line).
But with your desired change it kills this explanation by saying
f-strings aren't like this but some magical string that does all of
this stuff before normal string normalization occurs.
no, it’s simply the expression parts (that for normal formatting are
inside of the braces of .format(...)) are *interleaved* in between
string parts. they’re not part of the string. just regular plain python
code.
Agreed. The .format_map() analogy breaks down very quickly when you
consider f-literals like:
>>> f'a { \'b\' }'
'a b'
If the contents of the braces were simply keys in the namespace then we
wouldn't be able to put string literals in there. But because it is an
arbitrary expression, if we want to put string literals in the f-literal
(bearing in mind that we may be writing something more like
f'{x.partition(\'-\')[0]}'), the escaping rules become very messy very
quickly.
I don't think f'{x.partition('-')[0]}' is any less readable as a result
of the reused quotes, and it will certainly be easier for highlighters
to handle (assuming they're doing anything more complicated than simply
displaying the entire expression in a different colour).
So I too would like to see escapes made unnecessary within the
expression part of a f-literal. Possibly if we put together a simple
enough patch for the tokenizer it will be accepted?
Cheers,
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On Thu, Aug 18, 2016, at 12:17, Chris Angelico wrote: > The trouble with that way of thinking is that, to a human, the braces > contain something. They don't "uncontain" it. Those braced expressions > are still part of a string; they just have this bit of magic that gets > them evaluated. Consider this: There's a precedent. "$()" works this way in bash - call it a recursive parser context or whatever you like, but the point is that "$(command "argument with spaces")" works fine, and humans don't seem to have any trouble with it. Really it all comes down to what exactly the "bit of magic" is and how magical it is. > IMO it doesn't matter that much either way - people will have to > figure stuff out anyway. I like the idea that everything in the quotes > is a string (and then parts of it get magically evaluated), but could > live with there being some non-stringy parts in it. My suspicion is > that what's easiest to code (ie easiest for the CPython parser) is > also going to be easiest for all or most other tools (eg syntax > highlighters). Except the parser has to actually parse string literals into what string they represent (so it can apply a further transformation to the result). Syntax highlighters generally don't. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Let’s make escaping in f-literals impossible
On Fri, Aug 19, 2016 at 1:05 AM, Philipp A. wrote:
> the embedded expressions are just normal python. the embedded strings just
> normal strings. you can simply switch between both using <{> and
> <[format]}>.
>
The trouble with that way of thinking is that, to a human, the braces
contain something. They don't "uncontain" it. Those braced expressions
are still part of a string; they just have this bit of magic that gets
them evaluated. Consider this:
>>> "This is a number: {:0\u07c4}".format(13)
'This is a number: 0013'
Format codes are just text, so I should be able to use Unicode
escapes. Okay. Now let's make that an F-string.
>>> f"This is a number: {13:0\u07c4}"
'This is a number: 0013'
Format codes are still just text. So you'd have to say that the rules
of text stop at an unbracketed colon, which is a pretty complicated
rule to follow. The only difference between .format and f-strings is
that the bit before the colon is the actual expression, rather than a
placeholder that drags the value in from the format arguments. In
human terms, that's not all that significant.
IMO it doesn't matter that much either way - people will have to
figure stuff out anyway. I like the idea that everything in the quotes
is a string (and then parts of it get magically evaluated), but could
live with there being some non-stringy parts in it. My suspicion is
that what's easiest to code (ie easiest for the CPython parser) is
also going to be easiest for all or most other tools (eg syntax
highlighters).
ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
