[Python-ideas] Re: Pre PEP: Python Literals (was custom strings before)

[Guido van Rossum]

On Mon, Jul 5, 2021 at 5:05 PM Chris Angelico  wrote:

> On Tue, Jul 6, 2021 at 9:39 AM Greg Ewing 
> wrote:
> >
> > On 6/07/21 9:56 am, Jim Baker wrote:
> > >
> > > d = deferred_tag"Some expr: {:(x*2)}"
> > >
> > > All that is happening here is that this being wrapped in a lambda,
> which
> > > captures any scope lexically as usual.
> >
> > Is there reason to think this will be a common enough requirement
> > to justify having an abbreviated syntax for a parameterless lambda
> > that's only available in template expressions?
> >
>
> If it's just being wrapped in a lambda function, probably nothing, but
> I think that that would be *very* surprising behaviour. People will
> expect that the expressions' values will be collected at the point you
> hit the interpolated string, not later when it gets used.
>

It would be surprising indeed if this was the *default* behavior, that's
why you have to specially mark it using {:...} in Jim's proposal.

IIUC the idea is that the entire template becomes effectively a lambda.
It's useful if the evaluation is relatively expensive and may never be
needed, e.g. for logging at a level that is off in production. We can
debate whether it's better to mark individual substitutions with something
like {:...} or whether we should mark the template as a whole (obviously
the marking must be understandable by the parser).

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/TUAFAIOH2WEHZOKQZ4RF2M6H6KVI7CMY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Pre PEP: Python Literals (was custom strings before)

[Chris Angelico]

On Tue, Jul 6, 2021 at 10:44 AM Guido van Rossum  wrote:
>
> On Mon, Jul 5, 2021 at 5:05 PM Chris Angelico  wrote:
>>
>> On Tue, Jul 6, 2021 at 9:39 AM Greg Ewing  
>> wrote:
>> >
>> > On 6/07/21 9:56 am, Jim Baker wrote:
>> > >
>> > > d = deferred_tag"Some expr: {:(x*2)}"
>> > >
>> > > All that is happening here is that this being wrapped in a lambda, which
>> > > captures any scope lexically as usual.
>> >
>> > Is there reason to think this will be a common enough requirement
>> > to justify having an abbreviated syntax for a parameterless lambda
>> > that's only available in template expressions?
>> >
>>
>> If it's just being wrapped in a lambda function, probably nothing, but
>> I think that that would be *very* surprising behaviour. People will
>> expect that the expressions' values will be collected at the point you
>> hit the interpolated string, not later when it gets used.
>
>
> It would be surprising indeed if this was the *default* behavior, that's why 
> you have to specially mark it using {:...} in Jim's proposal.
>
> IIUC the idea is that the entire template becomes effectively a lambda. It's 
> useful if the evaluation is relatively expensive and may never be needed, 
> e.g. for logging at a level that is off in production. We can debate whether 
> it's better to mark individual substitutions with something like {:...} or 
> whether we should mark the template as a whole (obviously the marking must be 
> understandable by the parser).
>

Ahh, I see what you mean. It'll still be surprising at times (cf the
bizarre issues that can happen with mutable objects and Chrome's JS
developer tools), but could be beneficial, yeah.

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/P6COOXWN47PBC2TKT2QJREMVXAC2SQKR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Pre PEP: Python Literals (was custom strings before)

[Chris Angelico]

On Tue, Jul 6, 2021 at 9:39 AM Greg Ewing  wrote:
>
> On 6/07/21 9:56 am, Jim Baker wrote:
> >
> > d = deferred_tag"Some expr: {:(x*2)}"
> >
> > All that is happening here is that this being wrapped in a lambda, which
> > captures any scope lexically as usual.
>
> Is there reason to think this will be a common enough requirement
> to justify having an abbreviated syntax for a parameterless lambda
> that's only available in template expressions?
>

If it's just being wrapped in a lambda function, probably nothing, but
I think that that would be *very* surprising behaviour. People will
expect that the expressions' values will be collected at the point you
hit the interpolated string, not later when it gets used.

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/T5QSTL4UQQJH74ZGSV3D76PBVTSUCDI2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Pre PEP: Python Literals (was custom strings before)

[Greg Ewing]

On 6/07/21 9:56 am, Jim Baker wrote:


d = deferred_tag"Some expr: {:(x*2)}"

All that is happening here is that this being wrapped in a lambda, which 
captures any scope lexically as usual.


Is there reason to think this will be a common enough requirement
to justify having an abbreviated syntax for a parameterless lambda
that's only available in template expressions?

--
Greg

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/LCDEAE5DLEJ6ELDT5XLBONO4ZZPHS3TO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Pre PEP: Python Literals (was custom strings before)

[Greg Ewing]

On 6/07/21 8:39 am, Guido van Rossum wrote:
FWIW, we could make f-strings properly nest  too... So this is 
not an argument for backticks.


An argument might be that single and double quotes currently do
not nest and never have done, so having them start to nest but only
in the context of f-strings could be confusing. Whereas backticks
are currently unused, and they used to nest back when they were
used, so giving them a new nesting usage will be unsurprising to
those with long enough memories.

--
Greg
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/K3HXQ26X7TAHABRETMU2EILG4EQ2CXK4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Pre PEP: Python Literals (was custom strings before)

[Jim Baker]

On Mon, Jul 5, 2021, 2:40 PM Guido van Rossum  wrote:

> FWIW, we could make f-strings properly nest  too, like you are proposing
> for backticks. It's just that we'd have to change the lexer. But it would
> not be any harder than would be for backticks (since it would be the same
> algorithm), nor would it be backward incompatible. So this is not an
> argument for backticks.
>

Good point. At some point, I was probably thinking of backticks without a
tag, since JS supports this for their f-string like scenario. but if we
always require a tag - so long as it's not a prefix already in use (b, f,
r, fr, hopefully not forgetting as I type this email in a parking lot...) -
then it can be disambiguated using standard quotes.


> Separately, should there be a way to *delay* evaluation of the templated
> expressions (like we explored in our private little prototype last year)?
>

I think so, but probably with an explicit marker on *each* deferred
expression. I'm in favor of Julia's expression quote, which generally needs
to be enclosed in parentheses, but possibly not needed in  expression
braces (at the risk of looking like a standalone format spec).
https://docs.julialang.org/en/v1/manual/metaprogramming/

So this would like
x = 42
d = deferred_tag"Some expr: {:(x*2)}"

All that is happening here is that this being wrapped in a lambda, which
captures any scope lexically as usual. Then per that experiment you
mentioned, it's possible to use that scope using fairly standard - or at
least portable to other Python implementations - metaprogramming, including
the deferred evaluation of the lambda.
(No frame walking required!)

Other syntax could work for deferring. Maybe backticks, they could be
available? 

- Jim



> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him **(why is my pronoun here?)*
> 
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/2ECKVDW7A5PHJWJHI6QUS6CPW46X2IHA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Pre PEP: Python Literals (was custom strings before)

[Guido van Rossum]

FWIW, we could make f-strings properly nest  too, like you are proposing
for backticks. It's just that we'd have to change the lexer. But it would
not be any harder than would be for backticks (since it would be the same
algorithm), nor would it be backward incompatible. So this is not an
argument for backticks.

Separately, should there be a way to *delay* evaluation of the templated
expressions (like we explored in our private little prototype last year)?

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/E5JBLCI3J357CCEKNC7ME5ZX6KORPYBD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Pre PEP: Python Literals (was custom strings before)

[Jonathan Goble]

On Mon, Jul 5, 2021 at 4:12 PM Jim Baker  wrote:

> in the same way that we don't use l (that's the lower-case letter L if
> you're not reading this email with the numeric codepoints) as a variable
>

Speaking of grit on one's screen, I first thought that was a lowercase "i"
because I had actual light-colored grit on my screen in precisely the exact
place to make the lowercase L look like a lowercase i until I wiped it off
with my finger.

Which goes to show that we should not underestimate the grit-on-screen
concern. It's real, and it's a huge -1 against backticks for any purpose.
I'm +0 on the overall concept here, but please say no to backticks.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/HGOQKOJBBAEW7UHTGJXDMXD6CROLBYYO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Pre PEP: Python Literals (was custom strings before)

[Chris Angelico]

On Tue, Jul 6, 2021 at 6:10 AM Jim Baker  wrote:
>
> On Mon, Jul 5, 2021, 12:56 PM Barry Scott  wrote:
>>
>>
>>
>> On 5 Jul 2021, at 08:07, Thomas Güttler  wrote:
>>
>> This means backticks, but without the dollar sign.
>>
>>
>> In bash the backtick was so often a problem that $(cmd) was added.
>>
>> Having removes the grit-on-Tim's-screen backtick in python 3 I would
>> not like to see it return with its issue of being confused with single-quote.
>
>
> One mitigation is that the backtick should always require a tag as a prefix. 
> So seeing something like
>
> elem = html`Some item: {value}`
>
> is hopefully fairly obvious what's going on - it's not just going to be mixed 
> up with a single quote. Uses like log(f`foo`) should be hopefully 
> discouraged, in the same way that we don't use l (that's the lower-case 
> letter L if you're not reading this email with the numeric codepoints) as a 
> variable, but we are happy enough to write something like limit = 42 - it's 
> clear in the context.
>

Question: what's the advantage of this magic syntax over something much simpler:

elem = html(i'Some item: {value}')

That avoids the backtick problem because it actually IS an apostrophe.
Or a double quote. Or triple quotes, whatever you want to use. It's
the exact format already used for other special string literal types,
including f-strings.

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/UVKP74PGCWTZSFBDMFMIIWKXMU7PCJ2R/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Pre PEP: Python Literals (was custom strings before)

[Jim Baker]

On Mon, Jul 5, 2021, 12:56 PM Barry Scott  wrote:

>
>
> On 5 Jul 2021, at 08:07, Thomas Güttler  wrote:
>
> This means backticks, but without the dollar sign.
>
>
> In bash the backtick was so often a problem that $(cmd) was added.
>
> Having removes the grit-on-Tim's-screen backtick in python 3 I would
> not like to see it return with its issue of being confused with
> single-quote.
>

One mitigation is that the backtick should always require a tag as a
prefix. So seeing something like

elem = html`Some item: {value}`

is hopefully fairly obvious what's going on - it's not just going to be
mixed up with a single quote. Uses like log(f`foo`) should be hopefully
discouraged, in the same way that we don't use l (that's the lower-case
letter L if you're not reading this email with the numeric codepoints) as a
variable, but we are happy enough to write something like limit = 42 - it's
clear in the context.

However, I think the Bash example is exactly illustrative of the opposite.
So in reviewing at least this one FAQ on  the topic
http://mywiki.wooledge.org/BashFAQ/082, it reminds me of why I don't use
backticks in Bash - it's the lack of nesting support when compared to
$(...).

Ironically, this nesting is exactly what backticks can help here on - an
unused character as of Python 3, commonly used for building some type of
string/identifier in a variety of languages, that we can give some nice
semantics that allows for simple nesting when used in conjunction with
braces delimiting expressions. Such braces of course always give us a new
nesting, similar to the statement in the wiki above that "$() forces an
entirely new context for quoting, so that everything within the command
substitution is protected and can be treated as though it were on its own,
with no special concern over quoting and escaping." That this usage of
backticks has worked quite well for JavaScript provides some useful
confirmation.

I also expect that no one will confuse this with Bash usage, given that's
going to be in Python code - except perhaps in a readily written sh
function (so something like sh`...`, which returns a list of stdout or
something like that and makes use of shlex, etc). I will leave what horrors
that could actually look like to the reader :) although in limited form, it
could be quite useful.

- Jim
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/I7K57YBDLNM5ZAF4SQORFXPG7OIRPE3G/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Pre PEP: Python Literals (was custom strings before)

[Barry Scott]

> On 5 Jul 2021, at 08:07, Thomas Güttler  wrote:
> 
> This means backticks, but without the dollar sign. 

In bash the backtick was so often a problem that $(cmd) was added.

Having removes the grit-on-Tim's-screen backtick in python 3 I would
not like to see it return with its issue of being confused with single-quote.

Barry

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/3EU3RLCFRB3JEQ5Q6O7YMANATFOFYGNY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Pre PEP: Python Literals (was custom strings before)

[Jim Baker]

JavaScript's tagged template literals do provide nice ergonomics. These
should work well with some variation on PEP
501's types.InterpolationTemplate as the backend implementation. Some
thoughts on what that should look like:

* I also increasingly prefer the idea of using backticks to define such
templates, as opposed to further overloading standard quoting. More below.
* We should separate the static elements of the template from evaluated
expressions. This implies that a tagged template function would have a
signature like def html(template, *exprs). To provide the same user
interface as PEP 501, one could write a tag template function "i" that is
an identity function - thus demonstrating a basic equivalence of these
proposals.
* Literal static strings from the templates are available in both raw and
cooked forms (Unicode escapes applied). So there's no need to add a "r"
prefix for raw.
* Expressions are evaluated immediately using normal scoping and rendered
later (taking into account escaping rules, et). Such rendering can be into
any Python object. So using some "html" tag function would return a DOM
(let's call it HtmlElement in this example) which can be further rendered
eventually into returned text in the scenario of serving HTML.
* Greater likelihood of syntax support. Being able to specify html`Some
expr: {x * y}` probably simplifies being able to provide an IDE
extension that is aware that the html tag function was imported from some
popular library, and can therefore provide assistance with HTML tags or any
other aspects of the templating language used.
* Backticks are multiline and they nest in the context of expressions. This
means there's no need to invent new templating syntax, such as Jinja's {%
for ... %} ... {% endfor %} to provide iteration over a sequence. Just use
Python in the evaluated expressions. So with reference to the example in
the Jinja doc https://jinja.palletsprojects.com/en/3.0.x/templates/#synopsis,
we could have functions like the following that can be further composed to
create larger DOM objects:

def ulistify(id: str, linkage: dict[str, str]) -> HtmlElement:
# uses a dict to represent the linkage in this example, so slightly
different than the Jinja example
return html`
{
  html`{caption}` for href, caption in
linkage.items()
}`

It's possible to write something similar using f-string syntax now, but it
involves careful tracking of which quotes are being used to support
nesting. The interaction of backticks and expression syntax keeps it simple
in comparison.

An example library in the JavaScript ecosystem that builds out from the JS
tagged template literal support is Lit (https://lit.dev/,  note that I
haven't used this library.) Lit provides some nice functionality by
composing HTML DOM out of DOM fragments; and doing this in a lazy fashion
(virtual DOM, as ReactJS demonstrated; I am not aware of a fast virtual DOM
library in Python as a C Extension, but it would seem like a
straightforward thing to write). Anyway, Lit and other similar libraries in
JS could help understand the scope of the available innovation if we were
to add such functionality.

Format specifiers and ! conversions should presumably still be available in
expressions and thus available from the template literal object to be used
as part of any rendering. However, I'm leaving this open for the moment as
to exactly what this looks like - especially if I overlooked any parsing
issues for such support!

Lastly, it's worth noting that types.InterpolationTemplate could look quite
similar to https://262.ecma-international.org/6.0/#sec-tagged-templates if
it's a constant object - a template literal in other words - which a tagged
function is then applied to with the evaluated expressions.

- Jim


On Mon, Jul 5, 2021 at 1:10 AM Thomas Güttler 
wrote:

>
> Am Fr., 2. Juli 2021 um 12:06 Uhr schrieb Nick Coghlan  >:
>
>>
>>
>> On Fri, 2 Jul 2021, 5:12 pm Thomas Güttler, 
>> wrote:
>>
>>> Hi Nick and all other Python ideas friends,
>>>
>>> yes, you are right. There is not much difference between PEP-501 or my
>>> proposal.
>>>
>>> One argument why I would like to prefer backticks:
>>>
>>> Some IDEs detect that you want to use a f-string automatically:
>>>
>>> You type:
>>>
>>> name = 'Peter'
>>> print('Hello {name...
>>>
>>> and the IDE automatically adds the missing "f" in front of the string:
>>>
>>> name = 'Peter'
>>> print(f'Hello {name...
>>>
>>> This is a handy feature (of PyCharm), which would not work reliably if
>>> there are two different prefixes.
>>>
>>> ---
>>>
>>> You mentioned these things:
>>>
>>> eager rendering: I think deferred rendering would increase the
>>> complexity a lot. And I think it is not needed.
>>>
>>
>> Eager rendering is f-strings. Any templating proposal necessarily
>> involves a delayed rendering step, when the template is combined with the
>> interpolated values.
>>
>> runtime value interpolation: It is up to the receiver of
>>> types.InterpolationTemplate 

[Python-ideas] Re: Pre PEP: Python Literals (was custom strings before)

[Thomas Güttler]

Am Fr., 2. Juli 2021 um 12:06 Uhr schrieb Nick Coghlan :

>
>
> On Fri, 2 Jul 2021, 5:12 pm Thomas Güttler, 
> wrote:
>
>> Hi Nick and all other Python ideas friends,
>>
>> yes, you are right. There is not much difference between PEP-501 or my
>> proposal.
>>
>> One argument why I would like to prefer backticks:
>>
>> Some IDEs detect that you want to use a f-string automatically:
>>
>> You type:
>>
>> name = 'Peter'
>> print('Hello {name...
>>
>> and the IDE automatically adds the missing "f" in front of the string:
>>
>> name = 'Peter'
>> print(f'Hello {name...
>>
>> This is a handy feature (of PyCharm), which would not work reliably if
>> there are two different prefixes.
>>
>> ---
>>
>> You mentioned these things:
>>
>> eager rendering: I think deferred rendering would increase the complexity
>> a lot. And I think it is not needed.
>>
>
> Eager rendering is f-strings. Any templating proposal necessarily involves
> a delayed rendering step, when the template is combined with the
> interpolated values.
>
> runtime value interpolation: It is up to the receiver of
>> types.InterpolationTemplate to handle the data structure.
>>
>
> I really meant runtime template parsing here (i.e. str.format).
>
>
> dedicated templating libraries: One temp after the other. I think HTML and
>> SQL libraries would adapt as soon as the foundation
>> is available.
>>
>
> The existence of i-strings likely wouldn't change the syntax of jinja2
> templates, Django templates, SQL Alchemy, pandas, etc.
>
> I would be happy if PEP-501 would come true.
>>
>
> So would I, but I still don't have a compelling answer to the "but it's
> yet another subtly different way to do it" objection.
>


Today I read the replies to this thread again.

Of course there where some "-1" replies, but overall there was positive
feedback.

Today I played with template literals from Javascript:

[image: image.png]

I like it.

Nick, would you be open to adapting to the JS syntax for PEP-501?

I propose `...{var} ...`

This means backticks, but without the dollar sign.

I would make it like in JS: The string in backticks can span several lines.

So what is the next step now?

Regards,
  Thomas
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/NRJ7NBRF6KTFWP6SFQLH4FIEK3P2W2AT/
Code of Conduct: http://python.org/psf/codeofconduct/