On Thu, Jul 8, 2021 at 6:25 PM Nick Coghlan <ncogh...@gmail.com> wrote:
> On Tue, 6 Jul 2021, 7:56 am Jim Baker, <jim.ba...@python.org> wrote: > >> >> >> On Mon, Jul 5, 2021, 2:40 PM Guido van Rossum <gu...@python.org> wrote: >> >>> FWIW, we could make f-strings properly nest too, like you are proposing >>> for backticks. It's just that we'd have to change the lexer. But it would >>> not be any harder than would be for backticks (since it would be the same >>> algorithm), nor would it be backward incompatible. So this is not an >>> argument for backticks. >>> >> >> Good point. At some point, I was probably thinking of backticks without a >> tag, since JS supports this for their f-string like scenario. but if we >> always require a tag - so long as it's not a prefix already in use (b, f, >> r, fr, hopefully not forgetting as I type this email in a parking lot...) - >> then it can be disambiguated using standard quotes. >> > > There's a deferred PEP proposing a resolution to the f-string nesting > limitations: > https://www.python.org/dev/peps/pep-0536/#motivation > > Thanks for pointing that PEP out - this looks quite reasonable for both f-strings and for the tagged templates proposed here. I believe some limitations on nesting expressions in f-strings with quote changing was introduced in a relatively recent fix in 3.8 or 3.9, but I need to find the specifics. >> >>> Separately, should there be a way to *delay* evaluation of the templated >>> expressions (like we explored in our private little prototype last year)? >>> >> >> I think so, but probably with an explicit marker on *each* deferred >> expression. I'm in favor of Julia's expression quote, which generally needs >> to be enclosed in parentheses, but possibly not needed in expression >> braces (at the risk of looking like a standalone format spec). >> https://docs.julialang.org/en/v1/manual/metaprogramming/ >> >> So this would like >> x = 42 >> d = deferred_tag"Some expr: {:(x*2)}" >> >> All that is happening here is that this being wrapped in a lambda, which >> captures any scope lexically as usual. Then per that experiment you >> mentioned, it's possible to use that scope using fairly standard - or at >> least portable to other Python implementations - metaprogramming, >> including the deferred evaluation of the lambda. >> (No frame walking required!) >> >> Other syntax could work for deferring. >> > > It reminds me of the old simple implicit lambda proposal: https://www. > python.org/dev/peps/pep-0312/ > > The old proposal has more ambiguities to avoid now due to type hinting > syntax but a parentheses-required version could work: "(:call_result)" > > PEP 312 - suitably modified - could be quite useful for both deferring evaluation in tagged templates and function calls (or other places where we might want to put in a lambda). Ideally we can use minimum parens as well, so it's tag"{:x+1}" f(:x+1, :y+2) (but I haven't looked at ambiguities here). A further idea that might make the approach in PEP 312 more powerful than simply being an implicit lambda is if this was like Julia's quoted expressions https://docs.julialang.org/en/v1/manual/metaprogramming/#Quoting and we recorded the *text body of the expression*. (Such recording would presumably be done in any template scheme, but let's make this explicit now.) So let's call this implicit lambda with recorded text body a *quoted expression*. What's nice about doing such quoted expressions is that the lambda also captures any lexical scope. So if I have an expression tag"{:x*y}", the variables x and y are referenced appropriately. This means it would be possible to do such things as rewrite complex expressions - eg an index query on a Pandas data frame - while avoiding the use of dynamic scope. I wrote a small piece of code to show how this can be exercised. The text of the expression is simply an attribute on the lambda named "body", but we might have a new type defined (eg types.QuotedExpression). ``` from functools import update_wrapper from textwrap import dedent from types import FunctionType def rewrite(f, new_body): # Create a temporary outer function with arguments for the free variables of # the original lambda wrapping an expression. This approach allows the new # inner function, which has a new body, to compile with its dependency on # free vars. # # When called, this new inner function code object can continue to access # these variables from the cell vars in the closure - even without this # outer function, which we discard. # # This rewriting is generalizable to arbitrary functions, although the # syntax we are exploring is only for expressions. # # A similar idea - for monkeypatching an inner function - is explored in # https://stackoverflow.com/a/27550237 by Martijn Pieters' detailed answer. # # Related ideas include https://en.wikipedia.org/wiki/Lambda_lifting and # lambda dropping where the free vars are lifted up/dropped from the # function parameters. This requires the (usual :) extra level of # indirection by another function. scoped = dedent(f""" def outer({", ".join(f.__code__.co_freevars)}): def inner(): return {new_body} """) capture = {} exec(scoped, f.__globals__, capture) # NOTE: outer is co_consts[0], inner is co_consts[1] - this may not be # guaranteed by the compiler, so iterate over if necessary. inner_code = capture["outer"].__code__.co_consts[1] new_f = FunctionType( inner_code, f.__globals__, closure=f.__closure__) update_wrapper(new_f, f) new_f.body = new_body return new_f def test_rewrite(): x = 2 y = 3 # x, y and free vars in scopeit, and its own nested lambdas, assigned # to f and g below. We will also introduce an additional free var z # in the parameter of scopeit. def scopeit(z): f = lambda: x * y + z f.body = "x * y + z" print(f"{f()=}") # We can do an arbitrary manipulation on the above expression, so # long as we use variables that are in the original scope (or symtab). # Note that adding new variables will look up globally (may or may # not exist, if not a NameError is raised). print(f.__code__.co_freevars) g = rewrite(f, "-(x * y + z)") print(f"{g()=}") scopeit(5) if __name__ == "__main__": test_rewrite() ``` Similar ideas were explored in https://github.com/jimbaker/fl-string-pep/issues - Jim On Thu, Jul 8, 2021 at 6:25 PM Nick Coghlan <ncogh...@gmail.com> wrote: > On Tue, 6 Jul 2021, 7:56 am Jim Baker, <jim.ba...@python.org> wrote: > >> >> >> On Mon, Jul 5, 2021, 2:40 PM Guido van Rossum <gu...@python.org> wrote: >> >>> FWIW, we could make f-strings properly nest too, like you are proposing >>> for backticks. It's just that we'd have to change the lexer. But it would >>> not be any harder than would be for backticks (since it would be the same >>> algorithm), nor would it be backward incompatible. So this is not an >>> argument for backticks. >>> >> >> Good point. At some point, I was probably thinking of backticks without a >> tag, since JS supports this for their f-string like scenario. but if we >> always require a tag - so long as it's not a prefix already in use (b, f, >> r, fr, hopefully not forgetting as I type this email in a parking lot...) - >> then it can be disambiguated using standard quotes. >> > > There's a deferred PEP proposing a resolution to the f-string nesting > limitations: > https://www.python.org/dev/peps/pep-0536/#motivation > > >> >>> Separately, should there be a way to *delay* evaluation of the templated >>> expressions (like we explored in our private little prototype last year)? >>> >> >> I think so, but probably with an explicit marker on *each* deferred >> expression. I'm in favor of Julia's expression quote, which generally needs >> to be enclosed in parentheses, but possibly not needed in expression >> braces (at the risk of looking like a standalone format spec). >> https://docs.julialang.org/en/v1/manual/metaprogramming/ >> >> So this would like >> x = 42 >> d = deferred_tag"Some expr: {:(x*2)}" >> >> All that is happening here is that this being wrapped in a lambda, which >> captures any scope lexically as usual. Then per that experiment you >> mentioned, it's possible to use that scope using fairly standard - or at >> least portable to other Python implementations - metaprogramming, including >> the deferred evaluation of the lambda. >> (No frame walking required!) >> >> Other syntax could work for deferring. >> > > It reminds me of the old simple implicit lambda proposal: > https://www.python.org/dev/peps/pep-0312/ > > The old proposal has more ambiguities to avoid now due to type hinting > syntax but a parentheses-required version could work: "(:call_result)" > > Cheers, > Nick. > >>
_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/LWQVMYRZVC6C7K477V7I3L4O3QQIEWPJ/ Code of Conduct: http://python.org/psf/codeofconduct/
