On Tue, Sep 15, 2020, at 23:54, Guido van Rossum wrote:
> On Tue, Sep 15, 2020 at 7:31 PM Benjamin Peterson <benja...@python.org> wrote:
> > On Tue, Sep 15, 2020, at 20:10, Greg Ewing wrote:
> > > Maybe the PEP should propose an AST of its own, which would initially
> > > be a third thing separate from either of the existing ones, with
> > > the possibility of adopting it as the ast module representation
> > > some time in the future.
> >
> > The rust approach to this problem to pass only a token stream to the macro.
> > Then, the macro writer brings their own parser.
>
> And (I looked this up [1]) it returns another token stream which is
> then re-parsed by the standard parser. I'm not quite sure who is
> responsible for determining where the macro ends -- the standard parser
> or the macro parser.
The standard parser determines the macro boundaries and passes the macro
implementation only the relevant tokens.
>
> A nice side effect of this would be that (hopefully) the macro syntax
> doesn't need to have the ugly special cases needed to allow `import!
> foo as bar` and `from! mod import foo as bar` (though these still need
> to be predefined macros).
>
> I'm not entirely sure how this would preserve the hygiene of the macros
> though, since the macro's parser could just do a token-by-token
> substitution without checking the context, effectively what a macro in
> C does. Forcing an AST on the macros restricts macros somewhat (e.g. it
> requires the input arguments to look like well-formed expressions) but
> that's probably a good thing.
The rust macro TokenStream is a bit higher level than a traditional token
stream. In particular, the macro implementation can request that new
identifiers it inserts into the stream are only resolvable within the macro for
hygiene. There's a bit about that here:
https://doc.rust-lang.org/proc_macro/struct.Ident.html#impl
>
> (Hm, or not. How would you implement something like PEP 505 using
> macros? It introduces new tokens like `?.` and `??`.
Is there any proposal that would let you add new tokens?
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/GFDSVQ4Q7KBPUOKCIPGU724ANQZGVWV3/
Code of Conduct: http://python.org/psf/codeofconduct/
