Hello Paul (and everyone else), On Sun, Nov 29, 2020 at 12:00 PM Paul Sokolovsky <pmis...@gmail.com> wrote:
> ... > > > # Experimental Syntax Proposal > > > > I would like to propose that Python adopts a modified process > > before introducing significant changes of its syntax. > > [] > > > ## New suggested process > > > > Assuming one of the options mentioned above is adopted, > > before changes to the syntax are introduced in a Python version, > > they would be made available to be used either as an encoding variant > > or an import hook giving enough time for interested pythonistas to > > experiment with the new syntax, writing actual code and possibly > > trying out various alternatives. > > Thanks for posting this proposal. It should be pretty clear that's > the best process which should be followed. Thank you. > It's also should be pretty > clear that (almost) nobody follows it. If anything, lack of responses > by the substance of the matter proposed, i.e. the *process*, not > technical means to achieve it, is very telling. > I had taken the lack of response as indicative that my idea was simply "wrong". > > So again, I fully agree that trying to implement any Python changes in > the Python itself (instead of rushing to hack a particular > implementation in C) should the best practice. > > Now let's talk why it's not that way. The main reasons would be lack of > technical best practices to achieve that, and perceived complexity of > existing ways to do that. > > Even from your proposal lack of the "best" solution is clear: > > > * By using an import hook. This is the most powerful approach as it > > allows one to do changes either at the source level > > (prior to parsing), or at the AST level, or both. > > > > * By using a custom encoding. This only allows transformations at the > > source level. > > So, "custom encoding" way is essentially a hack. As it operates on the > surface "stream of characters" representation of a program, only > trivial, or imprecise transformations can be implemented. > Aside: some fairly complex transformations can be implemented using codecs; see https://github.com/dropbox/pyxl for example. > Alternatively, a program can be parsed, modified and dumped again as a > stream of characters, just to immediately be parsed by the Python > interpreter again. In either case, it's a hack, and should be rejected > as a viable approach to a problem domain. > > Now you say that the second choice is "import hooks". But import hooks > in Python is its own kingdom of "wonderland". Import hooks aren't > intended to "allow to experiment with syntax", they are intended to do > anything what's possible with module loading (and many things that > aren't possible, I would add). So, they have rather complex, > non-intuitive API, thru which anyone would need to wade, neck-deep, > before implementing something useful. > In my original email, I linked to a project I created ("Ideas": https://aroberge.github.io/ideas/docs/html/) which meant to simplify this as much as possible. > > In other words, both choices you list aren't really viable to experiment > with syntax/semantics. I respectfully disagree, based on my own experimentation, documented in "Ideas" mentioned previously. For example, I implemented module level constants using this approach. They *almost* work ... except for some corner cases due to the fact that I am using module objects created using Python's type() as I have not figured out (yet?) how to create a module object with a custom dict. > That may be a good explanation why nobody rushes > to. > > I'd formulate choices to experiment with syntax/semantics differently. > Let's remember the compilation pipeline: the source is tokenized, then > it's parsed into AST, then AST is compiled into bytecode. > > I tried to represent this visually on https://aroberge.github.io/ideas/docs/html/possible.html > 1. The central part of the pipeline is the AST. Many interesting > features can be implemented on AST level. Others can be at least > prototyped, using existing syntactic elements (by assigning to them > new semantics). Of course, arbitrary syntax changes can't be > implemented this way. Exactly. Which is why I suggested using either import hooks or encodings. [Snip: explanation leading to discussion of Python2 "compiler" package which is referred to again at the end] > What's needed is higher-level API specifically for the usecase of > letting Python source of modules to run thru custom > tokenizer/parser/bytecode compiler. The best known (non-adhoc) approach > to that is PEP511: https://www.python.org/dev/peps/pep-0511/. I believe that the intention of PEP-511 was to optimize the code generated, and not to experiment with different syntax. I *think* that my ideas project is essentially a superset of what is described in pep-511, but with a completely different goal. Best, André > > -------- > > This turned out to be a long intro. As I said, I fully agree with you, > that changes to Python should be prototyped in Python. And there's no > better way to "agree" than actually dogfood this approach to oneself. > Yes. One additional comment, point of emphasis about my original proposal. Take PEP 505 (None-aware operators) written 5 years ago. It is still deferred. Imagine that someone wants to experiment with a custom implementation. Using the approach I mentioned, it should be possible to do so. Suppose that the same person wishes to also experiment with block-level scoping. By using the approach I mentioned in my original message, it would be possible to "stack" transformations and experiment with code that support either None aware operators and/or block-level scoping. Different syntax variants could be implemented separately and, provided they do not create conflicts, could be tested together so that people could get a feel as to whether or not some proposed change should be included in Python. > > As I lately argued here on the list that implementing block-level > scoping for Python is "not rocket science at all" and "a notch above > trivial", I decided to do just that - and subject myself to coding it > up, using the very python-compiler project I mentioned above. The > result is a branch on that repo: > https://github.com/pfalcon/python-compiler/tree/for-block-scope > > (I'll post a separate message with details.) > I look forward to reading it. Best regards, André > > > > > -- > Best regards, > Paul mailto:pmis...@gmail.com >
_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/S3R5IIX3QKUOUA53JINSRUPFXG62XA57/ Code of Conduct: http://python.org/psf/codeofconduct/
