On 16.09.2020 16:33, Benjamin Peterson wrote:
Is there any proposal that would let you add new tokens?
Not a proposal, but drafted a project a while back. By patching tokenizer and the parser at runtime (generated from lib2to3.pgen), allowed to introduce different tokens / rules, and worked like a transpiler (so that you can take the source code with creating a fake encoding, giving it as a raw source stream to the new parser with updated rules and then return the result as the decoded source);
https://github.com/isidentical/freesyntax/blob/master/example2.py _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/SBQUKFSJDXOJBPXM42RHRNDJAB3J4I2J/ Code of Conduct: http://python.org/psf/codeofconduct/
