I'm trying to add a '?' token to the parser, and weird things are happening.
I've added a #define to token.h, an entry to _PyParser_TokenNames in tokenizer.c and case for it in PyToken_OneChar(). But it's behaving as though the tokenizer is not recognising my token. I put in some printfs to find out whether PyToken_OneChar() is recognising it. The results are confusing: while running pgen, PyToken_OneChar() is being called and recognising the new token correctly. However, it doesn't seem to be getting called *at all* when parsing Python code. I don't see how this can happen, because pgen seems to use the same tokenizing code as Python itself. Is there anything else I need to do? Does some file need to be manually re-made? -- Greg _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
