Albert-Jan Nijburg added the comment: > I prefer to add tokenize tokens directly in Lib/token.py, and then get > COMMENT, NL and ENCODING using tok_name.index().
That would make more sense from a breaking change perspective, but we would step on the toes of anyone adding `COMMENT`, `NL`, or `ENCODING` to `token.h` because `token.py` is generated from that. It would also make much more sense to have them as fields on `token` if they are in `tok_name` in `token`. ---------- nosy: +Albert-Jan Nijburg _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25324> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
