Nick Coghlan <[email protected]> added the comment:
Ah, I didn't read it as suggested documentation at all - you moved seamlessly
from personal commentary to a docs suggestion without separating the two, so it
appeared to be a complete non sequitur to me.
As for the docs suggestion, I think it works as the explanation of which tokens
are affected once the concept of the token stream simplification is introduced:
=====
To simplify token stream handling, all literal tokens (':', '{', etc) are
returned using the generic 'OP' token type. This allows them to be simply
handled using common code paths (e.g. for literal transcription directly from
input to output). Specific tokens can be distinguished by checking the "string"
attribute of OP tokens for a match with the expected character sequence.
The affected tokens are all symbols and symbol combinations that are
syntactically significant in expressions (as listed in the token module).
Anything which is not an independent token (i.e. '#' which marks comments, '_'
which is just part of a name, '\' which is used for line continuations, the
contents of string literals and any symbols which are not a defined part of
Python's syntax) is completely unaffected by this difference in behaviour.
===========
If "exact_type" is introduced in 3.3, then the first paragraph can be adjusted
accordingly.
----------
_______________________________________
Python tracker <[email protected]>
<http://bugs.python.org/issue2134>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
