Terry J. Reedy tjre...@udel.edu added the comment:
I have not looked at this, but a new parameter would be a new feature. Its a
moot point until there is an agreed on patch for a current version.
--
nosy: +tjreedy
type: behavior - feature request
versions: +Python 3.2 -Python 2.4,
A.M. Kuchling li...@amk.ca added the comment:
Unfortunately I think this will break many users of tokenize.py.
e.g. http://browsershots.googlecode.com/svn/trunk/devtools/pep8/pep8.py
has code like:
if (token_type == tokenize.OP and text in '([' and ...):
If tokenize now returns LPAR, this
Guilherme Polo added the comment:
I'm attaching a patch that solves this and updates tests.
Added file: http://bugs.python.org/file9459/tokenize_r60884.diff
__
Tracker [EMAIL PROTECTED]
http://bugs.python.org/issue2134
__
New submission from Guilherme Polo:
function generate_tokes at tokenize.py yields token OP (51) for colon,
while it should be token COLON (11). It probably affects other python
versions as well.
I'm attaching a minor sample that demonstrates this, running it returns
the following output:
1
