Trent Nelson <[EMAIL PROTECTED]> added the comment: I've attached a patch to test_tokenizer.py and a bunch of text files (that should be dropped into Lib/test) that highlight this issue a *lot* better than the current state of affairs.
The existing implementation defines roundup() in the doctest, then proceeds to define it again in the code body. The last for loop in the doctest is failing every so often -- what it's failing on isn't at all clear as a) ten random files are selected out of 332 in Lib/test, and b) there's no way of figuring out which files are causing it to fail unless you hack another method into the test case to try and replicate what the doctest is doing, with some additional print statements (which is the approach I took, only to get bitten by the fact that roundup() was being resolved to the bogus definition that's in the code body, not the functional one in the doctest, which resulted in even more misleading behaviour). FWIW, the file that causes the exception is test_doctest2.py as it contains encoded characters. So, the approach this patch takes is to drop the 'pick ten random test files and untokenize/tokenize' approach and add a class that specifically tests for the tokenizer's compliance with PEP 0263. I'll move on to a patch to tokenizer.py now, but this patch is ok to commit now -- it'll clean up the misleading errors being reported by the plethora of red 3.0 buildbots at the moment at the very least. ---------- nosy: +Trent.Nelson ____________________________________ Tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue719888> ____________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
