[issue2495] tokenize doesn't handle __future__.unicode_literals correctly

Change by Łukasz Langa : -- pull_requests: +6271 ___ Python tracker ___ ___ Python-bugs-list

[issue2495] tokenize doesn't handle __future__.unicode_literals correctly

New submission from Christian Heimes [EMAIL PROTECTED]: See r61976 Clear the blacklist and run the test with ./python Lib/test/regrtest.py -uall test_tokenize to reproduce the issue. -- components: Library (Lib) messages: 64582 nosy: tiran priority: normal severity: normal status:

[issue2495] tokenize doesn't handle __future__.unicode_literals correctly

Amaury Forgeot d'Arc [EMAIL PROTECTED] added the comment: Actually, the problem is that untokenize does not put spaces between two consecutive string literals: '' '' = Corrected with r61979. Will backport -- nosy: +amaury.forgeotdarc resolution: - fixed status: open - closed