Terry J. Reedy added the comment:
I read the manual more carefully and noticed that the guarantee is that
tokenizing the result of untokenize matches the input to untokenize. " The
result is guaranteed to tokenize back to match the input so that the conversion
is lossless and round-trips are assured."
I ran the test in 3.4 and noticed the exactness and direction of the matching
does not matter because only the 2nd \t is replaced by a space, making the
invalid code that raises IndentationError.
So this is definitely a bug and I withdraw the close suggestion.
I believe the test should be that both lines of the body begin with the same
indent.
from tokenize import tokenize, untokenize
import io
def test_tab_indent(self):
code = b"if False:\n\tx=3\n\ty=3\n"
codelines = untokenize(tokenize(io.BytesIO(s).readline)).split(\n)
assertEqual(codelines[1], codelines[2])
Of course, the test of tokenize and untokenize should be separated.
----------
stage: -> needs patch
versions: +Python 3.3
_______________________________________
Python tracker <[email protected]>
<http://bugs.python.org/issue20387>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
