[issue25661] tokenize.untokenize does not maintain the order of tabbed indentations and leading spaces

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Python 2.7 is no longer supported.

--
nosy: +serhiy.storchaka
resolution:  -> out of date
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25661] tokenize.untokenize does not maintain the order of tabbed indentations and leading spaces

[Henk-Jaap Wagenaar]

Henk-Jaap Wagenaar  added the comment:

Python 2.7 is end-of-life, see e.g. https://www.python.org/doc/sunset-python-2/ 
so this is unlikely to be fixed.

This behaviour continues on Python 3 it seems (I used 3.8.1), *however* on 
Python 3 this is not valid code as it mixes spaces and tabs, see e.g. 
https://www.python.org/dev/peps/pep-0008/#tabs-or-spaces

I think therefore this can be closed, though I might well be mistaken.

--
nosy: +Henk-Jaap Wagenaar

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25661] tokenize.untokenize does not maintain the order of tabbed indentations and leading spaces

[Joseph Gordon]

Change by Joseph Gordon :


--
nosy:  -josephgordon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25661] tokenize.untokenize does not maintain the order of tabbed indentations and leading spaces

[Joseph Gordon]

Changes by Joseph Gordon :


--
nosy: +josephgordon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25661] tokenize.untokenize does not maintain the order of tabbed indentations and leading spaces

[Blake Hilliard]

New submission from Blake Hilliard:

Here's a simple test case that shows the bug:

# python
Python 2.7.9 (default, Apr  2 2015, 14:49:18) 
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import StringIO, tokenize
>>> code = "if False:\n\tx=3\n \ty=3\n"
>>> g = list(tokenize.generate_tokens(StringIO.StringIO(code).readline))
>>> assert tokenize.untokenize(g) == code
Traceback (most recent call last):
  File "", line 1, in 
AssertionError


This is very similar to: https://bugs.python.org/issue20387

I'm using the patch that was provided to fix that bug, but with the patch this 
slightly different test case still fails. On the last line of "code" from my 
example, where the line begins with a single space then a tab, the untokenized 
string reverses the order, and the tab precedes the space. This generates code 
that will throw an IndentationError if run.

--
components: Library (Lib)
messages: 254857
nosy: Blake Hilliard
priority: normal
severity: normal
status: open
title: tokenize.untokenize does not maintain the order of tabbed indentations 
and leading spaces
type: behavior
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com