[issue16221] tokenize.untokenize() "compat" mode misses the encoding when using an iterator

[Irit Katriel]

Irit Katriel  added the comment:

This was fixed here under issue719888.

--
resolution:  -> out of date
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16221] tokenize.untokenize() "compat" mode misses the encoding when using an iterator

[Irit Katriel]

Irit Katriel  added the comment:

I think this is fixed now - the tests in the patch are passing.

--
nosy: +iritkatriel

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16221] tokenize.untokenize() "compat" mode misses the encoding when using an iterator

[Mark Lawrence]

Changes by Mark Lawrence :


--
nosy:  -BreamoreBoy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16221] tokenize.untokenize() "compat" mode misses the encoding when using an iterator

[Eric Snow]

Changes by Eric Snow :


--
assignee: eric.snow -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16221] tokenize.untokenize() compat mode misses the encoding when using an iterator

[Mark Lawrence]

Mark Lawrence added the comment:

#16224 has been closed as a duplicate of #8478.

--
nosy: +BreamoreBoy
status: pending - open

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16221
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16221] tokenize.untokenize() compat mode misses the encoding when using an iterator

[Eric Snow]

Eric Snow added the comment:

issue16224 _may_ supercede this ticket.

--
components: +Library (Lib)

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16221
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16221] tokenize.untokenize() compat mode misses the encoding when using an iterator

[Eric Snow]

Eric Snow added the comment:

The patch that I have in #16224 takes care of this issue.  If that issue goes 
in another direction however...

--
status: open - pending
superseder:  - tokenize.untokenize() misbehaves when moved to compatiblity 
mode

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16221
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16221] tokenize.untokenize() compat mode misses the encoding when using an iterator

[Eric Snow]

New submission from Eric Snow:

While traversing the passed iterable, untokenize() will go to compatibility 
mode one it hits a 2-tuple instead of a 5-tuple (coming from the iterable).  
That token will not go through the normal steps that other tokens do.  Most 
critically, if that token is the ENCODING token, the source is not encoded to 
that encoding at the end (even though it should have been).  I'll have a patch 
up in a moment (for the tip of default).

There are a couple of other issues at hand that I will bring up separately.

--
assignee: eric.snow
messages: 172848
nosy: eric.snow
priority: normal
severity: normal
stage: test needed
status: open
title: tokenize.untokenize() compat mode misses the encoding when using an 
iterator
type: behavior
versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16221
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16221] tokenize.untokenize() compat mode misses the encoding when using an iterator

[Eric Snow]

Changes by Eric Snow ericsnowcurren...@gmail.com:


--
keywords: +patch
stage: test needed - patch review
Added file: http://bugs.python.org/file27559/untokenize_compat.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16221
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com