[issue17061] tokenize unconditionally emits NL after comment lines & blank lines

2015-10-04 Thread Martin Panter
Martin Panter added the comment: The plain Python shell does respond to lines with only a comment and/or horizontal space with a continuation prompt. It only treats completely blank lines without any horizontal space specially: >>> ... # Indented blank line above; completely blank line

[issue17061] tokenize unconditionally emits NL after comment lines blank lines

2013-02-13 Thread Thomas Kluyver
Thomas Kluyver added the comment: Hmm, that's interesting. For our purposes, a blank line or a comment line shouldn't result in a continuation prompt. This is consistent with what the plain Python shell does. As part of this, we're tokenizing the code, and if the final \n results in a NL

[issue17061] tokenize unconditionally emits NL after comment lines blank lines

2013-02-02 Thread Terry J. Reedy
Changes by Terry J. Reedy tjre...@udel.edu: -- nosy: +meador.inge ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17061 ___ ___ Python-bugs-list

[issue17061] tokenize unconditionally emits NL after comment lines blank lines

2013-02-02 Thread Meador Inge
Meador Inge added the comment: The current behavior seems consistent with the lexical definition for blank lines [1]: A logical line that contains only spaces, tabs, formfeeds and possibly a comment, is ignored (i.e., no NEWLINE token is generated). NL and COMMENT are used for items that the

[issue17061] tokenize unconditionally emits NL after comment lines blank lines

2013-01-28 Thread Thomas Kluyver
New submission from Thomas Kluyver: The docs describe the NL token as Token value used to indicate a non-terminating newline. The NEWLINE token indicates the end of a logical line of Python code; NL tokens are generated when a logical line of code is continued over multiple physical lines.