[issue3028] tokenize module: normal lines, not logical

2008-06-03 Thread Senthil
Senthil [EMAIL PROTECTED] added the comment: The documentation of the tokenize module says: The line passed is the *logical* line; continuation lines are included. I suggest that this will be changed to something like The line passed is the index of the string returned by the readline

[issue3028] tokenize module: normal lines, not logical

2008-06-02 Thread Noam Raphael
New submission from Noam Raphael [EMAIL PROTECTED]: Hello, The documentation of the tokenize module says: The line passed is the *logical* line; continuation lines are included. Some background: The tokenize module splits a python source into tokens, and says for each token where it begins and