> It would be trivial to add another yield to tokenize.py when
> the backslah is detected

+1



> I think that it should probably yield a single NL pseudo-token
> whose value is a backslash followed by a newline; or perhaps it
> should yield the backslash as a comment token, or as a new token.

The first option is likely the most compatible with existing uses of tokenize.  
If a comment token were emitted, an existing colorizer or pretty-printer would 
markup the continuation as a comment (possibly not what the tool author 
intended).  If a new token were created, it might break if-elif-else chains in 
tools that thought they knew the universe of possible token types.


Raymond
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Reply via email to