On 15 June 2013 14:08, Greg Ewing <greg.ew...@canterbury.ac.nz> wrote: > Guido van Rossum wrote: >> >> Not a bug. The same is done for file input -- CRLF is changed to LF before >> tokenizing. > > > I'm not convinced it's reasonable behaviour to re-scan the > string as though it's being read from a file. It's a Python > string, so it's already been through whatever line-ending > transformation is appropriate to get it into memory.
No, that's not the way the Python compiler works. The transformation Guido is talking about is the way the tokenizer identifiers "NEWLINE" tokens: >>> list(tokenize.tokenize((l for l in (b"""'\r\n'""", b"")).__next__))[2] TokenInfo(type=4 (NEWLINE), string='\r\n', start=(1, 1), end=(1, 3), line="'\r\n'") This long predates universal newlines mode - it's part of the compilation process, not part of the IO system. The compiler then sees the NEWLINE token in the tokenizer output, and inserts a "\n" into the triple-quoted string. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
