I think it is a relic from the distant past, when the lexer did generate NEWLINE for every blank line. I think the only case where you can still get a NEWLINE by itself is in interactive mode. This code is extremely convoluted and may be buggy in end cases; this could explain why you get a continuation prompt after entering a comment in interactive mode...
--Guido On 10/6/05, Michael Hudson <[EMAIL PROTECTED]> wrote: > "Matthew F. Barnes" <[EMAIL PROTECTED]> writes: > > > I posted this question to python-help, but I think I have a better chance > > of getting the answer here. > > > > I'm looking for clarification on when NEWLINE tokens are generated during > > lexical analysis of Python source code. In particular, I'm confused about > > some of the top-level components in Python's grammar (file_input, > > interactive_input, and eval_input). > > > > Section 2.1.7 of the reference manual states that blank lines (lines > > consisting only of whitespace and possibly a comment) do not generate > > NEWLINE tokens. This is supported by the definition of a suite, which > > does not allow for standalone or consecutive NEWLINE tokens. > > > > suite ::= stmt_list NEWLINE | NEWLINE INDENT statement+ DEDENT > > I don't have the spare brain cells to think about your real problem > (sorry) but something to be aware of is that the pseudo EBNF of the > reference manual is purely descriptive -- it is not actually used in > the parsing of Python code at all. Among other things this means it > could well just be wrong :/ > > The real grammar is Grammar/Grammar in the source distribution. > > Cheers, > mwh > > -- > The Internet is full. Go away. > -- http://www.disobey.com/devilshat/ds011101.htm > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
