[issue19335] codeop misclassifies incomplete code with 'nonlocal'

[Terry J. Reedy]

Terry J. Reedy  added the comment:

I reread Nick's comment "the C level loop simply blocks on stdin, waiting until 
the parser spits out a complete AST."  I interpret that now as meaning that the 
REPL compiles user code only once per statement, which IDLE and 
code.InteractiveInterpreter compile a statement once per line, which codeop 
expands to 3 compiles in an attempt to determine whether more is needed.  This 
can all be considered to be a hack, so adding 1 more does not bother me now.

(Side note: if the first compile succeeds in producing a code object, it is 
unconditionally returned after the additional compiles, so they seem like a 
waste.)

I looked at a git blame listing for codeop.  It was extracted from code by 
Guido in 1998 so that Jython (JPython) could replace compile_command.  It was 
not revised after the addition of nonlocal broke it.

I traced the flow of calls from when a user hits  in the entry area to 
the call of compile, and of the returns back.  The attached 
idle-shell-compile.txt records what I found.  Of particular interest to me:

1. IDLE strips trailing ' 's, '\t's, and 1 '\n' from user input before sending 
it to be compiled. (This goes back to the original IDLE commit in 2000.) So 
there is a trailing '\n' only when the user has hit '' twice to complete 
a compound statement.  It would solve this issue *for IDLE* to only print the 
nonlocal error message when there is a trailing '\n'.  I am willing to give up 
the special casing needed to get 'def a():\n  nonlocal c' (no double , 
no trailing '\n') to raise immediately.

Nothing in code says that it expects source to be stripped as IDLE does.  The 
reason for the latter might just be to get rid of the 'smart indent' that will 
precede the first Enter, and possibly anything the user adds before the second. 
 

2. compile is passed the undocumented flag PyCF_DONT_IMPLY_DEDENT = 0x200.  Can 
anyone explain the full intent?  I found this difference.

>>> compile("def f():\n  nonlocal c", '', 'single')
Traceback (most recent call last):
  File "", line 1, in 
  File "", line 2
SyntaxError: no binding for nonlocal 'c' found

# Same code, flag added, different message.
>>> compile("def f():\n  nonlocal c", '', 'single', 0x200)
Traceback (most recent call last):
  File "", line 1, in 
  File "", line 2
nonlocal c
 ^
SyntaxError: unexpected EOF while parsing

# Add '\n' and message is same as without flag.
>>> compile("def f():\n  nonlocal c\n", '', 'single', 0x200)
Traceback (most recent call last):
  File "", line 1, in 
  File "", line 2
SyntaxError: no binding for nonlocal 'c' found

To use the message difference, the first compile-with-flag message would have 
to be saved for comparison, and the difference would only appear if source had 
no trailing '\n'.  We could make that be true, or we could wait until another 
user of codeop complains.

--
Added file: https://bugs.python.org/file49282/idle-shell-compile.txt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19335] codeop misclassifies incomplete code with 'nonlocal'

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

first case -> second case

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19335] codeop misclassifies incomplete code with 'nonlocal'

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> At least the fact that the SyntaxError message is some for both of

Well, is the same because the semantics are the same: a symbol is missing in 
the expected scopes. The only difference is that in the first case is not even 
possible to "fix it" because that scope cannot exist if the function is not 
'nested'. But I do agree that it may be some value on specializing the error 
message to that case (but is not enough to have the nesting, you would need 
also to add a variable - the error is really 'you need a symbol in an 
intermediate scope' -> 'Oh, i don't even have an intermediate scope' -> 'then 
*first* you need to add an intermediate scope). Although this particular 
problem is indeed different from the main one (the one about codeop).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19335] codeop misclassifies incomplete code with 'nonlocal'

[Lysandros Nikolaou]

Lysandros Nikolaou  added the comment:

some -> same

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19335] codeop misclassifies incomplete code with 'nonlocal'

[Lysandros Nikolaou]

Lysandros Nikolaou  added the comment:

That's right, but the SyntaxError message is somewhat confusing. At least the 
fact that the SyntaxError message is some for both of

def a():
def b():
nonlocal x

and 

def a():
nonlocal x

seems a bit misleading. No?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19335] codeop misclassifies incomplete code with 'nonlocal'

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

>1) We should alter code in symtable.c to check whether the namespace in which 
>the nonlocal statement appears in is a function block and whether it is nested 
>or not.

Also, the 'nesting' has very specific meanings in the symbol table: scoping. 
The validity of the keyword translates there if is able to find a variable in 
the enclosing intermediate scope (when its not the global one).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19335] codeop misclassifies incomplete code with 'nonlocal'

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> 1) We should alter code in symtable.c to check whether the namespace in which 
> the nonlocal statement appears in is a function block and whether it is 
> nested or not.

We already raise in this case. Consider this 'test.py' file:

x = 34
def f():
nonlocal x
x = 24

f()

Executing python test.py:


  File "test.py", line 3
nonlocal x
^
SyntaxError: no binding for nonlocal 'x' found

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19335] codeop misclassifies incomplete code with 'nonlocal'

[Lysandros Nikolaou]

Lysandros Nikolaou  added the comment:

I feel that I wasn't clear at all in my previous responses, so let me try to 
have another go at explaining how I see this:

We have two distinct problems, that are totally unrelated to one another:

1) nonlocal should raise when it's not in a nested function (question for the 
more experienced here: is this the *only* place where nonlocal is allowed?)

2) comc should return None, signifying an incomplete input, when presented with 
the bloack:

def a():
def b():
nonlocal c

because c, the variable, could be declared in a, the function, some time after 
the definition of b, the function, is done.

The way I see this, like I have expressed in my previous comments, is that 
whatever solution we come up with should not involve the parser, because it 
should not care about statement semantics. I'm not really sure what the best 
solutions are, but I'd propose the following for the problems I listed above:

1) We should alter code in symtable.c to check whether the namespace in which 
the nonlocal statement appears in is a function block and whether it is nested 
or not.

2) A check for the SyntaxError message to see if it comes from a nonlocal 
statement surely sounds like a hack, but I don't think that's a dealbreaker. 
I'd probably go with this rather than undergoing the effort of exposing the 
indentation stack in Python and then examining it in codeop.

Thoughts?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19335] codeop misclassifies incomplete code with 'nonlocal'

[Terry J. Reedy]

Terry J. Reedy  added the comment:

_maybe_compile currently compiles the possibly complete statement up to 3 times 
-- with C-coded compile.  Without doing any timing tests, I wondered if 3 times 
is really necessary.  Nick suggested that using the tokenize module to 
determine the number of hanging indents, combined with some yet to be 
determined logic, might be an alternative.  But since tokenize is written in 
Python, the result might not be any faster.

Checking the error message aims at specifically fixing this issue.  But I am 
not sure if the check is sufficient or if more logic is needed.
  "def a():\n   nonlocal c\n" should ideally raise.
  "def a():\n   def b():\nnonlocal c\n" must not.

At least the second should be used for a new test.

I would start by adding debug prints to see the result of each compile and then 
testing with those two lines.

--
type:  -> behavior
versions: +Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19335] codeop misclassifies incomplete code with 'nonlocal'

[Rahul Jha]

Rahul Jha  added the comment:

>  Note that these are two solution that take very different approaches. What 
> Nick is suggesting with "checking for two or more hanging INDENTS" would 
> drastically change how codeop._maybe_compile does its thing, while his other 
> proposed solution, ""hardcoding a check for nonlocal SyntaxErrors in 
> codeop._maybe_compile", would just fix this issue.

Got it!

> If there's consensus around one proposed approach, you could certainly take 
> this up. I'd be glad to help out with workflow stuff or provide a first 
> review. Let me know if you've got more questions.

Thank you so much Lysandros!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19335] codeop misclassifies incomplete code with 'nonlocal'

[Lysandros Nikolaou]

Lysandros Nikolaou  added the comment:

> From what I understand, "checking for two or more hanging INDENTS" and, 
> "hardcoding a check for nonlocal SyntaxErrors in codeop._maybe_compile" are 
> two different solutions, right?  If yes, do we have an answer to which one of 
> them is more cleaner, and henceforth, the preferable solution?

Note that these are two solution that take very different approaches. What Nick 
is suggesting with "checking for two or more hanging INDENTS" would drastically 
change how codeop._maybe_compile does its thing, while his other proposed 
solution, ""hardcoding a check for nonlocal SyntaxErrors in 
codeop._maybe_compile", would just fix this issue.


> I, personally, like the idea of checking INDENTS primarily because of it's 
> reduced specificity, but I am in no position to comment on this (I already 
> kinda did ':D), and you folks know better! For all we know, we should be 
> optimizing for specificity.

You're right that this idea is more general. It would require significantly 
more effort from whoever tackles this though.

> Also, reading Nick's comments and the comc's code, gives me the feeling that 
> a fix for this wouldn't require drastic changes.

That really depends on what solution is chosen out of the two.

> I'm slowly starting my journey with CPython, and I'd like to contribute a 
> patch if that is the case. Thanks!

If there's consensus around one proposed approach, you could certainly take 
this up. I'd be glad to help out with workflow stuff or provide a first review. 
Let me know if you've got more questions.

--
keywords: +3.3regression

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19335] codeop misclassifies incomplete code with 'nonlocal'

[Rahul Jha]

Rahul Jha  added the comment:

> That may actually be another alternative: instead of doing the "try
> appending newlines and see if it works or generates different errors",
> we may be able to switch to the tokenizer if the initial compilation
> fails and check for hanging INDENT tokens (i.e. INDENTS without a
> corresponding DEDENT). That would get us much closer to what the real
> eval loop is doing.

>From what I understand, "checking for two or more hanging INDENTS" and, 
>"hardcoding a check for nonlocal SyntaxErrors in codeop._maybe_compile" are 
>two different solutions, right?  If yes, do we have an answer to which one of 
>them is more cleaner, and henceforth, the preferable solution?

I, personally, like the idea of checking INDENTS primarily because of it's 
reduced specificity, but I am in no position to comment on this (I already 
kinda did ':D), and you folks know better! For all we know, we should be 
optimizing for specificity.

Also, reading Nick's comments and the comc's code, gives me the feeling that a 
fix for this wouldn't require drastic changes.  I'm slowly starting my journey 
with CPython, and I'd like to contribute a patch if that is the case. Thanks!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19335] codeop misclassifies incomplete code with 'nonlocal'

[Lysandros Nikolaou]

Lysandros Nikolaou  added the comment:

> If I understand correctly, Nick is talking about modifying the different 
> iterations for different errors in codeop no?

I was talking about msg200936, where Nick proposed to just hardcode a check for 
nonlocal SyntaxErrors in codeop._maybe_compile, which should return None (which 
means incomplete) when the SyntaxError comes as a result of a nonlocal binding 
not existing.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19335] codeop misclassifies incomplete code with 'nonlocal'

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> Pablo, correct me if I'm wrong, but I think that the parser has nothing to do 
> with this. It's not the parser that produces this SyntaxError, it's some 
> later compilation phase (I guess symtable creation phase?), which I know very 
> very little about, so I can't really offer an opinion on what can be done.

Yeah, this is correct the parser has nothing to be done with this, what is 
involved in this is the compiler, as the generated AST looks correct.

>  When the new parser compiles a function, does it know or could it know 
> whether it is nested?

Nop, the parser does not even know what a 'function' is while it parses, as 
that is only determined in the product of the parser itself: the AST. The piece 
that understands semantically what a function is and if is nested or not is the 
compiler.

> Parsing should not care about statement semantics, in that a nonlocal 
> statement should be like any other statement in the parser's point of view.

Yup, I very much agree with this. The parser must not know about the semantics 
of the AST elements (forcing it to do so will be not only very complex but also 
will make maintenance much more difficult as the parser will be coupled with 
the semantics of the language is parsing as opposed to only having to know 
about the grammar).

> All in all, I think that Nick's hack is the way to go here, since it's comc's 
> responsibility to correctly check if a statement is incomplete, right?

If I understand correctly, Nick is talking about modifying the different 
iterations for different errors in codeop no?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19335] codeop misclassifies incomplete code with 'nonlocal'

[Lysandros Nikolaou]

Lysandros Nikolaou  added the comment:

> What do either of you think?  Can the new parser handle it better?

Pablo, correct me if I'm wrong, but I think that the parser has nothing to do 
with this. It's not the parser that produces this SyntaxError, it's some later 
compilation phase (I guess symtable creation phase?), which I know very very 
little about, so I can't really offer an opinion on what can be done.

In any case, this should actually indicate that this is not a parser issue:

(venv) ➜  cpython git:(master) ./python.exe
Python 3.10.0a0 (heads/master:7f569c9bc0, Jun 29 2020, 14:48:39)
[Clang 11.0.3 (clang-1103.0.32.62)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> a = '''\
... def a():
... def b():
... nonlocal c
... '''
>>> a
'def a():\ndef b():\nnonlocal c\n'
>>> compile(a, '?', 'single')
Traceback (most recent call last):
  File "", line 1, in 
  File "?", line 3
SyntaxError: no binding for nonlocal 'c' found
>>> import ast
>>> ast.dump(ast.parse(a, mode='single'))
"Interactive(body=[FunctionDef(name='a', args=arguments(posonlyargs=[], 
args=[], kwonlyargs=[], kw_defaults=[], defaults=[]), 
body=[FunctionDef(name='b', args=arguments(posonlyargs=[], args=[], 
kwonlyargs=[], kw_defaults=[], defaults=[]), body=[Nonlocal(names=['c'])], 
decorator_list=[])], decorator_list=[])])"

All in all, I think that Nick's hack is the way to go here, since it's comc's 
responsibility to correctly check if a statement is incomplete, right?


> When the new parser compiles a function, does it know or could it know 
> whether it is nested?

It currently doesn't, but the tokenizer holds a stack of indent tokens, which 
can be accessed by the parser. This way we could indeed check if there are two 
or more pending indent tokens, before generating an AST for nonlocal. However, 
I don't think that this is generally a good idea. Parsing should not care about 
statement semantics, in that a nonlocal statement should be like any other 
statement in the parser's point of view.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19335] codeop misclassifies incomplete code with 'nonlocal'

[Matthias Bussonnier]

Change by Matthias Bussonnier :


--
nosy: +mbussonn

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19335] codeop misclassifies incomplete code with 'nonlocal'

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Note: Recently in #40807, Cheryl and I patched codeop._maybecompile to only 
emits warnings once in a given call.  I don't know if 3 calls (2 '\n' 
additions) to compile are really needed today.  The logic that handles the 
results is not clear to me either.  It could stand review by compile() experts.

--
nosy: +cheryl.sabella

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19335] codeop misclassifies incomplete code with 'nonlocal'

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Pablo and Lysandros: this issue is about a corner-case bug in either 
compile(,,'single') or its use by REPL and codeop._maybe_compile (comc).  What 
do either of you think?  Can the new parser handle it better?

Should we just add the hack suggested by Nick in msg200936 or is there a better 
solution?  Is interactive mode python calling compile() differently?  Or is it 
doing a check that could be incorporated into compile?  (Perhap after 
correction, see below.)

The issue above is about interactive entry of

def a():
  def b():
nonlocal c

REPL does not raise SyntaxError, comc does.  Not raising is correct here 
because additional lines added to the nonlocal context may make the code valid.

Additional experiment: the same is true (comc raises, REPL not) for

def a():
  nonlocal c

Here, REPL not raising (until a blank is entered) could be considered a glitch 
because there is no pending nonlocal context to be completed.  Though raising 
later than necessary is better than raising too soon.  Nick's hack (and the 
REPL) could check that there are at least 2 pending indents.

When the new parser compiles a function, does it know or could it know whether 
it is nested?  It should in that the legal grammer (use of nonlocal) is 
different.

--
nosy: +lys.nikolaou, pablogsal
versions: +Python 3.10 -Python 2.7, Python 3.3, Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19335] codeop misclassifies incomplete code with 'nonlocal'

[Rahul Jha]

Change by Rahul Jha :


--
nosy: +RJ722

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19335] codeop misclassifies incomplete code with 'nonlocal'

[Esa Peuha]

Esa Peuha added the comment:

 Someone needs to compare _maybe_compile to the equivalent C code used by the 
 real interpreter.

Well, _maybe_compile() just calls the built-in function compile() internally, 
so I'm not sure what sort of comparison you want...

--
nosy: +Esa.Peuha
title: codeop misclassifies incomple code with 'nonlocal' - codeop 
misclassifies incomplete code with 'nonlocal'

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19335
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19335] codeop misclassifies incomplete code with 'nonlocal'

[Mark Shannon]

Changes by Mark Shannon m...@hotpy.org:


--
nosy: +Mark.Shannon

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19335
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19335] codeop misclassifies incomplete code with 'nonlocal'

[Nick Coghlan]

Nick Coghlan added the comment:

The comment at the top of codeop explains the problem (and why Terry is 
interested in what the C code is doing, since it's clearly different):

===
Compile three times: as is, with \n, and with \n\n appended.  If it
compiles as is, it's complete.  If it compiles with one \n appended,
we expect more.  If it doesn't compile either way, we compare the
error we get when compiling with \n or \n\n appended.  If the errors
are the same, the code is broken.  But if the errors are different, we
expect more.  Not intuitive; not even guaranteed to hold in future
releases; but this matches the compiler's behavior from Python 1.4
through 2.2, at least.
===

Unfortunately, the way the main interactive loop works isn't useful to codeop: 
the C level loop simply blocks on stdin, waiting until the parser spits out a 
complete AST. By contrast, codeop is handed complete strings, and has to decide 
whether they're a potentially incomplete or not.

nonlocal c is unique in that it's a name lookup that is *checked by the 
compiler*, so it triggers the *same* exception in all 3 cases that 
codeop._maybe_compile tries:

 src = def a():\n   def b():\nnonlocal c
 compile(src, , single)
Traceback (most recent call last):
  File stdin, line 1, in module
  File , line 3
SyntaxError: no binding for nonlocal 'c' found
 compile(src + \n, , single)
Traceback (most recent call last):
  File stdin, line 1, in module
  File , line 3
SyntaxError: no binding for nonlocal 'c' found
 compile(src + \n\n, , single)
Traceback (most recent call last):
  File stdin, line 1, in module
  File , line 3
SyntaxError: no binding for nonlocal 'c' found

So it's a SyntaxError that *could be fixed* by later code, but that code never 
gets a chance to run, because codeop assumes that getting the same error in the 
last two cases means it will never pass.

--
nosy: +ncoghlan

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19335
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19335] codeop misclassifies incomplete code with 'nonlocal'

[Nick Coghlan]

Nick Coghlan added the comment:

As a potential fix (albeit an ugly hack), try changing this part of 
codeop._maybe_compile:

if not code1 and repr(err1) == repr(err2):
raise err1

To something like:

if not code1 and repr(err1) == repr(err2):
if isinstance(err1, SyntaxError) and no binding for nonlocal in 
str(err1) and not source.endswith(\n\n):
# Allow a nonlocal namebinding to be supplied *after* a
# a function definition (the standard interpreter loop
# handles this by blocking on stdin, but this module accepts
# input as complete strings rather than as a stream)
return None
raise err1

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19335
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19335] codeop misclassifies incomplete code with 'nonlocal'

[R. David Murray]

R. David Murray added the comment:

A complete fix is going to require setting a flag that we have a pending 
non-local, and check that flag when the code input is complete to raise the 
SyntaxError at that point if the non-local hasn't been set.

--
nosy: +r.david.murray

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19335
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19335] codeop misclassifies incomplete code with 'nonlocal'

[Nick Coghlan]

Nick Coghlan added the comment:

codeop compiles the whole pending statement every time, so that part
shouldn't be needed (from the compiler's point of view, this input
looks complete - there's nothing dangling, any more than there is for
a normal syntax error like name name). As far as I can tell, the
only reason it works in the internal parser case is because the parser
doesn't consider the input complete until it sees the expected
dedents, which the tokenizer doesn't generate until it sees the
trailing newline.

That may actually be another alternative: instead of doing the try
appending newlines and see if it works or generates different errors,
we may be able to switch to the tokenizer if the initial compilation
fails and check for hanging INDENT tokens (i.e. INDENTS without a
corresponding DEDENT). That would get us much closer to what the real
eval loop is doing.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19335
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com