[issue26806] IDLE not displaying RecursionError tracebacks and hangs

[Terry J. Reedy]

Terry J. Reedy  added the comment:


New changeset f54f062f68ac82fc3e7c92345b4d5fbe2534dc84 by Terry Jan Reedy (Miss 
Islington (bot)) in branch '3.7':
bpo-26806: IDLE should run without docstrings (GH-14657) (GH-14678)
https://github.com/python/cpython/commit/f54f062f68ac82fc3e7c92345b4d5fbe2534dc84


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26806] IDLE not displaying RecursionError tracebacks and hangs

[Terry J. Reedy]

Terry J. Reedy  added the comment:


New changeset b82188d9bad1774624cb1788dbdec2f0d7d65688 by Terry Jan Reedy (Miss 
Islington (bot)) in branch '3.8':
bpo-26806: IDLE should run without docstrings (GH-14657) (GH-14677)
https://github.com/python/cpython/commit/b82188d9bad1774624cb1788dbdec2f0d7d65688


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26806] IDLE not displaying RecursionError tracebacks and hangs

[miss-islington]

Change by miss-islington :


--
pull_requests: +14483
pull_request: https://github.com/python/cpython/pull/14677

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26806] IDLE not displaying RecursionError tracebacks and hangs

[Terry J. Reedy]

Terry J. Reedy  added the comment:


New changeset 6aeb2fe606408aae14c246470794f1303b3be812 by Terry Jan Reedy in 
branch 'master':
bpo-26806: IDLE should run without docstrings (#14657)
https://github.com/python/cpython/commit/6aeb2fe606408aae14c246470794f1303b3be812


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26806] IDLE not displaying RecursionError tracebacks and hangs

[miss-islington]

Change by miss-islington :


--
pull_requests: +14484
pull_request: https://github.com/python/cpython/pull/14678

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26806] IDLE not displaying RecursionError tracebacks and hangs

[Terry J. Reedy]

Change by Terry J. Reedy :


--
pull_requests: +14464
pull_request: https://github.com/python/cpython/pull/14657

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26806] IDLE not displaying RecursionError tracebacks and hangs

[Tal Einat]

Change by Tal Einat :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26806] IDLE not displaying RecursionError tracebacks and hangs

[miss-islington]

miss-islington  added the comment:


New changeset d666217b26c373784761e3a84f243f02682bccb1 by Miss Islington (bot) 
in branch '3.7':
bpo-26806: add 30 to the recursion limit in IDLE's shell (GH-13944)
https://github.com/python/cpython/commit/d666217b26c373784761e3a84f243f02682bccb1


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26806] IDLE not displaying RecursionError tracebacks and hangs

[miss-islington]

miss-islington  added the comment:


New changeset d4af55391f56286ab8d478591017174a5a0a5ce2 by Miss Islington (bot) 
in branch '3.8':
bpo-26806: add 30 to the recursion limit in IDLE's shell (GH-13944)
https://github.com/python/cpython/commit/d4af55391f56286ab8d478591017174a5a0a5ce2


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26806] IDLE not displaying RecursionError tracebacks and hangs

[Tal Einat]

Tal Einat  added the comment:


New changeset fcf1d003bf4f0100c9d0921ff3d70e1127ca1b71 by Tal Einat in branch 
'master':
bpo-26806: add 30 to the recursion limit in IDLE's shell (GH-13944)
https://github.com/python/cpython/commit/fcf1d003bf4f0100c9d0921ff3d70e1127ca1b71


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26806] IDLE not displaying RecursionError tracebacks and hangs

[miss-islington]

Change by miss-islington :


--
pull_requests: +14435
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/14621

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26806] IDLE not displaying RecursionError tracebacks and hangs

[miss-islington]

Change by miss-islington :


--
pull_requests: +14436
pull_request: https://github.com/python/cpython/pull/14622

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26806] IDLE not displaying RecursionError tracebacks and hangs

[Terry J. Reedy]

Terry J. Reedy  added the comment:

[Initially written before you posted the PR]
Serhiy's result suggests that 25 might not always be enough and he suggested 
50.  Plus IDLE sometimes makes more calls internally.  Lets go with at least 30.

I agree that "get...(set...(n)) == n" (when n is positive, a check we should 
emulate) and "set...(get...())" leaving limit as it was are desirable.  The 
docs and docstrings just need to say what we do.  Possible additions.
"This IDLE wrapper adds 30 to prevent possible uninterruptible loops."
"This IDLE wrapper subtracts 30 to compensate for the 30 IDLE adds when setting 
the limit."

The main reason I can think of for users to play with low recursion limits is 
to explore the implementation.  They simply cannot get stock CPython behavior 
when running with IDLE.  So I want the IDLE wrappers to not be completely 
transparent. For that to be obvious in calltips, the first line should say so.

The main reason I want to intervene is to prevent the not-nice behavior of 
hanging until a manual restart.  I checked, and it is still possible.

In 3.8.0b1, the min n is not 1 but current stack + 5.
>>> sys.setrecursionlimit(11)
Traceback (most recent call last):
  File "", line 1, in 
sys.setrecursionlimit(11)
RecursionError: cannot set the recursion limit to 11 at the recursion depth 6: 
the limit is too low

Our increment will prevent a user from seeing this, but this is necessary to 
prevent ...

>>> sys.setrecursionlimit(12)
# hangs, once after printing "Exception ignored in thread started by"
# until manual restart.
# 18 hangs after printing 60 lines of a double exception.
# 20 does not hang and calling 'f' from the original message
# gets a truncated traceback ending with
   [Previous line repeated 12 more times]

--
stage: patch review -> needs patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26806] IDLE not displaying RecursionError tracebacks and hangs

[Tal Einat]

Tal Einat  added the comment:

See PR GH-13944.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26806] IDLE not displaying RecursionError tracebacks and hangs

[Tal Einat]

Change by Tal Einat :


--
keywords: +patch
pull_requests: +13811
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/13944

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26806] IDLE not displaying RecursionError tracebacks and hangs

[Tal Einat]

Tal Einat  added the comment:

Setting the recursion limit too low also causes issues in a plain Python 
interactive session. With current master (to be 3.9) on Win10, I get issues 
with values up to 6. In IDLE values up to 20 cause an immediate exception 
similar to the one described by Serhiy.

Anyways, I think we should increment by 25, to always set "safe" values.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26806] IDLE not displaying RecursionError tracebacks and hangs

[Tal Einat]

Tal Einat  added the comment:

> 'max(n, 50)' should have been 'min(n, 50)'  Tal, does you comment avove about 
> the former apply when corrected to the latter?

Yes. To use your terms, I prefer incrementing over thresholding.

> This is all that is needed for getrecursionlimit.

If we go with incrementing, then getrecursionlimit should likely decrement the 
returned value by the same amount, so that 
setrecursionlimit(getrecursionlimit()) doesn't change the recursion limit.

> Currently, at least on CPython, n must be >= 1 or ValueError is raised.  I 
> don't like changing that.

This is another issue that incrementing avoids.

Since it looks like we're in general agreement, Terry, I'll work up a PR.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26806] IDLE not displaying RecursionError tracebacks and hangs

[Terry J. Reedy]

Terry J. Reedy  added the comment:

'max(n, 50)' should have been 'min(n, 50)'  Tal, does you comment avove about 
the former apply when corrected to the latter?

Both get... and setrecursionlimit should be wrapped in a function, such as 
'wrap_recursionlimits', called from run.main, each decorated with 
functools.wraps.  Both docstrings should be augmented with ... .__doc__ += 
"\n\nWhen running under IDLE, ..."  This is all that is needed for 
getrecursionlimit.  The question is to do with setrecursionlimit.

Whatever is chosen should be added to the IDLE doc section mentioned above.

Currently, at least on CPython, n must be >= 1 or ValueError is raised.  I 
don't like changing that.  I don't have a strong feeling about threshholding 
versus incrementing, but see my question above.

--
versions:  -Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26806] IDLE not displaying RecursionError tracebacks and hangs

[Tal Einat]

Tal Einat  added the comment:

IMO just updating the documentation would not be enough.

I suggest wrapping sys.setrecursionlimit() and *adding* 30 to the given limit, 
to offset the call stack depth added by IDLE. This makes more sense to me than 
max(n, 50), which I imagine would cause a lot of confusion in cases where it 
comes into effect.

If that's not acceptable, then I suggest wrapping it to raise an informative 
exception if the given limit is too low.

--
nosy: +taleinat

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26806] IDLE not displaying RecursionError tracebacks and hangs

[Tal Einat]

Change by Tal Einat :


--
components:  -Documentation
versions: +Python 3.8, Python 3.9 -Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26806] IDLE not displaying RecursionError tracebacks and hangs

[Terry J. Reedy]

Terry J. Reedy added the comment:

I ran into this again.  Raising priority.

--
priority: normal -> high
title: IDLE not displaying RecursionError tracebacks -> IDLE not displaying 
RecursionError tracebacks and hangs
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26806] IDLE not displaying RecursionError tracebacks

[Terry J. Reedy]

Terry J. Reedy added the comment:

Interesting idea.  We avoid the problem by adding something like the following 
to run.py

_setrecursionlimit = sys.setrecursionlimit
def setrecursionlimit(n):
_setrecursionlimit(max(n, 50))
sys.setrecursionlimit = setrecursionlimit

This works when entered interactively, and I presume it would within run.py.  
For _setrecursionlimit to be accessible from user code (to reverse the monkey 
patching), it would have to be attached to the function as an attribute.

setrecursionlimit.original = _setrecursionlimit

Though user-friendly for most users, this would make IDLE execute code 
differently from raw Python.  The builtin has a lower limit, based on the 
current stack depth, but it raises instead of setting a higher limit.  I 
presume we could use len(inspect.stack()) to get the current 'recursion depth', 
and add, say, 30 rather than 3. (The current error message could be more 
helpful.)

>>> sys.setrecursionlimit(3)
Traceback (most recent call last):
  File "", line 1, in 
RecursionError: cannot set the recursion limit to 3 at the recursion depth 1: 
the limit is too low
>>> sys.setrecursionlimit(4)
>>> f()
>>>
>>> def f():
...   print('f')
...   f()
...
>>> f()
>>>

The call to f seems to be silently ignored.  This does not seem helpful.

Whatever we do, I would mention it in a revision of the proposed paragraph 
above.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26806] IDLE not displaying RecursionError tracebacks

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

We can monkeypatch sys.setrecursionlimit() to set recursion limit to at least 
say 50.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26806] IDLE not displaying RecursionError tracebacks

[Terry J. Reedy]

Terry J. Reedy added the comment:

Thanks for the investigation.  I copied the limit from a python-ideas example 
and did not realize that IDLE adds so much to the call stack.  I should add a 
bit on the subject to the IDLE doc in "3.2. IDLE-console differences".  (It can 
also impact tracebacks -- as reported on a couple of tracker issues.)

"IDLE adds it own calls to the call stack before (and possibly after) running 
user code.  Tracebacks are filtered to remove these, but the filtering may not 
be perfect.  Setting a recursion limit that is too low, such as with 
sys.setrecursionlimit(20), will result in mysterious malfunctions."

--
components: +Documentation, IDLE
stage: test needed -> needs patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26806] IDLE not displaying RecursionError tracebacks

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Experimentally found that minimal recursion level for 2.7 is 29, for 3.2-3.3 is 
24, and for 3.4-3.6 is 25.

3.2 produces following output and restart the shell:


Unhandled server exception!
Thread: SockThread
Client Address:  ('127.0.0.1', 37227)
Request:  
Traceback (most recent call last):
  File "/home/serhiy/py/cpython-3.2/Lib/socketserver.py", line 295, in 
_handle_request_noblock
self.process_request(request, client_address)
  File "/home/serhiy/py/cpython-3.2/Lib/socketserver.py", line 321, in 
process_request
self.finish_request(request, client_address)
  File "/home/serhiy/py/cpython-3.2/Lib/socketserver.py", line 334, in 
finish_request
self.RequestHandlerClass(request, client_address, self)
  File "/home/serhiy/py/cpython-3.2/Lib/idlelib/rpc.py", line 503, in __init__
socketserver.BaseRequestHandler.__init__(self, sock, addr, svr)
  File "/home/serhiy/py/cpython-3.2/Lib/socketserver.py", line 648, in __init__
self.handle()
  File "/home/serhiy/py/cpython-3.2/Lib/idlelib/run.py", line 285, in handle
rpc.RPCHandler.getresponse(self, myseq=None, wait=0.05)
  File "/home/serhiy/py/cpython-3.2/Lib/idlelib/rpc.py", line 280, in 
getresponse
response = self._getresponse(myseq, wait)
  File "/home/serhiy/py/cpython-3.2/Lib/idlelib/rpc.py", line 300, in 
_getresponse
response = self.pollresponse(myseq, wait)
  File "/home/serhiy/py/cpython-3.2/Lib/idlelib/rpc.py", line 421, in 
pollresponse
self.putmessage(message)
  File "/home/serhiy/py/cpython-3.2/Lib/idlelib/rpc.py", line 324, in putmessage
s = pickle.dumps(message)
RuntimeError: maximum recursion depth exceeded while pickling an object

*** Unrecoverable, server exiting!


3.3 hangs without any terminal output.

3.4 produces the most detailed output and hangs:


Unhandled server exception!
Thread: SockThread
Client Address:  ('127.0.0.1', 46394)
Request:  
Traceback (most recent call last):
Exception in thread SockThread:
Traceback (most recent call last):
  File "/home/serhiy/py/cpython-3.4/Lib/socketserver.py", line 305, in 
_handle_request_noblock
self.process_request(request, client_address)
  File "/home/serhiy/py/cpython-3.4/Lib/socketserver.py", line 331, in 
process_request
self.finish_request(request, client_address)
  File "/home/serhiy/py/cpython-3.4/Lib/socketserver.py", line 344, in 
finish_request
self.RequestHandlerClass(request, client_address, self)
  File "/home/serhiy/py/cpython-3.4/Lib/idlelib/rpc.py", line 508, in __init__
socketserver.BaseRequestHandler.__init__(self, sock, addr, svr)
  File "/home/serhiy/py/cpython-3.4/Lib/socketserver.py", line 673, in __init__
self.handle()
  File "/home/serhiy/py/cpython-3.4/Lib/idlelib/run.py", line 318, in handle
rpc.RPCHandler.getresponse(self, myseq=None, wait=0.05)
  File "/home/serhiy/py/cpython-3.4/Lib/idlelib/rpc.py", line 288, in 
getresponse
response = self._getresponse(myseq, wait)
  File "/home/serhiy/py/cpython-3.4/Lib/idlelib/rpc.py", line 308, in 
_getresponse
response = self.pollresponse(myseq, wait)
  File "/home/serhiy/py/cpython-3.4/Lib/idlelib/rpc.py", line 426, in 
pollresponse
self.putmessage(message)
  File "/home/serhiy/py/cpython-3.4/Lib/idlelib/rpc.py", line 332, in putmessage
s = dumps(message)
  File "/home/serhiy/py/cpython-3.4/Lib/idlelib/rpc.py", line 60, in dumps
p.dump(obj)
RuntimeError: maximum recursion depth exceeded while pickling an object

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/serhiy/py/cpython-3.4/Lib/threading.py", line 911, in 
_bootstrap_inner
self.run()
  File "/home/serhiy/py/cpython-3.4/Lib/threading.py", line 859, in run
self._target(*self._args, **self._kwargs)
  File "/home/serhiy/py/cpython-3.4/Lib/idlelib/run.py", line 162, in 
manage_socket
server.handle_request() # A single request only
  File "/home/serhiy/py/cpython-3.4/Lib/socketserver.py", line 290, in 
handle_request
self._handle_request_noblock()
  File "/home/serhiy/py/cpython-3.4/Lib/socketserver.py", line 307, in 
_handle_request_noblock
self.handle_error(request, client_address)
  File "/home/serhiy/py/cpython-3.4/Lib/idlelib/run.py", line 288, in 
handle_error
traceback.print_exc(file=erf)
  File "/home/serhiy/py/cpython-3.4/Lib/traceback.py", line 252, in print_exc
print_exception(*sys.exc_info(), limit=limit, file=file, chain=chain)
  File "/home/serhiy/py/cpython-3.4/Lib/traceback.py", line 169, in 
print_exception
for line in _format_exception_iter(etype, value, tb, limit, chain):
  File "/home/serhiy/py/cpython-3.4/Lib/traceback.py", line 153, in 
_format_exception_iter
yield from _format_list_iter(_extract_tb_iter(tb, limit=limit))
  File "/home/serhiy/py/cpython-3.4/Lib/traceback.py", line 18, in 
_format_list_iter
  

[issue26806] IDLE not displaying RecursionError tracebacks

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

In 3.6 just setting the recursion level to 20 produces following output on 
terminal:


Unhandled server exception!
Thread: SockThread
Client Address:  ('127.0.0.1', 41515)
Unhandled exception in thread started by 

and a hang.

In 2.7 it produces:


Unhandled server exception!
Thread: SockThread
Client Address:  ('127.0.0.1', 35043)
Request:  
Traceback (most recent call last):
  File "/home/serhiy/py/cpython-2.7-debug/Lib/SocketServer.py", line 290, in 
_handle_request_noblock
self.process_request(request, client_address)
  File "/home/serhiy/py/cpython-2.7-debug/Lib/SocketServer.py", line 318, in 
process_request
self.finish_request(request, client_address)
  File "/home/serhiy/py/cpython-2.7-debug/Lib/SocketServer.py", line 331, in 
finish_request
self.RequestHandlerClass(request, client_address, self)
  File "/home/serhiy/py/cpython-2.7-debug/Lib/idlelib/rpc.py", line 500, in 
__init__
SocketServer.BaseRequestHandler.__init__(self, sock, addr, svr)
  File "/home/serhiy/py/cpython-2.7-debug/Lib/SocketServer.py", line 652, in 
__init__
self.handle()
  File "/home/serhiy/py/cpython-2.7-debug/Lib/idlelib/run.py", line 292, in 
handle
rpc.RPCHandler.getresponse(self, myseq=None, wait=0.05)
  File "/home/serhiy/py/cpython-2.7-debug/Lib/idlelib/rpc.py", line 280, in 
getresponse
response = self._getresponse(myseq, wait)
  File "/home/serhiy/py/cpython-2.7-debug/Lib/idlelib/rpc.py", line 300, in 
_getresponse
response = self.pollresponse(myseq, wait)
RuntimeError: maximum recursion depth exceeded

*** Unrecoverable, server exiting!


and restarts the shell.

Definitely the 20 limit is too low for IDLE.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26806] IDLE not displaying RecursionError tracebacks

[Terry J. Reedy]

New submission from Terry J. Reedy:

Test program:

import sys
sys.setrecursionlimit(20)
def f(): return f()
f()

F:\Python\mypy>python tem.py
Traceback (most recent call last):
  File "tem.py", line 4, in 
f()
  File "tem.py", line 3, in f
def f(): return f()
...
RecursionError: maximum recursion depth exceeded

In 2.7.11, the error is caught and the user process restarted.

=== RESTART: F:\Python\mypy\tem.py ===

=== RESTART: Shell ===
>>> 

In 3.5.1, the user process hangs, ^C does not work, and a restart explicitly 
requested either with Shell => Restart or running another program.

This behavior is either peculiar to this test case, or a regression, as I 
remember getting proper RecursionError tracebacks in the past.

--
assignee: terry.reedy
messages: 263785
nosy: terry.reedy
priority: normal
severity: normal
stage: test needed
status: open
title: IDLE not displaying RecursionError tracebacks
type: behavior
versions: Python 2.7, Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com