On 18 September 2017 at 20:52, Antoine Pitrou <solip...@pitrou.net> wrote: > On Mon, 18 Sep 2017 20:35:02 +1000 > Nick Coghlan <ncogh...@gmail.com> wrote: >> Rather than being thread local or context local state, whether or not >> to keep the full frames in the traceback could be a yet another >> setting on the *exception* object, whereby we tweaked the logic that >> drops the reference at the end of an except clause as follows: > > That doesn't solve the problem, since the issue is that exceptions can > be raised (and then silenced) in many places, and you don't want such > exception-raising code (such as socket.create_connection) to start > having to set an option on the exceptions it raises.
Bleh, in trying to explain why my proposal would be sufficient to break the problematic cycles, I realised I was wrong: if we restrict the frame clearing to already terminated frames (as would be necessary to avoid breaking any still executing functions), then that means we won't clear the frame running the exception handler, and that's the frame that creates the problematic cyclic reference. However, I still think it makes more sense to focus on the semantics of preserving an exception beyond the life of the stack being unwound, rather than on the semantics of raising the exception in the first place. In the usual case, the traceback does keep the whole stack alive while the stack is being unwound, but then the exception gets thrown away at the end when sys.exc_info() gets reset back to (None, None, None), and then all the frames still get cleaned up fairly promptly (this is also the case in Python 2). We only get problems when one of the exception handlers in the stack grabs an additional reference to either the traceback or the exception and hence creates a persistent cyclic reference from one of the frames back to itself. The difference in Python 3 is that saving the exception is reasonably common, while explicitly saving the traceback is relatively rare, so the "exc.__traceback__" is keeping tracebacks alive that would otherwise have been cleaned up more deterministically. Putting the problem that way gave me an idea, though: what if, when the interpreter was setting "sys.exc_info()" back to (None, None, None) (or otherwise dropping an exception instance from being the "currently active exception") it automatically set exc.__traceback__ to None? That way, if you wanted the *traceback* (rather than just the exception) to live beyond the stack being unwound, you'd have to preserve the entire sys.exc_info() triple (or at least save "exc.__traceback__" separately from "exc"). By doing that, we'd have the opportunity to encourage folks that are considering preserving the entire traceback to extract a TracebackException instead and some themselves from some potentially nasty reference management issues: https://docs.python.org/3/library/traceback.html#tracebackexception-objects Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
