Re: [Python-Dev] Evil reference cycles caused Exception.__traceback__

Hi, just curious on this, On 09/18/2017 10:54 PM, Antoine Pitrou wrote: >> I'm not an expert on GC at all, but intuitively it sure seems like >> allocation size might be a useful piece of information to feed into a >> heuristic. Our current heuristic is just, run a small collection after >> every

Re: [Python-Dev] Evil reference cycles caused Exception.__traceback__

On Mon, 18 Sep 2017 13:25:47 -0700 Nathaniel Smith wrote: > >> If it is then it might make sense to look at the cycle collection > >> heuristics; IIRC they're based on a fairly naive count of how many > >> allocations have been made, without regard to their size. > > > > Yes...

Re: [Python-Dev] Evil reference cycles caused Exception.__traceback__

On Mon, Sep 18, 2017 at 10:59 AM, Antoine Pitrou wrote: > Le 18/09/2017 à 19:53, Nathaniel Smith a écrit : >>> Why are reference cycles a problem that needs solving? >>> >>> Because sometimes they are holding up costly resources in memory when >>> people don't expect them

Re: [Python-Dev] Evil reference cycles caused Exception.__traceback__

Thanks for working on this and writing up the details Victor. For those confused about why this matters, routinely having every object in your application participating in one (or more) giant reference cycles makes reasoning about and fixing resource issues very difficult. For instance, a while

Re: [Python-Dev] Evil reference cycles caused Exception.__traceback__

Le 18/09/2017 à 19:53, Nathaniel Smith a écrit : >> >>> Why are reference cycles a problem that needs solving? >> >> Because sometimes they are holding up costly resources in memory when >> people don't expect them to. Such as large Numpy arrays :-) > > Do we have any reason to believe that this

Re: [Python-Dev] Evil reference cycles caused Exception.__traceback__

On Mon, Sep 18, 2017 at 9:50 AM, Antoine Pitrou wrote: > On Mon, 18 Sep 2017 09:42:45 -0700 > Nathaniel Smith wrote: >> >> Obviously it's nice when the refcount system is able to implicitly clean >> things up in a prompt and deterministic way, but there are

Re: [Python-Dev] Evil reference cycles caused Exception.__traceback__

On Mon, 18 Sep 2017 09:42:45 -0700 Nathaniel Smith wrote: > > Obviously it's nice when the refcount system is able to implicitly clean > things up in a prompt and deterministic way, but there are already tools to > handle the cases where it doesn't (ResourceWarning, context

Re: [Python-Dev] Evil reference cycles caused Exception.__traceback__

On Sep 18, 2017 07:58, "Antoine Pitrou" wrote: Le 18/09/2017 à 16:52, Guido van Rossum a écrit : > > In Python 2 the traceback was not part of the exception object because > there was (originally) no cycle GC. In Python GC we changed the awkward > interface to something more

Re: [Python-Dev] Evil reference cycles caused Exception.__traceback__

Le 18/09/2017 à 16:52, Guido van Rossum a écrit : > > In Python 2 the traceback was not part of the exception object because > there was (originally) no cycle GC. In Python GC we changed the awkward > interface to something more useful, because we could depend on GC. Why > are we now trying to

Re: [Python-Dev] Evil reference cycles caused Exception.__traceback__

On Mon, Sep 18, 2017 at 2:48 AM, Antoine Pitrou wrote: > On Mon, 18 Sep 2017 11:31:12 +0200 > Victor Stinner wrote: > > > > Ideally, CPython 3.x should never create reference cycles. Removing > > Exception.__traceback__ is the obvious "fix" for the

Re: [Python-Dev] Evil reference cycles caused Exception.__traceback__

On 18 September 2017 at 09:31, Victor Stinner wrote: > Last years, I fixed many reference cycles in various parts of the > Python 3 standard library. Sometimes, it takes years to become aware > of the reference cycle and finally fix it. > > For example, recently, I

Re: [Python-Dev] Evil reference cycles caused Exception.__traceback__

On 18 September 2017 at 20:52, Antoine Pitrou wrote: > On Mon, 18 Sep 2017 20:35:02 +1000 > Nick Coghlan wrote: >> Rather than being thread local or context local state, whether or not >> to keep the full frames in the traceback could be a yet another >>

Re: [Python-Dev] Evil reference cycles caused Exception.__traceback__

On Mon, 18 Sep 2017 20:35:02 +1000 Nick Coghlan wrote: > On 18 September 2017 at 20:18, Victor Stinner > wrote: > > 2017-09-18 12:07 GMT+02:00 Nick Coghlan : > >> I wonder if it might be reasonable to have tracebacks only hold

Re: [Python-Dev] Evil reference cycles caused Exception.__traceback__

On Mon, 18 Sep 2017 20:07:42 +1000 Nick Coghlan wrote: > On 18 September 2017 at 19:48, Antoine Pitrou wrote: > > On Mon, 18 Sep 2017 11:31:12 +0200 > > Victor Stinner wrote: > >> > >> Ideally, CPython 3.x should never create

Re: [Python-Dev] Evil reference cycles caused Exception.__traceback__

On 18 September 2017 at 20:18, Victor Stinner wrote: > 2017-09-18 12:07 GMT+02:00 Nick Coghlan : >> I wonder if it might be reasonable to have tracebacks only hold a weak >> reference to their frame objects when "__debug__ == False". > > Please don't

Re: [Python-Dev] Evil reference cycles caused Exception.__traceback__

2017-09-18 12:07 GMT+02:00 Nick Coghlan : > I wonder if it might be reasonable to have tracebacks only hold a weak > reference to their frame objects when "__debug__ == False". Please don't change the Python behaviour depending on __debug__, or it will be a nightmare to debug

Re: [Python-Dev] Evil reference cycles caused Exception.__traceback__

On 18 September 2017 at 19:48, Antoine Pitrou wrote: > On Mon, 18 Sep 2017 11:31:12 +0200 > Victor Stinner wrote: >> >> Ideally, CPython 3.x should never create reference cycles. Removing >> Exception.__traceback__ is the obvious "fix" for the

Re: [Python-Dev] Evil reference cycles caused Exception.__traceback__

Hum, my email is long. Maybe an short example is simpler to understand: --- import gc class VerboseDestructor: # Imagine an object using many limited resources like memory, # sockets and/or files def __del__(self): print("VerboseDestructor") def create_ref_cycle(): obj =

Re: [Python-Dev] Evil reference cycles caused Exception.__traceback__

On Mon, 18 Sep 2017 11:31:12 +0200 Victor Stinner wrote: > > Ideally, CPython 3.x should never create reference cycles. Removing > Exception.__traceback__ is the obvious "fix" for the issue. But I > expect that slowly, a lot of code started to rely on the attribute, >

[Python-Dev] Evil reference cycles caused Exception.__traceback__

Hi, Python 3 added a __traceback__ attribute to exception objects. I guess that it was added to be able to get the original traceback when an exception is re-raised. Artifical example (real code is more complex with subfunctions and conditional code): try: ... except Exception as exc: ...