Re: [Python-Dev] Class destructor
"Guido van Rossum" <[EMAIL PROTECTED]> wrote: > > Can you explain the reason for cleaning up in this scenario? Are you > rapidly creating and destroying temporary class objects? Why can't you > rely on the regular garbage collection process? Or does you class > create an external resource like a temp file? Effectively the latter. The C level defines a meta-class, which is instantiated with a specific precision, range etc. to derive the class that can actually be used. There can be an arbitrary number of such derived classes, with different properties. Very like Decimal, but with the context as part of the derived class. The instantiation creates quite a lot of constants and scratch space, some of which are Python objects but others of which are just Python memory (PyMem_Malloc); this is where an anonymous storage built-in type would be useful. The contents of these are of no interest to any Python code, and even the objects are ones which mustn't be accessed by the exported interfaces. Also, on efficiency grounds, all of those need to be accessible by C pointers from the exported class. Searching by name every time they are needed is far too much overhead. Note that, as with Decimal, the issue is that they are arbitrary sized and therefore can't simply be put in the class structure. Now, currently, I have implemented the suggestion of using the callback on the C object that points to the structure that contains the pointers to all of those. I need to investigate it in more detail, because I have had mixed success - that could well be the result of another bug in my code, so let's not worry about it. In THIS case, I am now pretty sure that I don't need any more, but I can imagine classes where it wouldn't be adequate. In particular, THIS code doesn't need to do anything other than free memory, so I don't care whether the C object attribute callback is called before or after the class object is disposed of. But that is obviously not the case in general. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: [EMAIL PROTECTED] Tel.: +44 1223 334761Fax: +44 1223 334679 ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Class destructor
Sorry about a second message, but I mentioned this aspect earlier, and it is semi-independent. If I want to produce statistics, such as the times spent in various operations, I need a callback when the class is disposed of. Now, that WOULD be inconvenient to use the C object attribute callback, unless I could be sure that would be called while the class structure is still around. That could be resolved by taking a copy, of course, but that is messy. This also relates to one of my problems with the callback. I am not being called back if the class is still live at program termination; ones that have had their use counts drop to zero do cause a callback, but not ones whose use count is above zero. I am not sure whether this is my error or a feature of the garbage collector. If the latter, it doesn't matter from the point of view of freeing space, but is assuredly a real pain for producing statistics. I haven't looked into it, as it is not an immediate task. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: [EMAIL PROTECTED] Tel.: +44 1223 334761Fax: +44 1223 334679 ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] except Exception as err, tb [was: with_traceback]
Guido van Rossum wrote: > Since this can conceivably be going on in parallel in multiple > threads, we really don't ever want to be sharing whatever object > contains the head of the chain of tracebacks since it mutates at every > frame bubble-up. So (full) exceptions can't be unitary objects. In theory, raising an already-instantiated instance could indicate "no traceback", which could make pre-cooked exceptions even lighter. Otherwise, there is no way to make exceptions reference their traceback directly. Using a property which depends on both the exception and the frame asking (catching frame? thread?) is ... probably worse than keeping them separate. >>> except MyException as err, tb isn't so awful, and makes it clear that the traceback is no longer needed after the more typical >>> except MyException as err ... # no raise statements needing a context or cause -jJ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] except Exception as err, tb [was: with_traceback]
On 3/2/07, Jim Jewett <[EMAIL PROTECTED]> wrote: > Guido van Rossum wrote: > > > Since this can conceivably be going on in parallel in multiple > > threads, we really don't ever want to be sharing whatever object > > contains the head of the chain of tracebacks since it mutates at every > > frame bubble-up. > > So (full) exceptions can't be unitary objects. > > In theory, raising an already-instantiated instance could indicate "no > traceback", which could make pre-cooked exceptions even lighter. > > Otherwise, there is no way to make exceptions reference their > traceback directly. Using a property which depends on both the > exception and the frame asking (catching frame? thread?) is ... > probably worse than keeping them separate. > > >>> except MyException as err, tb > > isn't so awful, and makes it clear that the traceback is no longer > needed after the more typical > > >>> except MyException as err > ... # no raise statements needing a context or cause > Just because it popped in my head, I think:: except Exception as exc with tb: ... reads better. -Brett ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] except Exception as err, tb [was: with_traceback]
"Jim Jewett" <[EMAIL PROTECTED]> wrote: > Guido van Rossum wrote: > > > Since this can conceivably be going on in parallel in multiple > > threads, we really don't ever want to be sharing whatever object > > contains the head of the chain of tracebacks since it mutates at every > > frame bubble-up. > > So (full) exceptions can't be unitary objects. > > In theory, raising an already-instantiated instance could indicate "no > traceback", which could make pre-cooked exceptions even lighter. Grrk. I think that this is right, but the wrong way to think of it! If we regard a kind of exception as a class, and an actual occurrence as an instance, things become a lot cleaner. The class is very simple, because all it says is WHAT happened - let's say divide by zero, or an attempt to finagle an object of class chameleon. The instance contains all of the information about the details, such as the exact operation, the values and the context (including the traceback). It CAN'T be an object, because it is not 'assignable' (i.e. a value) - it is inherently bound to its context. You can turn it into an object by copying its context into an assignable form, but the actual instance is not assignable. This becomes VERY clear when you try to implement advanced exception handling - rare nowadays - including the ability to trap exceptions, fix up the failure and continue (especially in a threaded environment). This makes no sense whatsoever in another context, and it becomes clear that the action of turning an instance into an object disables the ability to fix up the exception and continue. You can still raise a Python-style exception (i.e. abort up to the closest handler), but you can't resume transparently. I have implemented such a system, IBM CEL was one, and VMS had/has one. I don't know of any in the Unix or Microsoft environments, but there may be a few in specialised areas. Harking back to your point, your "already-instantiated instance" is actually an object derived directly from the exception class, and everything becomes clear. Because it is an object, any context it includes was a snapshot and is no longer valid. In your case, you would want it to have "context: unknown". Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: [EMAIL PROTECTED] Tel.: +44 1223 334761Fax: +44 1223 334679 ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] except Exception as err, tb [was: with_traceback]
Brett Cannon wrote: > except Exception as exc with tb: > ... I was thinking of that too, plus a matching raise Exception with tb The only use for a 'catch' object would then be for returning from sys.exc_info(), and I'm not sure it's worth creating one just for that rather than using a 2-tuple. -- Greg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Another traceback idea [was: except Exception as err, tb]
Nick Maclaren wrote:
> The instance contains all of the information about the details, such
> as the exact operation, the values and the context (including the
> traceback). It CAN'T be an object, because it is not 'assignable'
> (i.e. a value) - it is inherently bound to its context. You can
> turn it into an object by copying its context into an assignable form,
> but the actual instance is not assignable.
This has given me another idea:
Instead of instantiating the exception when it's raised,
what about instantiating it when it's *caught* instead?
Suppose for a moment that we take what might seem to be
a retrograde step, and go back to the old way of raising
exceptions, where you supply the type and arguments
separately:
raise MyException, ("Bogus value:", x)
Now instead of immediately "normalising" this by creating
the exception, we keep all three parts (type, args, traceback)
separate while we search for a handler.
If we find one of the form
except MyException as e:
then at that point we instantiate the exception, attach the
traceback and assign it to e.
However, if we find one of the form
except MyException:
then we don't need to instantiate it at all! This would
address the issue of efficiently using exceptions for flow
control, since for that kind of use you're probably not
going to want the exception object, in which case your
except clauses will be of the latter form.
Now, I'm not proposing that the raise statement should
actually have the above syntax -- that really would be
a step backwards. Instead it would be required to have
one of the following forms:
raise ExceptionClass
or
raise ExceptionClass(args)
plus optional 'with traceback' clauses in both cases.
However, the apparent instantiation call wouldn't be
made -- it's just part of the syntax.
--
Greg
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] except Exception as err with tb [was: with_traceback]
Proposal changed from "err, tb" to "err with tb" at Brett's good suggestion. The rest of this message is a bit of a sidetrack; skimmers can stop now. Nick Maclaren wrote: >"Jim Jewett" wrote: >> Guido van Rossum wrote: >> > [multiple threads => don't share] whatever object >> > contains the head of the chain of tracebacks >> So (full) exceptions can't be unitary objects. As Shane pointed out[1], the *traceback* could be a unitary object holding all three pieces of information. The problem is that we don't want every except clause adding boilerplate to get the exception back out. [1] http://mail.python.org/pipermail/python-dev/2007-February/071417.html >> In theory, raising an already-instantiated instance could indicate "no >> traceback", which could make pre-cooked exceptions even lighter. > The instance contains all of the information about the details, such > as the exact operation, the values and the context (including the > traceback). ... > ... the action of turning an instance into an object disables > the ability to fix up the exception and continue. ... > I don't know of any in the Unix or Microsoft environments, but > there may be a few in specialised areas. Lisp does it. The cost is that instead of dying, those frames become a continuation, and you need to keep around lots of extra (probable) garbage. Dealing with this cleanly was (once upon a time) one of the advantages of Stackless Python. Unfortunately, extension modules can call back into python from the middle of a C function, so the clean restarting was largely limited to pure python frames; trying to get as close as possible for other code made the implementation fairly complicated. > Harking back to your point, your "already-instantiated instance" > is actually an object derived directly from the exception class, and > everything becomes clear. Because it is an object, any context it > includes was a snapshot and is no longer valid. In your case, you > would want it to have "context: unknown". Rather, I want "context: irrelevant" so it won't bother to keep the context alive. That is clearly the right thing for a normal StopIteration. It isn't the right thing for all pre-instantiated exceptions. Whether it is the right thing often enough to ask those others to use the typical idiom -- maybe. -jJ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Another traceback idea [was: except Exception as err, tb]
On Sat, Mar 03, 2007 at 11:00:53AM +1300, Greg Ewing wrote: > Now, I'm not proposing that the raise statement should > actually have the above syntax -- that really would be > a step backwards. Instead it would be required to have > one of the following forms: > > raise ExceptionClass > > or > > raise ExceptionClass(args) Eep, that's awkward. If you are using exceptions for flow control, why would you use the second form? Why not just allow both exception classes and exception instances to be raised, and only instantiate-at-catch in the case of a raise of a class and a catch with an "as" clause? Then the auto-instantiation becomes a "convenience" feature of catch, safely relegating it to an easily-understood and easily-ignored corner of the user's conceptualization of exception handling. The above also looks a lot like the current syntax, but (unless I'm mistaken) ExceptionClass will be instantiated immediately right now. It seems best not to change the semantics of existing syntax if not necessary. I've been snoozing though this conversation until now, so if I've spoken out of turn, please forgive me. Dustin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] with_traceback
On 3/1/07, Guido van Rossum <[EMAIL PROTECTED]> wrote: > Since by far the most common use case is to create the > exception in the raise statement, the behavior there won't be any > different than it is today; the traceback on precreated objects will > be useless, but folks who precreate them for performance reasons > presumably won't care; and those that create global exception > instances by mistakenly copying the wrong idiom, well, they'll learn > quickly (and a lot more quickly than when we try to override the > exception). Here's a few more examples of code which don't follow the idiom raise ExceptionClass(args) Zope's ZConfig/cmdline.py def addOption(self, spec, pos=None): if pos is None: pos = "", -1, -1 if "=" not in spec: e = ZConfig.ConfigurationSyntaxError( "invalid configuration specifier", *pos) e.specifier = spec raise e The current xml.sax.handler.Error handler includes def error(self, exception): "Handle a recoverable error." raise exception def fatalError(self, exception): "Handle a non-recoverable error." raise exception and is used like this in xml.sax.expatreader.ExpatParser.feed try: # The isFinal parameter is internal to the expat reader. # If it is set to true, expat will check validity of the entire # document. When feeding chunks, they are not normally final - # except when invoked from close. self._parser.Parse(data, isFinal) except expat.error, e: exc = SAXParseException(expat.ErrorString(e.code), e, self) # FIXME: when to invoke error()? self._err_handler.fatalError(exc) Note that the handler may decide to ignore the exception, based on which error occured. The traceback should show where in the handler the exception was raised, and not the point at which the exception was created. ZODB/Connection.py: ... if isinstance(store_return, str): assert oid is not None self._handle_one_serial(oid, store_return, change) else: for oid, serial in store_return: self._handle_one_serial(oid, serial, change) def _handle_one_serial(self, oid, serial, change): if not isinstance(serial, str): raise serial ... Andrew [EMAIL PROTECTED] ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 344 (was: with_traceback)
On 2/26/07, Andrew Dalke <[EMAIL PROTECTED]> wrote:
> My concern when I saw Guido's keynote was the worry that
> people do/might write code like this
>
> NO_END_OF_RECORD = ParserError("Cannot find end of record")
>
> def parse_record(input_file):
>...
> raise NO_END_OF_RECORD
>...
>
>
> That is, create instances at the top of the module, to be used
> later. This code assume that the NO_END_OF_RECORD
> exception instance is never modified.
>
> If the traceback is added to its __traceback__ attribute then
> I see two problems if I were to write code like the above:
>
> - the traceback stays around "forever"
> - the code is no longer thread-safe.
While there are now quite a few threads discussing the problems
related to the proposed __traceback__ attribute (thread-safety being
the primary one, from what I can tease out), I haven't seen anyone
address the rest of PEP 344: the __cause__ and __context__ attributes.
Do those who oppose __traceback__ also oppose __cause__ and
__context__? Should PEP 344 be rejected on these grounds, or should we
just learn not to pre-instance exceptions if we're interested in these
attributes?
Collin Winter
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Another traceback idea [was: except Exception as err, tb]
[EMAIL PROTECTED] wrote: > Why not just allow both exception classes and exception instances to be > raised, and only instantiate-at-catch in the case of a raise of a class > and a catch with an "as" clause? Because that doesn't solve the problem of pre-instantiated exceptions. What I'm proposing is a way of ensuring that a new instance is always created for each raise, so that a traceback can safely be attached to it -- but only if it's going to be used. > The above also looks a lot like the current syntax, but (unless I'm > mistaken) ExceptionClass will be instantiated immediately right now. I think the number of situations where it would make any difference in existing code would be very small. You'd have to go to considerable lengths to detect whether the instantiation was being done at the time of raising or the time of catching. The main difference would be that use of preinstantiated exceptions would become illegal. But this is a 3.0 suggestion, so we can cope with that if we think it's a good enough idea. > It > seems best not to change the semantics of existing syntax if not > necessary. A different syntax could be used, but it would be a big upheaval to ask everyone to un-learn the current way of raising exceptions and go back to something that we thought we had left behind as embarrassingly old-fashioned. And there wouldn't really be any option of supporting both ways, because if you can still raise a preinstantiated exception, it wouldn't be safe to attach a traceback to it, meaning we'd have to support two different ways of handling tracebacks as well. It could all get terribly messy. -- Greg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 344 (was: with_traceback)
Collin Winter wrote: > Do those who oppose __traceback__ also oppose __cause__ and > __context__? They would seem to have the same problems. Whatever solution is adopted for the traceback should probably be applied to them as well, perhaps by generalising the traceback into an "exception context" object that can hold other info. -- Greg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Another traceback idea [was: except Exception as err, tb]
On 3/2/07, Greg Ewing <[EMAIL PROTECTED]> wrote: > This has given me another idea: ... > Now, I'm not proposing that the raise statement should > actually have the above syntax -- that really would be > a step backwards. Instead it would be required to have > one of the following forms: > > raise ExceptionClass > > or > > raise ExceptionClass(args) > > plus optional 'with traceback' clauses in both cases. > However, the apparent instantiation call wouldn't be > made -- it's just part of the syntax. Elsewhere here I listed several examples of existing code which raises an instance which was caught or created earlier. That would not be supported if the raise had to be written in your given forms. Andrew [EMAIL PROTECTED] ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] with_traceback
Greg Ewing wrote: > Michael Foord wrote: > >> With the >> proposed changes, modules that do this would *continue* to work, surely >> ? >> > > Probably, but it might mean they were no longer thread > safe. An exception caught and raised in one thread would > be vulnerable to having its traceback clobbered by > another thread raising the same instance. > Right - but that would still be *no worse* than the current situation where that information isn't available on the instance. The current patterns would continue to work unchanged, but the new information wouldn't be available because a single instance is being reused. > There's also the possibility of a traceback unexpectedly > kept alive causing GC problems -- cycles, files not > closed when you expect, etc. > That *could* be a problem, although explicitly closing files is always a good practise :-) Michael Foord > -- > Greg > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > > ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 344 (was: with_traceback)
On 3/2/07, Collin Winter <[EMAIL PROTECTED]> wrote:
> On 2/26/07, Andrew Dalke <[EMAIL PROTECTED]> wrote:
> > My concern when I saw Guido's keynote was the worry that
> > people do/might write code like this
> >
> > NO_END_OF_RECORD = ParserError("Cannot find end of record")
> >
> > def parse_record(input_file):
> >...
> > raise NO_END_OF_RECORD
> >...
> >
> >
> > That is, create instances at the top of the module, to be used
> > later. This code assume that the NO_END_OF_RECORD
> > exception instance is never modified.
> >
> > If the traceback is added to its __traceback__ attribute then
> > I see two problems if I were to write code like the above:
> >
> > - the traceback stays around "forever"
> > - the code is no longer thread-safe.
>
> While there are now quite a few threads discussing the problems
> related to the proposed __traceback__ attribute (thread-safety being
> the primary one, from what I can tease out), I haven't seen anyone
> address the rest of PEP 344: the __cause__ and __context__ attributes.
>
> Do those who oppose __traceback__ also oppose __cause__ and
> __context__? Should PEP 344 be rejected on these grounds, or should we
> just learn not to pre-instance exceptions if we're interested in these
> attributes?
Those attributes have exactly the same problems as __traceback__. They
exist mostly to save the butt of those folks who didn't even know they
had a problem, so I don't think that saying "if you want this
functionality, don't do X" is very helpful.
I'm afraid we're back at square zero; perhaps we should keep the
existing (type, value, traceback) API rather than going for radical
(and unproven) innovation like "except E as v with tb: ". In
the future we will still have the path open to a "Catch" object as
proposed by Glyph and Greg; all we'd have to do is give the Catch
object sequence semantics during the transition.
I wonder if we should also roll back the new semantics of "except E as
v: " of deleting v when exiting ; that was proposed in
anticipation of the traceback attribute on exceptions. If we're not
doing that, using vanilla variable semantics will give better
migration from 2.6 and fewer surprises.
On 3/2/07, Greg Ewing <[EMAIL PROTECTED]> wrote:
> They would seem to have the same problems. Whatever
> solution is adopted for the traceback should probably
> be applied to them as well, perhaps by generalising
> the traceback into an "exception context" object
> that can hold other info.
That would work for the "Catch" object. It wouldn't work so well for
the original (t, v, tb) API, nor for the "except E as v with tb"
syntax ("except E as v with tb in context for cause" anyone? :-)
The one thing that makes me not want to give up yet is that having the
traceback, context, cause etc. as attributes of the exception object
would seem to make the API for handling exceptions that much cleaner,
compared to having to call sys.exc_info() or even "except E as v with
tb".
So, despite the existence of libraries that pre-create exceptions, how
bad would it really be if we declared that use unsafe? It wouldn't be
hard to add some kind of boobytrap that goes off when pre-created
exceptions are raised multiple times. If this had always been the
semantics I'm sure nobody would have complained and I doubt that it
would have been a common pitfall either (since if it doesn't work,
there's no bad code abusing it, and so there are no bad examples that
newbies could unwittingly emulate).
--
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Another traceback idea [was: except Exception as err, tb]
On 3/2/07, Andrew Dalke <[EMAIL PROTECTED]> wrote:
> On 3/2/07, Greg Ewing <[EMAIL PROTECTED]> wrote:
> > This has given me another idea:
>...
> > Now, I'm not proposing that the raise statement should
> > actually have the above syntax -- that really would be
> > a step backwards. Instead it would be required to have
> > one of the following forms:
> >
> > raise ExceptionClass
> >
> > or
> >
> > raise ExceptionClass(args)
> >
> > plus optional 'with traceback' clauses in both cases.
> > However, the apparent instantiation call wouldn't be
> > made -- it's just part of the syntax.
>
> Elsewhere here I listed several examples of existing
> code which raises an instance which was caught or
> created earlier. That would not be supported if the
> raise had to be written in your given forms.
Raising an instance that was just caught or created a bit earlier
should definitely be supported. Especially this pattern is useful, in
all its variations:
def handle_error(self, exc):
"""Subclass can override to ignore it."""
raise exc
def foo(self):
if self.bar():
self.handle_error(RuntimeError("can't foo because of bar"))
# if it returns, ignore bar
# do foo's thing
The question is, how to distinguish this from an exception created
*much* earlier and raised repeatedly? Adding an attribute pointing to
the "owning thread" might help catch concurrent use, but that wouldn't
help if the pattern is used in a single-threaded environment, even
though it could still be an error there (e.g. in used in a recursive
context).
--
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 344
Guido van Rossum wrote: > [snip..] > The one thing that makes me not want to give up yet is that having the > traceback, context, cause etc. as attributes of the exception object > would seem to make the API for handling exceptions that much cleaner, > compared to having to call sys.exc_info() or even "except E as v with > tb". > > So, despite the existence of libraries that pre-create exceptions, how > bad would it really be if we declared that use unsafe? It wouldn't be > hard to add some kind of boobytrap that goes off when pre-created > exceptions are raised multiple times. If this had always been the > semantics I'm sure nobody would have complained and I doubt that it > would have been a common pitfall either (since if it doesn't work, > there's no bad code abusing it, and so there are no bad examples that > newbies could unwittingly emulate). > > Personally, I think the new attributes and the new syntax are *great* improvements to exception handling. I would be sad to see parts of these proposals not happen. Michael Foord ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] with_traceback
Michael Foord wrote: > Greg Ewing wrote: > > > An exception caught and raised in one thread would > > be vulnerable to having its traceback clobbered by > > another thread raising the same instance. > > Right - but that would still be *no worse* than the current situation > where that information isn't available on the instance. Um -- yes, it would, because currently you don't *expect* the traceback to be available from the exception. If that became the standard way to handle tracebacks, then you would expect it to work reliably. -- Greg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] NOARGS_NULL
Martin v. Löwis wrote: > The style-guide in C is that macro names are spelled in all-caps, so > if you see an all-caps identifier, you expect it to be a macro name. Hi Martin. OK, so somewhere we have: /* NOARGS_NULL: A parameter declared in this way is present only because * the function is registered as a METH_NOARGS function. Due to the * way such functions are called [reference], the parameter will always * be NULL (as the name suggests) and should always be ignored. The * reason for this is [reference|"historical"]. */ #define NOARGS_NULL _noargs_null_ ... this is tongue-in-cheek, but I'm only half-joking (perhaps this is an acceptable compromise). My whole reason for naming it that way was to help people who might be grepping through the source to understand what's going on (and the two references/grep strings are in upper-case elsewhere). FWIW, I'm not overly concerned about the spelling - as the SF comment indicated I thought that the all-caps name might be an issue, but really I'd rather you called it "PyObject *_" than not have it in at all because of the amount of work I'd have to do each time there was a new release without it ;) Thanks, Kev. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 344 (was: with_traceback)
On 3/2/07, Guido van Rossum <[EMAIL PROTECTED]> wrote: > So, despite the existence of libraries that pre-create exceptions, how > bad would it really be if we declared that use unsafe? It wouldn't be > hard to add some kind of boobytrap that goes off when pre-created > exceptions are raised multiple times. If this had always been the > semantics I'm sure nobody would have complained and I doubt that it > would have been a common pitfall either (since if it doesn't work, > there's no bad code abusing it, and so there are no bad examples that > newbies could unwittingly emulate). Here's code from os._execvpe which reraises an exception instance which was created earlier saved_exc = None saved_tb = None for dir in PATH: fullname = path.join(dir, file) try: func(fullname, *argrest) except error, e: tb = sys.exc_info()[2] if (e.errno != ENOENT and e.errno != ENOTDIR and saved_exc is None): saved_exc = e saved_tb = tb if saved_exc: raise error, saved_exc, saved_tb raise error, e, tb Would the boobytrap go off in this case? I think it would, because a "saved_exc" is raised twice. Andrew [EMAIL PROTECTED] ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] with_traceback
Greg Ewing wrote: > Michael Foord wrote: > >> Greg Ewing wrote: >> >> >>> An exception caught and raised in one thread would >>> be vulnerable to having its traceback clobbered by >>> another thread raising the same instance. >>> >> Right - but that would still be *no worse* than the current situation >> where that information isn't available on the instance. >> > > Um -- yes, it would, because currently you don't *expect* > the traceback to be available from the exception. If > that became the standard way to handle tracebacks, then > you would expect it to work reliably. > Um... except that the new attributes *obviously* means that the traceback information is obviously not going to work where you reuse a single instance and to expect otherwise seems naive. If the new pattern *doesn't* break existing code, but means that using a single instance for optimisation (the only justification put forward - re-raising being a slightly different case) makes that information unreliable; then I don't see that as a reason to reject the change. Michael Foord > -- > Greg > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > > ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 344 (was: with_traceback)
Guido van Rossum wrote: > I'm afraid we're back at square zero; perhaps we should keep the > existing (type, value, traceback) API Whatever happens, we should be able to get that down to at most two things: (exception, context) where exception is either a class or an instance, and context includes the traceback and anything else that varies from one raising to another. > rather than going for radical > (and unproven) innovation like "except E as v with tb: ". The "with something" clause isn't mutually exclusive with that -- rather it goes hand-in-hand as a nicer way of getting hold of the context than calling sys.exc_info(). I'd consider it an optional extra. > "except E as v with tb in context for cause" anyone? :-) I was thinking more of except E as v with context: and context would have attributes such as traceback, cause, etc. (Although calling it a "context" could be seriously confusing, considering we also have a "with" statement having something called a context that's completely different.) > The one thing that makes me not want to give up yet is that having the > traceback, context, cause etc. as attributes of the exception object > would seem to make the API for handling exceptions that much cleaner, If we can get it down to a 2-tuple, with no fudging around to allow for sometimes having the args separate from the type but not always, that should simplify it considerably. One thing I would like to get out of it if possible is the chance to raise and catch an exception class without instantiating it at all under favourable circumstances. That should be possible if we allow the first item of the tuple to be either a class or instance, and not bother instantiating if it's a class and gets caught with except E: or except E with context: -- Greg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 344
Personally, +1 for new attributes, -1 for more syntax, +0 for making it bad style to pre-create exceptions. One object should have all you need. On 3/2/07, Michael Foord <[EMAIL PROTECTED]> wrote: > Guido van Rossum wrote: > > [snip..] > > The one thing that makes me not want to give up yet is that having the > > traceback, context, cause etc. as attributes of the exception object > > would seem to make the API for handling exceptions that much cleaner, > > compared to having to call sys.exc_info() or even "except E as v with > > tb". > > > > So, despite the existence of libraries that pre-create exceptions, how > > bad would it really be if we declared that use unsafe? It wouldn't be > > hard to add some kind of boobytrap that goes off when pre-created > > exceptions are raised multiple times. If this had always been the > > semantics I'm sure nobody would have complained and I doubt that it > > would have been a common pitfall either (since if it doesn't work, > > there's no bad code abusing it, and so there are no bad examples that > > newbies could unwittingly emulate). > > > > > Personally, I think the new attributes and the new syntax are *great* > improvements to exception handling. I would be sad to see parts of these > proposals not happen. > > Michael Foord > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] with_traceback
Michael Foord wrote: > Um... except that the new attributes *obviously* means that the > traceback information is obviously not going to work where you reuse a > single instance and to expect otherwise seems naive. Yes, but this means that the __traceback__ attribute couldn't be *the* way of handling tracebacks in Py3k. The old way would still have to be there, and the new way could only ever be a convenience feature that might not be available and might not work properly if it is. That doesn't seem like a tidy situation to me. Py3k is supposed to me making things cleaner, not more messy. -- Greg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 344 (was: with_traceback)
On 3/2/07, Andrew Dalke <[EMAIL PROTECTED]> wrote: > On 3/2/07, Guido van Rossum <[EMAIL PROTECTED]> wrote: > > So, despite the existence of libraries that pre-create exceptions, how > > bad would it really be if we declared that use unsafe? It wouldn't be > > hard to add some kind of boobytrap that goes off when pre-created > > exceptions are raised multiple times. If this had always been the > > semantics I'm sure nobody would have complained and I doubt that it > > would have been a common pitfall either (since if it doesn't work, > > there's no bad code abusing it, and so there are no bad examples that > > newbies could unwittingly emulate). > > Here's code from os._execvpe which reraises an exception > instance which was created earlier > > saved_exc = None > saved_tb = None > for dir in PATH: > fullname = path.join(dir, file) > try: > func(fullname, *argrest) > except error, e: > tb = sys.exc_info()[2] > if (e.errno != ENOENT and e.errno != ENOTDIR > and saved_exc is None): > saved_exc = e > saved_tb = tb > if saved_exc: > raise error, saved_exc, saved_tb > raise error, e, tb > > Would the boobytrap go off in this case? I think it would, > because a "saved_exc" is raised twice. It shouldn't. This is a requirement for the boobytrap (which is vaporware as yet :-). Perhaps we could allow reraising whenever the existing traceback chain contains a reference to a frame that is an ancestor of (or equal to) the newly raising frame? -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 344 (was: with_traceback)
On 3/2/07, Greg Ewing <[EMAIL PROTECTED]> wrote: > Guido van Rossum wrote: > > > I'm afraid we're back at square zero; perhaps we should keep the > > existing (type, value, traceback) API > > Whatever happens, we should be able to get that down > to at most two things: (exception, context) where > exception is either a class or an instance, and > context includes the traceback and anything else > that varies from one raising to another. > > > rather than going for radical > > (and unproven) innovation like "except E as v with tb: ". > > The "with something" clause isn't mutually exclusive > with that -- rather it goes hand-in-hand as a nicer > way of getting hold of the context than calling > sys.exc_info(). I'd consider it an optional extra. > > > "except E as v with tb in context for cause" anyone? :-) > > I was thinking more of > >except E as v with context: > > and context would have attributes such as traceback, > cause, etc. > > (Although calling it a "context" could be seriously > confusing, considering we also have a "with" statement > having something called a context that's completely > different.) > > > The one thing that makes me not want to give up yet is that having the > > traceback, context, cause etc. as attributes of the exception object > > would seem to make the API for handling exceptions that much cleaner, > > If we can get it down to a 2-tuple, with no fudging > around to allow for sometimes having the args separate > from the type but not always, that should simplify it > considerably. > > One thing I would like to get out of it if possible > is the chance to raise and catch an exception class > without instantiating it at all under favourable > circumstances. That should be possible if we allow > the first item of the tuple to be either a class or > instance, and not bother instantiating if it's a > class and gets caught with > >except E: > > or > >except E with context: But what's the advantage of not instantiating the exception if we instantiate the context instead? I really think that it would be nicer if the exception object *was* the context. We'd still not have to instantiate if it was just "except E:" but that can be handled behind the scenes. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Performance of pre-creating exceptions?
$ python2.5 -m timeit -r 10 -n 100 -s 'class Foo(Exception): pass' 'try: raise Foo()' 'except: pass' 100 loops, best of 10: 2.49 usec per loop $ python2.5 -m timeit -r 10 -n 100 -s 'class Foo(Exception):' -s ' def __init__(self): pass' 'try: raise Foo()' 'except: pass' 100 loops, best of 10: 3.15 usec per loop $ python2.5 -m timeit -r 10 -n 100 -s 'e = Exception()' 'try: raise e' 'except: pass' 100 loops, best of 10: 2.03 usec per loop We can get more than half of the benefit simply by using a default __init__ rather than a python one. If you need custom attributes but they're predefined you could subclass the exception and have them as class attributes. Given that, is there really a need to pre-create exceptions? -- Adam Olsen, aka Rhamphoryncus ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 344 (was: with_traceback)
Guido van Rossum wrote: > But what's the advantage of not instantiating the exception if we > instantiate the context instead? Probably not much. But most control-flow-exception catching will just be 'except E:' in which case you don't need to instantiate anything. (Assuming we get rid of traceback objects and just link the frames together directly.) -- Greg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 344 (was: with_traceback)
Guido van Rossum wrote: > Perhaps we could allow reraising whenever the > existing traceback chain contains a reference to a frame that is an > ancestor of (or equal to) the newly raising frame? This is starting to sound terribly hairy. Would it help if a different syntax were used for raising and reraising? Raising would complain if there were already a traceback, and reraising would expect an existing traceback and extend it. -- Greg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Performance of pre-creating exceptions?
Often people build mental models of performance that have little bearing on reality. Thanks for measuring! On 3/2/07, Adam Olsen <[EMAIL PROTECTED]> wrote: > $ python2.5 -m timeit -r 10 -n 100 -s 'class Foo(Exception): pass' > 'try: raise Foo()' 'except: pass' > 100 loops, best of 10: 2.49 usec per loop > $ python2.5 -m timeit -r 10 -n 100 -s 'class Foo(Exception):' -s ' > def __init__(self): pass' 'try: raise Foo()' 'except: pass' > 100 loops, best of 10: 3.15 usec per loop > $ python2.5 -m timeit -r 10 -n 100 -s 'e = Exception()' 'try: > raise e' 'except: pass' > 100 loops, best of 10: 2.03 usec per loop > > We can get more than half of the benefit simply by using a default > __init__ rather than a python one. If you need custom attributes but > they're predefined you could subclass the exception and have them as > class attributes. Given that, is there really a need to pre-create > exceptions? > > -- > Adam Olsen, aka Rhamphoryncus > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Performance of pre-creating exceptions?
On 3/2/07, Adam Olsen <[EMAIL PROTECTED]> wrote:
> We can get more than half of the benefit simply by using a default
> __init__ rather than a python one. If you need custom attributes but
> they're predefined you could subclass the exception and have them as
> class attributes. Given that, is there really a need to pre-create
> exceptions?
The only real world example of (re)using pre-computed exceptions
I found was in pyparsing. Here are two examples:
def parseImpl( self, instring, loc, doActions=True ):
if (instring[loc] == self.firstMatchChar and
(self.matchLen==1 or instring.startswith(self.match,loc)) ):
return loc+self.matchLen, self.match
#~ raise ParseException( instring, loc, self.errmsg )
exc = self.myException
exc.loc = loc
exc.pstr = instring
raise exc
(The Token's constructor is
class Token(ParserElement):
def __init__( self ):
super(Token,self).__init__( savelist=False )
self.myException = ParseException("",0,"",self)
and the exception class uses __slots__ thusly:
class ParseBaseException(Exception):
"""base exception class for all parsing runtime exceptions"""
__slots__ = ( "loc","msg","pstr","parserElement" )
# Performance tuning: we construct a *lot* of these, so keep this
# constructor as small and fast as possible
def __init__( self, pstr, loc, msg, elem=None ):
self.loc = loc
self.msg = msg
self.pstr = pstr
self.parserElement = elem
so you can see that each raised exception modifies 2 of
the 4 instance variables in a ParseException.)
-and-
# this method gets repeatedly called during backtracking with the
same arguments -
# we can cache these arguments and save ourselves the trouble of re-parsing
# the contained expression
def _parseCache( self, instring, loc, doActions=True, callPreParse=True ):
lookup = (self,instring,loc,callPreParse)
if lookup in ParserElement._exprArgCache:
value = ParserElement._exprArgCache[ lookup ]
if isinstance(value,Exception):
if isinstance(value,ParseBaseException):
value.loc = loc
raise value
return value
else:
try:
ParserElement._exprArgCache[ lookup ] = \
value = self._parseNoCache( instring, loc,
doActions, callPreParse )
return value
except ParseBaseException, pe:
ParserElement._exprArgCache[ lookup ] = pe
raise
The first definitely has the look of a change for better performance.
I have not asked the author nor researched to learn how much gain
there was with this code.
Because the saved exception is tweaked each time (hence not
thread-safe), your timing tests aren't directly relevant as your
solution of creating the exception using the default constructor
then tweaking the instance attributes directly would end up
doing 4 setattrs instead of 2.
% python -m timeit -r 10 -n 100 -s 'e = Exception()' 'try: raise
e' 'except: pass'
100 loops, best of 10: 1.55 usec per loop
% python -m timeit -r 10 -n 100 -s 'e = Exception()' 'try:
e.x=1;e.y=2;raise e' 'except: pass'
100 loops, best of 10: 1.98 usec per loop
so 4 attributes should be about 2.5usec, or 25% slower than 2 attributes.
There's also a timing difference between looking for the exception
class name in module scope vs. using self.myException
I've tried to find other examples but couldn't in the 20 or so packages
I have on my laptop. I used searches like
# find module variables assigned to exception instances
egrep "^[a-z].*=.*Error\(" *.py
egrep "^[a-z].*=.*Exception\(" *.py
# find likely instances being raised
grep "^ *raise [a-z]" *.py
# find likely cases of 3-arg raises
grep "^ *raise .*,.*," *.py
to find candidates. Nearly all false positives.
Andrew
[EMAIL PROTECTED]
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
