Re: Missing something on exception handling in Python 3

[Steven D'Aprano]

On Tue, 27 Aug 2013 00:00:36 -0500, Skip Montanaro wrote:

 I found this question/answer on Stack Overflow:
 
 http://stackoverflow.com/questions/15123137
 
 but after fiddling around with it, I can't find a solution that works
 for Python 3.2 and 3.3, let alone 2.x.  In 3.2, exceptions have both
 __cause__ and __context__ attributes.  I tried setting both to None (in
 3.2), but I still get the full double traceback:


Yes, raise from None only works in Python 3.3.

My way of dealing with this is to use raise from None and just accept 
that 3.1 and 3.2 users will see double tracebacks :-(

All the more reason to encourage people to go straight to 3.3 or better 
and skip 3.1 and 3.2 :-)



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Missing something on exception handling in Python 3

[Skip Montanaro]

I have code in the pylockfile package that looks roughly like this
(apologies for the indentation - I'm growing to dislike Gmail for
typing code fragments):

try:
write_pid_to_lockfile(somefile)
except OSError as exc:
if conditions_i_can_handle:
   do_other_stuff...
else:
raise LockFailed(Failed to create %s % self.path)

Thinking I would just get the final exception (not the one I caught
originally), I was surprised to see this output on my screen:

 lock.acquire()
Traceback (most recent call last):
  File lockfile/pidlockfile.py, line 80, in acquire
write_pid_to_pidfile(self.path)
  File lockfile/pidlockfile.py, line 163, in write_pid_to_pidfile
pidfile_fd = os.open(pidfile_path, open_flags, open_mode)
OSError: [Errno 13] Permission denied: '/tmp/skip/lock'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File stdin, line 1, in module
  File lockfile/pidlockfile.py, line 94, in acquire
raise LockFailed(failed to create %s % self.path)
lockfile.LockFailed: failed to create /tmp/skip/lock

When I rung the same code with Python 2.7, I get the exception output
I anticipate:

 lock.acquire()
Traceback (most recent call last):
  File stdin, line 1, in module
  File lockfile/pidlockfile.py, line 94, in acquire
raise LockFailed(failed to create %s % self.path)
LockFailed: failed to create /tmp/skip/lock

It appears exception handling changed in Python 3.  How do I suppress
the lower level OSError?

Thx,

Skip
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Missing something on exception handling in Python 3

[MRAB]

On 27/08/2013 02:48, Skip Montanaro wrote:

I have code in the pylockfile package that looks roughly like this
(apologies for the indentation - I'm growing to dislike Gmail for
typing code fragments):

try:
 write_pid_to_lockfile(somefile)
except OSError as exc:
 if conditions_i_can_handle:
do_other_stuff...
 else:
 raise LockFailed(Failed to create %s % self.path)

Thinking I would just get the final exception (not the one I caught
originally), I was surprised to see this output on my screen:


lock.acquire()

Traceback (most recent call last):
   File lockfile/pidlockfile.py, line 80, in acquire
 write_pid_to_pidfile(self.path)
   File lockfile/pidlockfile.py, line 163, in write_pid_to_pidfile
 pidfile_fd = os.open(pidfile_path, open_flags, open_mode)
OSError: [Errno 13] Permission denied: '/tmp/skip/lock'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
   File stdin, line 1, in module
   File lockfile/pidlockfile.py, line 94, in acquire
 raise LockFailed(failed to create %s % self.path)
lockfile.LockFailed: failed to create /tmp/skip/lock

When I rung the same code with Python 2.7, I get the exception output
I anticipate:


lock.acquire()

Traceback (most recent call last):
   File stdin, line 1, in module
   File lockfile/pidlockfile.py, line 94, in acquire
 raise LockFailed(failed to create %s % self.path)
LockFailed: failed to create /tmp/skip/lock

It appears exception handling changed in Python 3.  How do I suppress
the lower level OSError?


Do this:

raise LockFailed(Failed to create %s % self.path) from None

--
http://mail.python.org/mailman/listinfo/python-list

Re: Missing something on exception handling in Python 3

[Skip Montanaro]

 Do this:

 raise LockFailed(Failed to create %s % self.path) from None

Thanks. Is there some construct which will work in 2.x and 3.x?

Skip
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Missing something on exception handling in Python 3

[Ethan Furman]

On 08/26/2013 07:49 PM, Skip Montanaro wrote:

Do this:

 raise LockFailed(Failed to create %s % self.path) from None


Thanks. Is there some construct which will work in 2.x and 3.x?


Something like this (untested):

exc = None
try:
write_pid_to_lockfile(somefile)
except OSError as exc:
exc = sys.exc_info()[0]   # not sure of index
if exc:
if conditions_i_can_handle:
do_other_stuff...
else:
raise LockFailed(Failed to create %s % self.path)

Since you're raising outside the try...except block you won't see the chained 
exception in Python3.

--
~Ethan~
--
http://mail.python.org/mailman/listinfo/python-list

Re: Missing something on exception handling in Python 3

[Skip Montanaro]

I found this question/answer on Stack Overflow:

http://stackoverflow.com/questions/15123137

but after fiddling around with it, I can't find a solution that works
for Python 3.2 and 3.3, let alone 2.x.  In 3.2, exceptions have both
__cause__ and __context__ attributes.  I tried setting both to None
(in 3.2), but I still get the full double traceback:

 try: 1/0
... except ZeroDivisionError:
...exc = TypeError()
...exc.__context__ = None
...exc.__cause__ = None
...raise exc
...
Error in sys.excepthook:
IndexError: tuple index out of range

Original exception was:
Traceback (most recent call last):
  File stdin, line 1, in module
ZeroDivisionError: division by zero

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File stdin, line 6, in module
TypeError

The sys.excepthook error is because I have a custom interactive
sys.excepthook, which is itself apparently broken in Python 3.2.

I've looked at PEP 409 and 415, but just get more confused.

*sigh*

Maybe I'll try again tomorrow when I've had a bit of sleep... and take
a closer look at Ethan's suggestion.

Skip
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exception handling in Python 3.x

[Arnaud Delobelle]

Paul Rubin no.em...@nospam.invalid writes:

 Steven D'Aprano steve+comp.lang.pyt...@pearwood.info writes:
 Apart from this horrible idiom:

 def func(iterable):
 it = iter(iterable)
 failed = False
 try:
 x = next(it)
 except StopIteration:
 failed = True
 if failed:
 raise ValueError(can't process empty iterable)
 print(x)


 or similar, is there really no way to avoid these chained exceptions?

 Seems like yet another example of people doing messy things with
 exceptions that can easily be done with iterators and itertools:

 from itertools import islice

 def func(iterable):
   xs = list(islice(iter(iterable), 1))
   if len(xs) == 0:
  raise ValueError(...)
   print xs[0]

 It's really unfortunate, though, that Python 3 didn't offer a way to
 peek at the next element of an iterable and test emptiness directly.

I missed the start of this discussion but there are two simpler ways:

def func(iterable):
for x in iterable:
print(x)
return
raise ValueError(... empty iterable)

Or using 3.x's next's optional second argument:

_nonext=object()
def func(iterable):
x = next(iter(iterable), _nonext)
if x is _nonext:
raise ValueError(... empty iterable)
print(x)

-- 
Arnaud
-- 
http://mail.python.org/mailman/listinfo/python-list

RE: Exception handling in Python 3.x

[Rob Richardson]

Arnaud,

Wouldn't your first suggestion exit after the first element in iterable?
And would your second suggestion throw an exception after normal
processing of all elements in the interator?

RobR

-Original Message-


I missed the start of this discussion but there are two simpler ways:

def func(iterable):
for x in iterable:
print(x)
return
raise ValueError(... empty iterable)

Or using 3.x's next's optional second argument:

_nonext=object()
def func(iterable):
x = next(iter(iterable), _nonext)
if x is _nonext:
raise ValueError(... empty iterable)
print(x)

-- 
Arnaud
-- 
http://mail.python.org/mailman/listinfo/python-list
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exception handling in Python 3.x

[Ethan Furman]

Please don't top-post.

Rob Richardson wrote:


-Original Message-


I missed the start of this discussion but there are two simpler ways:

def func(iterable):
for x in iterable:
print(x)
return
raise ValueError(... empty iterable)

Or using 3.x's next's optional second argument:

_nonext=object()
def func(iterable):
x = next(iter(iterable), _nonext)
if x is _nonext:
raise ValueError(... empty iterable)
print(x)



 Arnaud,

 Wouldn't your first suggestion exit after the first element in
 iterable?

No, it hit's return instead.

 And would your second suggestion throw an exception after normal
 processing of all elements in the interator?

Looks like the second solution doesn't process the entire iterable, just 
it's first element.


~Ethan~
--
http://mail.python.org/mailman/listinfo/python-list

Re: Exception handling in Python 3.x

[Arnaud Delobelle]

Rob Richardson rob.richard...@rad-con.com writes:

You shouldn't top-post!

 Arnaud,

 Wouldn't your first suggestion exit after the first element in iterable?

Yes, after printing that element, which is what the code I quoted did.

 And would your second suggestion throw an exception after normal
 processing of all elements in the interator?

No. It would have the same behaviour as the first one.

 RobR

 -Original Message-


 I missed the start of this discussion but there are two simpler ways:

 def func(iterable):
 for x in iterable:
 print(x)
 return
 raise ValueError(... empty iterable)

 Or using 3.x's next's optional second argument:

 _nonext=object()
 def func(iterable):
 x = next(iter(iterable), _nonext)
 if x is _nonext:
 raise ValueError(... empty iterable)
 print(x)

 -- 
 Arnaud

-- 
Arnaud
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exception handling in Python 3.x

[Ethan Furman]

Arnaud Delobelle wrote:


I missed the start of this discussion but there are two simpler ways:

def func(iterable):
for x in iterable:
print(x)
return
raise ValueError(... empty iterable)



For the immediate case this is a cool solution.

Unfortunately, it doesn't fix the unwanted nesting of exceptions problem.

~Ethan~
--
http://mail.python.org/mailman/listinfo/python-list

Re: Exception handling in Python 3.x

[Ethan Furman]

Ethan Furman wrote:

Please don't top-post.

Rob Richardson wrote:


-Original Message-


I missed the start of this discussion but there are two simpler ways:

def func(iterable):
for x in iterable:
print(x)
return
raise ValueError(... empty iterable)

Or using 3.x's next's optional second argument:

_nonext=object()
def func(iterable):
x = next(iter(iterable), _nonext)
if x is _nonext:
raise ValueError(... empty iterable)
print(x)



  Arnaud,
 
  Wouldn't your first suggestion exit after the first element in
  iterable?

No, it hit's return instead.


Doh -- Yes, it does.

It seems both solutions only get the first element, not all elements in 
the iterator...


Maybe this instead:

def func(iterable):
for x in iterable:
break
else:
raise ValueError(... empty iterable)
for xx in chain((x, ), iterable):
process(xx)

Can't say as I care for this -- better to fix the unwanted nesting in 
the tracebacks from raise.


~Ethan~
--
http://mail.python.org/mailman/listinfo/python-list

Re: Exception handling in Python 3.x

[Ethan Furman]

Ethan Furman wrote:

Arnaud Delobelle wrote:


I missed the start of this discussion but there are two simpler ways:

def func(iterable):
for x in iterable:
print(x)
return
raise ValueError(... empty iterable)



For the immediate case this is a cool solution.



Drat -- I have to take that back -- the OP stated:

 The intention is:

 * detect an empty iterator by catching StopIteration;
 * if the iterator is empty, raise a ValueError;
 * otherwise process the iterator.


Presumably, the print(x) would be replaced with code that processed the 
entire iterable (including x, of course), and not just its first element.


~Ethan~
--
http://mail.python.org/mailman/listinfo/python-list

Re: Exception handling in Python 3.x

[Arnaud Delobelle]

Ethan Furman et...@stoneleaf.us writes:

 Ethan Furman wrote:
 Arnaud Delobelle wrote:

 I missed the start of this discussion but there are two simpler ways:

 def func(iterable):
 for x in iterable:
 print(x)
 return
 raise ValueError(... empty iterable)


 For the immediate case this is a cool solution.


 Drat -- I have to take that back -- the OP stated:

 The intention is:

 * detect an empty iterator by catching StopIteration;
 * if the iterator is empty, raise a ValueError;
 * otherwise process the iterator.


 Presumably, the print(x) would be replaced with code that processed
 the entire iterable (including x, of course), and not just its first
 element.

As I had stated before, I didn't where the discussion started from.  I
replied to code posted by Steven D'Aprano and Paul Rubin.  My code
snippet was equivalent in functionality to theirs, only a little
simpler.

Now if one wants to raise an exception if an iterator is empty, else
process it somehow, it must mean that the iterator needs to have at
least one element for the processing to be meaningful and so it can be
thought of as a function of one element and of one iterator:

process(first, others)

which never needs to raise an exception (at least related to the number
of items in the iterator).  Therefore you can write your function as
follows: 

def func(iterable):
iterator = iter(iterable)
for first in iterable:
return process(first, iterator)
else:
raise ValueError(need non-empty iterable)

-- 
Arnaud
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exception handling in Python 3.x

[Mark Wooding]

John Nagle na...@animats.com writes:

PEP 255, like too much Python literature, doesn't distinguish
 clearly between the language definition and implementation detail.  It
 says The mechanics of StopIteration are low-level details, much like
 the mechanics of IndexError in Python 2.1.  Applications shouldn't be
 explicitly using StopIteration.

You've twisted the words by quoting them out of context, and have
attempted to force a misinterpretation of `low-level details' as
`implementation detail'.

That text comes from a question-and-answer section, in response to the
question `why not force termination to be spelled StopIteration?'.
This is a fine answer to the question: the details of the (preexisting
-- see PEP 234) iteration protocol are abstracted by the generator
syntax.  But it doesn't at all mean that the StopIteration exception
isn't an official, use-visible part of Python.

IronPython doesn't do StopIteration the same way CPython does.

 http://ironpython.codeplex.com/wikipage?title=IPy1.0.xCPyDifferences

IronPython's behaviour when you try to fetch items from a spent
generator is different.  It still implements the same iterator protocol,
and raises StopIteration when it has no more items to yield.

You're not stupid, but you'd have to be in order to think that these
references support your claim that

  You're not entitled to assume that StopIteration is how a generator
  exits.  That's a CPyton thing; generators were a retrofit, and
  that's how they were hacked in.  Other implementations may do
  generators differently.

I don't want to conclude that you're not arguing in good faith but I'm
not seeing many other possibilities.

-- [mdw]
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exception handling in Python 3.x

[Steve Holden]

On 12/7/2010 5:58 AM, John Nagle wrote:
PEP 255, like too much Python literature, doesn't distinguish clearly
 between the language definition and implementation detail. It says
 The mechanics of StopIteration are low-level details, much like the
 mechanics of IndexError in Python 2.1.  Applications shouldn't be
 explicitly using StopIteration.
  
So you don't think that we should rely on iterables with no __iter__()
method to raise IndexError to terminate iterations when their
__getitem__() is called with an invalid index? The IndexError mechanism
was, to the best of my less-than-complete knowledge, used by all pre-2.2
implementations. The quoted paragraph appears to be intended to reassure
the applications programmer that there is no normal need to handle
StopIteration specially - just as there was no need to handle IndexError
specially.

IronPython doesn't do StopIteration the same way CPython does.
 
 http://ironpython.codeplex.com/wikipage?title=IPy1.0.xCPyDifferences
 
Perhaps not, but the only difference is what happens on repeated calls
to next() after the iterator is exhausted. The iterator still terminates
by raising a StopIteration error.

I have no idea what Shed Skin does, but to the extent that iterators
don't raise StopIteration on exhaustion I'd say it is in error.

regards
 Steve
-- 
Steve Holden   +1 571 484 6266   +1 800 494 3119
PyCon 2011 Atlanta March 9-17   http://us.pycon.org/
See Python Video!   http://python.mirocommunity.org/
Holden Web LLC http://www.holdenweb.com/

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exception handling in Python 3.x

[Steve Holden]

On 12/7/2010 1:48 AM, MRAB wrote:
 Perhaps Python could use Guido's time machine to check whether the
 sequence will yield another object in the future. :-)

Since there's only one time machine that would effectively be a lock
across all Python interpreters.

regards
 Steve
-- 
Steve Holden   +1 571 484 6266   +1 800 494 3119
PyCon 2011 Atlanta March 9-17   http://us.pycon.org/
See Python Video!   http://python.mirocommunity.org/
Holden Web LLC http://www.holdenweb.com/
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exception handling in Python 3.x

[John Nagle]

On 12/3/2010 5:04 AM, Steven D'Aprano wrote:

Consider the following common exception handling idiom:

def func(iterable):
 it = iter(iterable)
 try:
 x = next(it)
 except StopIteration:
 raise ValueError(can't process empty iterable)
 print(x)

The intention is:

* detect an empty iterator by catching StopIteration;
* if the iterator is empty, raise a ValueError;
* otherwise process the iterator.

Note that StopIteration is an internal detail of no relevance whatsoever
to the caller. Expose this is unnecessary at best and confusing at worst.


Right.  You're not entitled to assume that StopIteration is
how a generator exits.  That's a CPyton thing; generators were
a retrofit, and that's how they were hacked in.  Other implementations
may do generators differently.

John Nagle
--
http://mail.python.org/mailman/listinfo/python-list

Re: Exception handling in Python 3.x

[Paul Rubin]

Steven D'Aprano steve+comp.lang.pyt...@pearwood.info writes:
 Apart from this horrible idiom:

 def func(iterable):
 it = iter(iterable)
 failed = False
 try:
 x = next(it)
 except StopIteration:
 failed = True
 if failed:
 raise ValueError(can't process empty iterable)
 print(x)


 or similar, is there really no way to avoid these chained exceptions?

Seems like yet another example of people doing messy things with
exceptions that can easily be done with iterators and itertools:

from itertools import islice

def func(iterable):
  xs = list(islice(iter(iterable), 1))
  if len(xs) == 0:
 raise ValueError(...)
  print xs[0]

It's really unfortunate, though, that Python 3 didn't offer a way to
peek at the next element of an iterable and test emptiness directly.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exception handling in Python 3.x

[Mark Wooding]

John Nagle na...@animats.com writes:

 Right.  You're not entitled to assume that StopIteration is how a
 generator exits.  That's a CPyton thing; generators were a retrofit,
 and that's how they were hacked in.  Other implementations may do
 generators differently.

This is simply wrong.  The StopIteration exception is a clear part of
the generator protocol as described in 5.2.8 of the language reference;
the language reference also refers to 3.5 of the library reference,
which describes the iterator protocol (note, not the generator
implementation -- all iterators work the same way), and explicitly
mentions StopIteration as part of the protocol.

-- [mdw]
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exception handling in Python 3.x

[Steven D'Aprano]

On Mon, 06 Dec 2010 13:13:40 -0800, Paul Rubin wrote:

 It's really unfortunate, though, that Python 3 didn't offer a way to
 peek at the next element of an iterable and test emptiness directly.

This idea of peekable iterables just won't die, despite the obvious flaws 
in the idea.

There's no general way of telling whether or not a lazy sequence is done 
except to actually generate the next value, and caching that value is not 
appropriate for all such sequences since it could depend on factors which 
have changed between the call to peek and the call to next.

If you want to implement a peek method in your own iterables, go right 
ahead. But you can't make arbitrary iterables peekable without making a 
significant class of them buggy.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exception handling in Python 3.x

[MRAB]

On 07/12/2010 00:23, Steven D'Aprano wrote:

On Mon, 06 Dec 2010 13:13:40 -0800, Paul Rubin wrote:


It's really unfortunate, though, that Python 3 didn't offer a way to
peek at the next element of an iterable and test emptiness directly.


This idea of peekable iterables just won't die, despite the obvious flaws
in the idea.

There's no general way of telling whether or not a lazy sequence is done
except to actually generate the next value, and caching that value is not
appropriate for all such sequences since it could depend on factors which
have changed between the call to peek and the call to next.

If you want to implement a peek method in your own iterables, go right
ahead. But you can't make arbitrary iterables peekable without making a
significant class of them buggy.


Perhaps Python could use Guido's time machine to check whether the
sequence will yield another object in the future. :-)
--
http://mail.python.org/mailman/listinfo/python-list

Re: Exception handling in Python 3.x

[John Nagle]

On 12/6/2010 2:24 PM, Mark Wooding wrote:

John Naglena...@animats.com  writes:


Right.  You're not entitled to assume that StopIteration is how a
generator exits.  That's a CPyton thing; generators were a retrofit,
and that's how they were hacked in.  Other implementations may do
generators differently.


This is simply wrong.  The StopIteration exception is a clear part of
the generator protocol as described in 5.2.8 of the language reference;
the language reference also refers to 3.5 of the library reference,
which describes the iterator protocol (note, not the generator
implementation -- all iterators work the same way), and explicitly
mentions StopIteration as part of the protocol.

-- [mdw]


   PEP 255, like too much Python literature, doesn't distinguish clearly
between the language definition and implementation detail. It says
The mechanics of StopIteration are low-level details, much like the
mechanics of IndexError in Python 2.1.  Applications shouldn't be
explicitly using StopIteration.

   IronPython doesn't do StopIteration the same way CPython does.

http://ironpython.codeplex.com/wikipage?title=IPy1.0.xCPyDifferences

   Neither does Shed Skin.

John Nagle
--
http://mail.python.org/mailman/listinfo/python-list

Re: Exception handling in Python 3.x

[John Nagle]

On 12/6/2010 4:23 PM, Steven D'Aprano wrote:

On Mon, 06 Dec 2010 13:13:40 -0800, Paul Rubin wrote:


It's really unfortunate, though, that Python 3 didn't offer a way to
peek at the next element of an iterable and test emptiness directly.


This idea of peekable iterables just won't die, despite the obvious flaws
in the idea.

There's no general way of telling whether or not a lazy sequence is done
except to actually generate the next value, and caching that value is not
appropriate for all such sequences since it could depend on factors which
have changed between the call to peek and the call to next.


   Right.

   Pascal had the predicates eoln(file) and eof(file), which
were tests for end of line and end of file made before reading.  This
caused much grief with interactive input, because the test would
stall waiting for the user to type something.  Wirth originally
intended Pascal for batch jobs, and his version didn't translate
well to interactive use.  (Wirth fell in love with his original
recursive-descent compiler, which was simple but limited.  He
hated to have language features that didn't fit his compiler model
well. This held the language back and eventually killed it.)

   C I/O returned a unique value on EOF, but there was no way to
test for it before reading.  Works much better.  The same issues apply
to pipes, sockets, qeueues, interprocess communication, etc.

John Nagle

--
http://mail.python.org/mailman/listinfo/python-list

Exception handling in Python 3.x

[Steven D'Aprano]

Consider the following common exception handling idiom:

def func(iterable):
it = iter(iterable)
try:
x = next(it)
except StopIteration:
raise ValueError(can't process empty iterable)
print(x)

The intention is:

* detect an empty iterator by catching StopIteration;
* if the iterator is empty, raise a ValueError;
* otherwise process the iterator.

Note that StopIteration is an internal detail of no relevance whatsoever 
to the caller. Expose this is unnecessary at best and confusing at worst.


In Python 2.6 this idiom works as intended:

 func([])
Traceback (most recent call last):
  File stdin, line 1, in module
  File stdin, line 6, in func
ValueError: can't process empty iterable

There is no sign of the StopIteration, and nor should there be.

But Python 3.1 changes this behaviour, exposing the unimportant 
StopIteration and leading to a more complicated and confusing traceback:

 func([])
Traceback (most recent call last):
  File stdin, line 4, in func
StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File stdin, line 1, in module
  File stdin, line 6, in func
ValueError: can't process empty iterable


I understand the rational of this approach -- it is to assist in 
debugging code where the except block is buggy and raises an error. But a 
deliberate and explicit call to raise is not a buggy except block. It is 
terribly inappropriate for the common use-case of catching one exception 
and substituting another.

I note that the PEP explicitly notes this use-case, but merely sweeps it 
under the carpet:

Open Issue: Suppressing Context
As written, this PEP makes it impossible to suppress '__context__',
since setting exc.__context__ to None in an 'except' or 'finally'
clause will only result in it being set again when exc is raised.

http://www.python.org/dev/peps/pep-3134/


Apart from this horrible idiom:

def func(iterable):
it = iter(iterable)
failed = False
try:
x = next(it)
except StopIteration:
failed = True
if failed:
raise ValueError(can't process empty iterable)
print(x)


or similar, is there really no way to avoid these chained exceptions?



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exception handling in Python 3.x

[Hrvoje Niksic]

Steven D'Aprano steve+comp.lang.pyt...@pearwood.info writes:

 Consider the following common exception handling idiom:

 def func(iterable):
 it = iter(iterable)
 try:
 x = next(it)
 except StopIteration:
 raise ValueError(can't process empty iterable)
 print(x)

Not exactly what you're looking for, but another way to express the
above is:

def func(iterable):
for x in iterable:
break
else:
raise ValueError(can't process empty iterable)
print(x)

Otherwise, I completely agree that being unable to completely replace
the original exception is an annoyance at best.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exception handling in Python 3.x

[Peter Otten]

Steven D'Aprano wrote:

 Consider the following common exception handling idiom:
 
 def func(iterable):
 it = iter(iterable)
 try:
 x = next(it)
 except StopIteration:
 raise ValueError(can't process empty iterable)
 print(x)
 
 The intention is:
 
 * detect an empty iterator by catching StopIteration;
 * if the iterator is empty, raise a ValueError;
 * otherwise process the iterator.
 
 Note that StopIteration is an internal detail of no relevance whatsoever
 to the caller. Expose this is unnecessary at best and confusing at worst.

http://mail.python.org/pipermail/python-list/2010-October/1258606.html
http://mail.python.org/pipermail/python-list/2010-October/1259024.html
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exception handling in Python 3.x

[Hrvoje Niksic]

Peter Otten __pete...@web.de writes:

 Note that StopIteration is an internal detail of no relevance whatsoever
 to the caller. Expose this is unnecessary at best and confusing at worst.

 http://mail.python.org/pipermail/python-list/2010-October/1258606.html
 http://mail.python.org/pipermail/python-list/2010-October/1259024.html

Both of these involve suppressing the chaining at the wrong place,
namely in the outer handler or, worse yet, in the exception display
mechanism.  Steven, on the other hand, wants his *inner* handler to
express that the original exception was an implementation detail, a
business exception such as StopIteration, that is completely irrelevant
to the actual exception being raised.  The outer handler is the wrong
place to suppress the chaining because it has no way of distinguishing
Steven's case from a genuine case of a new exception unexpectedly
occurring during handling of the original exception.

One solution would be for raise inside except to not use the context.
For example:

try:
  {}[1]
except KeyError:
  1/0

would behave as before, but:

But:

try:
  {}[1]
except KeyError:
  raise Exception(my error)

...would raise the custom error forgetting the KeyError.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exception handling in Python 3.x

[Peter Otten]

Hrvoje Niksic wrote:

 Peter Otten __pete...@web.de writes:
 
 Note that StopIteration is an internal detail of no relevance whatsoever
 to the caller. Expose this is unnecessary at best and confusing at
 worst.

 http://mail.python.org/pipermail/python-list/2010-October/1258606.html
 http://mail.python.org/pipermail/python-list/2010-October/1259024.html
 
 Both of these involve suppressing the chaining at the wrong place,
 namely in the outer handler or, worse yet, in the exception display
 mechanism.  Steven, on the other hand, wants his *inner* handler to
 express that the original exception was an implementation detail, a
 business exception such as StopIteration, that is completely irrelevant
 to the actual exception being raised.  The outer handler is the wrong
 place to suppress the chaining because it has no way of distinguishing
 Steven's case from a genuine case of a new exception unexpectedly
 occurring during handling of the original exception.

To quote the Rolling Stones: You can't always get what you want. 

After rereading the original post I still don't get why the workarounds 
provided in those links aren't worth considering.

Peter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exception handling in Python 3.x

[Paul Rubin]

Steven D'Aprano steve+comp.lang.pyt...@pearwood.info writes:
 def func(iterable):
 it = iter(iterable)
 failed = False
 try:
 x = next(it)
 except StopIteration:
 failed = True
 if failed:
 raise ValueError(can't process empty iterable)
 print(x)

Untested:

from itertools import islice

def func(iterable):
   xs = list(islice(iter(iterable), 1))
   if len(xs) == 0:
  raise ValueError(...)
   print xs[0]
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exception handling in Python 3.x

[Ethan Furman]

Peter Otten wrote:
  http://mail.python.org/pipermail/python-list/2010-October/1258606.html

http://mail.python.org/pipermail/python-list/2010-October/1259024.html


I found #6210 on bugs.python.org -- does anyone know if there are any 
others regarding this issue?  Or any progress on MRAB's idea?


MRAB wrote:
 Suggestion: an explicit 'raise' in the exception handler excludes the
 context, but if you want to include it then 'raise with'. For example:

 # Exclude the context
 try:
 command_dict[command]()
 except KeyError:
 raise CommandError(Unknown command)

 # Include the context
 try:
 command_dict[command]()
 except KeyError:
 raise with CommandError(Unknown command)

~Ethan~
--
http://mail.python.org/mailman/listinfo/python-list

Re: Exception handling in Python 3.x

[Ethan Furman]

Peter Otten wrote:

Hrvoje Niksic wrote:


Peter Otten __pete...@web.de writes:


Note that StopIteration is an internal detail of no relevance whatsoever
to the caller. Expose this is unnecessary at best and confusing at
worst.

http://mail.python.org/pipermail/python-list/2010-October/1258606.html
http://mail.python.org/pipermail/python-list/2010-October/1259024.html

Both of these involve suppressing the chaining at the wrong place,
namely in the outer handler or, worse yet, in the exception display
mechanism.  Steven, on the other hand, wants his *inner* handler to
express that the original exception was an implementation detail, a
business exception such as StopIteration, that is completely irrelevant
to the actual exception being raised.  The outer handler is the wrong
place to suppress the chaining because it has no way of distinguishing
Steven's case from a genuine case of a new exception unexpectedly
occurring during handling of the original exception.


To quote the Rolling Stones: You can't always get what you want. 

After rereading the original post I still don't get why the workarounds 
provided in those links aren't worth considering.


For me at least it's a matter of simplicity, clarity, and the Way of the 
Python  ;)


The workarounds are boiler-plate for a fairly common situation, and one 
of the things i _love_ about python is the *lack* of boilerplate.


I think the real question is is there any progress on dealing with the 
Open Issue in the PEP?


Open Issue: Suppressing Context
As written, this PEP makes it impossible to suppress '__context__',
since setting exc.__context__ to None in an 'except' or 'finally'
clause will only result in it being set again when exc is raised.

http://www.python.org/dev/peps/pep-3134/

~Ethan~
--
http://mail.python.org/mailman/listinfo/python-list

Re: Exception handling in Python 3.x

[Steven D'Aprano]

On Fri, 03 Dec 2010 10:15:58 -0800, Paul Rubin wrote:

 Steven D'Aprano steve+comp.lang.pyt...@pearwood.info writes:
 def func(iterable):
 it = iter(iterable)
 failed = False
 try:
 x = next(it)
 except StopIteration:
 failed = True
 if failed:
 raise ValueError(can't process empty iterable)
 print(x)
 
 Untested:
 
 from itertools import islice
 
 def func(iterable):
xs = list(islice(iter(iterable), 1))
if len(xs) == 0:
   raise ValueError(...)
print xs[0]


If you're intention was to make me feel better about the version above 
that sets a flag, you succeeded admirably!

:)


-- 
Steven



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exception handling in Python 3.x

[Steven D'Aprano]

On Fri, 03 Dec 2010 16:26:19 +0100, Hrvoje Niksic wrote:

 Peter Otten __pete...@web.de writes:
 
 Note that StopIteration is an internal detail of no relevance
 whatsoever to the caller. Expose this is unnecessary at best and
 confusing at worst.

 http://mail.python.org/pipermail/python-list/2010-October/1258606.html
 http://mail.python.org/pipermail/python-list/2010-October/1259024.html

Thanks for the links Peter.

 
 Both of these involve suppressing the chaining at the wrong place,
 namely in the outer handler or, worse yet, in the exception display
 mechanism.  Steven, on the other hand, wants his *inner* handler to
 express that the original exception was an implementation detail, a
 business exception such as StopIteration, that is completely irrelevant
 to the actual exception being raised.

Yes, exactly! Python 3.x exposes completely irrelevant and internal 
details in the traceback.


 The outer handler is the wrong
 place to suppress the chaining because it has no way of distinguishing
 Steven's case from a genuine case of a new exception unexpectedly
 occurring during handling of the original exception.
 
 One solution would be for raise inside except to not use the context.

I would have thought that was such an obvious solution that I was 
gobsmacked to discover the PEP 3134 hadn't already considered it. If you 
*explicitly* raise an exception inside an exception handler, surely it's 
because you want to suppress the previous exception as an internal detail?

If not, and you want to chain it with the previous exception, the 
solution is simple, obvious and straight-forward: explicit chaining.

try:
   something()
except SomeException as exc:
   raise MyException from exc




 For example:
 
 try:
   {}[1]
 except KeyError:
   1/0
 
 would behave as before, but:


Yes, that presumably would be a bug and should chain exceptions.


 But:
 
 try:
   {}[1]
 except KeyError:
   raise Exception(my error)
 
 ...would raise the custom error forgetting the KeyError.


That's exactly the behaviour I would expect and I'm surprised that this 
feature was put into production without some simple way to support this 
idiom.



-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exception handling in Python 3.x

[Steven D'Aprano]

On Fri, 03 Dec 2010 17:08:38 +0100, Peter Otten wrote:


 After rereading the original post I still don't get why the workarounds
 provided in those links aren't worth considering.


The first work-around:

http://mail.python.org/pipermail/python-list/2010-October/1258606.html

is unsuitable because it requires the caller to install a custom 
excepthook. It would be rude and unacceptable for arbitrary functions to 
install hooks, possibly stomping all over the caller's own custom 
excepthook. And even if I did, or the caller did, it has the unfortunate 
side-effect of suppressing the display of *all* chained exceptions, 
including those that come from the bugs in exception handlers.


The second work-around might be worth considering:

http://mail.python.org/pipermail/python-list/2010-October/1259024.html

however it adds unnecessary boilerplate to what should be a simple 
try...except...raise block, it obscures the intention of the code. As a 
work-around, it might be worth considering, but it's hardly elegant and 
it could very well be a fluke of the implementation rather than a 
guaranteed promise of the language.


In the absence of a supported way to suppress exception chaining, I'm 
leaning towards my original work-around: set a flag in the except block, 
then raise the exception once I leave the block.

But thanks again for the links.


-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exception Handling in Python 3

[John Ladasky]

On Oct 29, 8:53 am, rantingrick rantingr...@gmail.com wrote:

 I am the programmer, and when i say to my interpretor show this
 exception instead of that exception i expect my interpretor to do
 exactly as i say or risk total annihilation!! I don't want my
 interpreter interpreting my intentions and then doing what it
 thinks is best for me. I have a wife already, i don't need a virtual
 one!

OK, Ranting Rick, that was funny, and worthy of your name.  :^)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exception Handling in Python 3

[John Nagle]

On 10/24/2010 5:36 AM, Steve Holden wrote:

On 10/24/2010 2:22 AM, Lawrence D'Oliveiro wrote:

In messagemailman.176.1287896531.2218.python-l...@python.org, Steve
Holden wrote:



Yes, *if the exception is caught* then it doesn't make any difference.
If the exception creates a traceback, however, I maintain that the
additional information is confusing to the consumer (while helpful to
the debugger of the consumed code).


If an exception propagates all the way out to the top level and
all the user gets is a system traceback, the program isn't very
good.  Really.  If you're concerned about the display format of
Python tracebacks seen by end users, you have bigger problems
with your code.

If it's a server-side program, you need to catch
exceptions and log them somewhere, traceback and all.  If it's
a client-side GUI program, letting an exception unwind all the
way out of the program loses whatever the user was doing.

It's usually desirable to at least catch EnviromentError near
the top level of your program.  If some external problem causes a
program abort, the user should get an error message, not a traceback.
If it's a program bug, you can let the traceback unwind, displaying
information that should be sent in with a bug report.

John Nagle
--
http://mail.python.org/mailman/listinfo/python-list

Re: Exception Handling in Python 3

[Lawrence D'Oliveiro]

In message mailman.372.1288353590.2218.python-l...@python.org, Antoine 
Pitrou wrote:

 If you want to present exceptions to users in a different way ...

sys.stderr.write \
  (
Traceback (most recent call last):\n
...
AttributeError: blah blah blah ...\n
  )
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exception Handling in Python 3

[Gregory Ewing]

Steve Holden wrote:


Yeah, that's a given. Ruby would probably let you do that, but Python
insists that you don't dick around with the built-in types. And roghtly
so, IMHO.


Some restrictions on this are necessary -- it obviously
wouldn't be safe to allow replacing the class of an
object with one having an incompatible C layout, and
most built-in types have their own unique layout.

I think it's easier in Ruby, because all objects in
Ruby look pretty much the same at the C level. Not
sure of the details, though.

--
Greg
--
http://mail.python.org/mailman/listinfo/python-list

Re: Exception Handling in Python 3

[Gregory Ewing]

Chris Rebert wrote:

Your Traceback is merely being made slightly longer/more
complicated than you'd prefer; however, conversely, what if a bug was
to be introduced into your exception handler? Then you'd likely very
much appreciate the superfluous Traceback info.


I think what's disturbing about this is that the two halves of
the extended traceback are printed in the wrong order. We're
all used to looking down the bottom of the traceback to see
where the error originated, but with the new format, that point
is buried somewhere in the middle.

--
Greg
--
http://mail.python.org/mailman/listinfo/python-list

Re: Exception Handling in Python 3

[Chris Rebert]

On Fri, Oct 29, 2010 at 2:30 AM, Gregory Ewing
greg.ew...@canterbury.ac.nz wrote:
 Chris Rebert wrote:

 Your Traceback is merely being made slightly longer/more
 complicated than you'd prefer; however, conversely, what if a bug was
 to be introduced into your exception handler? Then you'd likely very
 much appreciate the superfluous Traceback info.

 I think what's disturbing about this is that the two halves of
 the extended traceback are printed in the wrong order. We're
 all used to looking down the bottom of the traceback to see
 where the error originated, but with the new format, that point
 is buried somewhere in the middle.

True, but swapping the order would only worsen Steve's problem. Most
of his users presumably won't care about the underlying KeyError and
would rather be presented with the AttributeError as the proximate
origin, despite that being technically inaccurate in the way you
suggest. Six of one, half dozen of the other though.

Cheers,
Chris
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exception Handling in Python 3

[rantingrick]

On Oct 24, 7:36 am, Steve Holden st...@holdenweb.com wrote:

 I don't want people to think this is a big deal, however.

Nonsense, this IS a big deal. (and Steve grow a spine already!) I was
not even aware of this issue until you brought it up -- although i
will admit your choice of title is completely misleading!

This new exception handling behavior is completely ridiculous and the
only thing more atrocious than the new exception handling BUG you
uncovered is the hoop jumping garbage people have come up with to
make the new work as the tried and tested old.

I am the programmer, and when i say to my interpretor show this
exception instead of that exception i expect my interpretor to do
exactly as i say or risk total annihilation!! I don't want my
interpreter interpreting my intentions and then doing what it
thinks is best for me. I have a wife already, i don't need a virtual
one! We are opening a Pandora's box of mediocrity and confusion when
we start down such a slippery slope as this.

Let me enlighten you fellows... If have i ever learned anything about
programming, i can tell you one rule should always be counted on above
all rules -- that the computer will do exactly what you tell it to do
NOTHING more, and NOTHING less. End of story!


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exception Handling in Python 3

[MRAB]

On 29/10/2010 11:24, Chris Rebert wrote:

On Fri, Oct 29, 2010 at 2:30 AM, Gregory Ewing
greg.ew...@canterbury.ac.nz  wrote:

Chris Rebert wrote:


Your Traceback is merely being made slightly longer/more
complicated than you'd prefer; however, conversely, what if a bug was
to be introduced into your exception handler? Then you'd likely very
much appreciate the superfluous Traceback info.


I think what's disturbing about this is that the two halves of
the extended traceback are printed in the wrong order. We're
all used to looking down the bottom of the traceback to see
where the error originated, but with the new format, that point
is buried somewhere in the middle.


True, but swapping the order would only worsen Steve's problem. Most
of his users presumably won't care about the underlying KeyError and
would rather be presented with the AttributeError as the proximate
origin, despite that being technically inaccurate in the way you
suggest. Six of one, half dozen of the other though.


I've just come across the same problem myself.

I wanted to raise an exception that would be more meaningful to the
caller, but the traceback included info on the original exception,
which is an implementation detail.

I understand that it can be useful, but IMHO there should be a simple
way of suppressing it.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Exception Handling in Python 3

[MRAB]

On 24/10/2010 13:28, Steve Holden wrote:

On 10/24/2010 4:48 AM, Martin v. Loewis wrote:

Am 24.10.2010 07:01, schrieb Steve Holden:

I was somewhat surprised to discover that Python 3 no longer allows an
exception to be raised in an except clause (or rather that it reports it
as a separate exception that occurred during the handling of the first).


I think you are misinterpreting what you are seeing. The exception being
raised actually *is* an attribute error, and it actually is the
attribute error that gets reported. It's only that reporting an
exception that has a __context__ first reports the context, then reports
the actual exception.


I don't believe I *am* misinterpreting it. The fact of the matter is
that the context is irrelevant to the user, and there should be some way
to suppress it to avoid over-complicating the traceback.

This behavior is quite reasonable during testing, but I would prefer to
exclude an explicit raise directly in the except handler since that is
hardly to be construed as accidental (whereas an exception in a function
called in the handler perhaps *should* be reported).


You may now wonder whether it is possible to set __context__ to None
somehow. See PEP 3134:

Open Issue: Suppressing Context

 As written, this PEP makes it impossible to suppress '__context__',
 since setting exc.__context__ to None in an 'except' or 'finally'
 clause will only result in it being set again when exc is raised.


I have already read that. Peter Otten has separately explained how to
suppress the behavior using sys.excepthook, which appears to be a
halfway satisfactory solution.

Suggestion: an explicit 'raise' in the exception handler excludes the 
context, but if you want to include it then 'raise with'. For example:


# Exclude the context
try:
command_dict[command]()
except KeyError:
raise CommandError(Unknown command)

# Include the context
try:
command_dict[command]()
except KeyError:
raise with CommandError(Unknown command)
--
http://mail.python.org/mailman/listinfo/python-list

Re: Exception Handling in Python 3

[Ethan Furman]

MRAB wrote:

On 24/10/2010 13:28, Steve Holden wrote:

On 10/24/2010 4:48 AM, Martin v. Loewis wrote:

Am 24.10.2010 07:01, schrieb Steve Holden:

I was somewhat surprised to discover that Python 3 no longer allows an
exception to be raised in an except clause (or rather that it 
reports it
as a separate exception that occurred during the handling of the 
first).


I think you are misinterpreting what you are seeing. The exception being
raised actually *is* an attribute error, and it actually is the
attribute error that gets reported. It's only that reporting an
exception that has a __context__ first reports the context, then reports
the actual exception.


I don't believe I *am* misinterpreting it. The fact of the matter is
that the context is irrelevant to the user, and there should be some way
to suppress it to avoid over-complicating the traceback.

This behavior is quite reasonable during testing, but I would prefer to
exclude an explicit raise directly in the except handler since that is
hardly to be construed as accidental (whereas an exception in a function
called in the handler perhaps *should* be reported).


You may now wonder whether it is possible to set __context__ to None
somehow. See PEP 3134:

Open Issue: Suppressing Context

 As written, this PEP makes it impossible to suppress '__context__',
 since setting exc.__context__ to None in an 'except' or 'finally'
 clause will only result in it being set again when exc is raised.


I have already read that. Peter Otten has separately explained how to
suppress the behavior using sys.excepthook, which appears to be a
halfway satisfactory solution.

Suggestion: an explicit 'raise' in the exception handler excludes the 
context, but if you want to include it then 'raise with'. For example:


# Exclude the context
try:
command_dict[command]()
except KeyError:
raise CommandError(Unknown command)

# Include the context
try:
command_dict[command]()
except KeyError:
raise with CommandError(Unknown command)


+1

Presumably, this would also keep the context if an actual error occured.

~Ethan~
--
http://mail.python.org/mailman/listinfo/python-list

Re: Exception Handling in Python 3

[Ethan Furman]

MRAB wrote:

On 29/10/2010 11:24, Chris Rebert wrote:

On Fri, Oct 29, 2010 at 2:30 AM, Gregory Ewing
greg.ew...@canterbury.ac.nz  wrote:

Chris Rebert wrote:


Your Traceback is merely being made slightly longer/more
complicated than you'd prefer; however, conversely, what if a bug was
to be introduced into your exception handler? Then you'd likely very
much appreciate the superfluous Traceback info.


I think what's disturbing about this is that the two halves of
the extended traceback are printed in the wrong order. We're
all used to looking down the bottom of the traceback to see
where the error originated, but with the new format, that point
is buried somewhere in the middle.


True, but swapping the order would only worsen Steve's problem. Most
of his users presumably won't care about the underlying KeyError and
would rather be presented with the AttributeError as the proximate
origin, despite that being technically inaccurate in the way you
suggest. Six of one, half dozen of the other though.


I've just come across the same problem myself.

I wanted to raise an exception that would be more meaningful to the
caller, but the traceback included info on the original exception,
which is an implementation detail.

I understand that it can be useful, but IMHO there should be a simple
way of suppressing it.


I agree.  It seems to me that the suppression should happen on the raise 
line, so that we aren't losing the extra information if an actual error 
occurs in the error handler.


~Ethan~
--
http://mail.python.org/mailman/listinfo/python-list

Re: Exception Handling in Python 3

[Greg Ewing]

Chris Rebert wrote:

On Fri, Oct 29, 2010 at 2:30 AM, Gregory Ewing
greg.ew...@canterbury.ac.nz wrote:



I think what's disturbing about this is that the two halves of
the extended traceback are printed in the wrong order. We're



True, but swapping the order would only worsen Steve's problem.


Yes, I can see that what Steve's problem requires is a way
of explicitly saying replace the current exception without
attaching any context.

However, in the case where the replacement is accidental,
I think it would make more sense to display them in the
opposite order. Both of the exceptions represent bugs in
that situation, so you will want to address them both, and
you might as well get the traceback in a non-confusing order.

--
Greg
--
http://mail.python.org/mailman/listinfo/python-list

Re: Exception Handling in Python 3

[Martin v. Loewis]

Am 24.10.2010 23:48, schrieb Steve Holden:
 On 10/24/2010 4:44 PM, John Nagle wrote:
 Are exception semantics changing in a way which would affect that?
 
 No, I don't believe so. I simply felt that the traceback gives too much
 information in the case where an exception is specifically being raised
 to replace the one currently being handled.

I think you have puzzled readers a lot (including me) with the statement:

that Python 3 no longer allows an exception to be raised in an except
clause

That certainly isn't the case.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exception Handling in Python 3

[Steve Holden]

On 10/25/2010 2:57 AM, Martin v. Loewis wrote:
 Am 24.10.2010 23:48, schrieb Steve Holden:
 On 10/24/2010 4:44 PM, John Nagle wrote:
 Are exception semantics changing in a way which would affect 
 that?
 
 No, I don't believe so. I simply felt that the traceback gives too 
 much information in the case where an exception is specifically 
 being raised to replace the one currently being handled.
 
 I think you have puzzled readers a lot (including me) with the 
 statement:
 
 that Python 3 no longer allows an exception to be raised in an 
 except clause
 
 That certainly isn't the case.
 
Of course it isn't. I believe the only readers puzzled  by my assertion
would be those who did not read the parenthesized comment immediately
following the sentence you quoted, which read:

 (or rather that it reports it as a separate exception that occurred
 during the handling of the first)

I understand that this behavior is deliberate. I just don't feel that it
is universally helpful.

regards
 Steve
-- 
Steve Holden   +1 571 484 6266   +1 800 494 3119
PyCon 2011 Atlanta March 9-17   http://us.pycon.org/
See Python Video!   http://python.mirocommunity.org/
Holden Web LLC http://www.holdenweb.com/

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exception Handling in Python 3

[Lawrence D'Oliveiro]

In message mailman.176.1287896531.2218.python-l...@python.org, Steve 
Holden wrote:

 I was somewhat surprised to discover that Python 3 no longer allows an
 exception to be raised in an except clause (or rather that it reports it
 as a separate exception that occurred during the handling of the first).

So what exactly is the problem? Exceptions are so easy to get wrong, it’s 
just trying to report more info in a complicated situation to help you track 
down the problem. Why is that bad?

 In a class's __getattr__() method this means that instead of being able
 to say
 
 try:
 value = _attrs[name]
 except KeyError:
 raise AttributeError ...
 
 I am forced to write
 
 if name not in _attrs:
 raise AttributeError ...
 value = _attrs[name]

I don’t see why. Presumably if you caught the exception in an outer try-
except clause, you would pick up your AttributeError, not the KeyError, 
right? Which is what you want, right?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exception Handling in Python 3

[Martin v. Loewis]

Am 24.10.2010 07:01, schrieb Steve Holden:
 I was somewhat surprised to discover that Python 3 no longer allows an
 exception to be raised in an except clause (or rather that it reports it
 as a separate exception that occurred during the handling of the first).

I think you are misinterpreting what you are seeing. The exception being
raised actually *is* an attribute error, and it actually is the
attribute error that gets reported. It's only that reporting an
exception that has a __context__ first reports the context, then reports
the actual exception.

You may now wonder whether it is possible to set __context__ to None
somehow. See PEP 3134:

Open Issue: Suppressing Context

As written, this PEP makes it impossible to suppress '__context__',
since setting exc.__context__ to None in an 'except' or 'finally'
clause will only result in it being set again when exc is raised.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exception Handling in Python 3

[Peter Otten]

Steve Holden wrote:

 On 10/24/2010 1:26 AM, Chris Rebert wrote:
 I was somewhat surprised to discover that Python 3 no longer allows an
  exception to be raised in an except clause (or rather that it reports
  it as a separate exception that occurred during the handling of the
  first).
 snip
 [snip]
  What
  is the correct paradigm for this situation?
 There doesn't seem to be one at the moment, although the issue isn't
 very serious. Your Traceback is merely being made slightly longer/more
 complicated than you'd prefer; however, conversely, what if a bug was
 to be introduced into your exception handler? Then you'd likely very
 much appreciate the superfluous Traceback info.
 
 Your quandary is due to the unresolved status of the Open Issue:
 Suppressing Context in PEP 3141
 (http://www.python.org/dev/peps/pep-3134/ ). I guess you could start a
 discussion about closing that issue somehow.
 
 You're right about the issue not being serious, (and about the possible
 solution, though I don't relish a lengthy discussion on python-dev) but
 it does seem that there ought to be some way to suppress that
 __context__. From the user's point of view the fact that I am raising
 AttributeError because of some implementation detail of __getattr__() is
 exposing *way* too much information.
 
 I even tried calling sys.exc_clear(), but alas that doesn't help :(

You can install a custom excepthook:

 import traceback, sys
 from functools import partial
 def f():
... try: 1/0
... except: raise AttributeError
...
 f()
Traceback (most recent call last):
  File stdin, line 2, in f
ZeroDivisionError: int division or modulo by zero

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File stdin, line 1, in module
  File stdin, line 3, in f
AttributeError
 sys.excepthook = partial(traceback.print_exception, chain=False)
 f()
Traceback (most recent call last):
  File stdin, line 1, in module
  File stdin, line 3, in f
AttributeError

Peter

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exception Handling in Python 3

[Steve Holden]

On 10/24/2010 4:48 AM, Martin v. Loewis wrote:
 Am 24.10.2010 07:01, schrieb Steve Holden:
 I was somewhat surprised to discover that Python 3 no longer allows an
 exception to be raised in an except clause (or rather that it reports it
 as a separate exception that occurred during the handling of the first).
 
 I think you are misinterpreting what you are seeing. The exception being
 raised actually *is* an attribute error, and it actually is the
 attribute error that gets reported. It's only that reporting an
 exception that has a __context__ first reports the context, then reports
 the actual exception.
 
I don't believe I *am* misinterpreting it. The fact of the matter is
that the context is irrelevant to the user, and there should be some way
to suppress it to avoid over-complicating the traceback.

This behavior is quite reasonable during testing, but I would prefer to
exclude an explicit raise directly in the except handler since that is
hardly to be construed as accidental (whereas an exception in a function
called in the handler perhaps *should* be reported).

 You may now wonder whether it is possible to set __context__ to None
 somehow. See PEP 3134:
 
 Open Issue: Suppressing Context
 
 As written, this PEP makes it impossible to suppress '__context__',
 since setting exc.__context__ to None in an 'except' or 'finally'
 clause will only result in it being set again when exc is raised.
 
I have already read that. Peter Otten has separately explained how to
suppress the behavior using sys.excepthook, which appears to be a
halfway satisfactory solution.

regards
 Steve
-- 
Steve Holden   +1 571 484 6266   +1 800 494 3119
PyCon 2011 Atlanta March 9-17   http://us.pycon.org/
See Python Video!   http://python.mirocommunity.org/
Holden Web LLC http://www.holdenweb.com/
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exception Handling in Python 3

[Steve Holden]

On 10/24/2010 2:22 AM, Lawrence D'Oliveiro wrote:
 In message mailman.176.1287896531.2218.python-l...@python.org, Steve 
 Holden wrote:
 
 I was somewhat surprised to discover that Python 3 no longer allows an
 exception to be raised in an except clause (or rather that it reports it
 as a separate exception that occurred during the handling of the first).
 
 So what exactly is the problem? Exceptions are so easy to get wrong, it’s 
 just trying to report more info in a complicated situation to help you track 
 down the problem. Why is that bad?
 
 In a class's __getattr__() method this means that instead of being able
 to say

 try:
 value = _attrs[name]
 except KeyError:
 raise AttributeError ...

 I am forced to write

 if name not in _attrs:
 raise AttributeError ...
 value = _attrs[name]
 
 I don’t see why. Presumably if you caught the exception in an outer try-
 except clause, you would pick up your AttributeError, not the KeyError, 
 right? Which is what you want, right?

Yes, *if the exception is caught* then it doesn't make any difference.
If the exception creates a traceback, however, I maintain that the
additional information is confusing to the consumer (while helpful to
the debugger of the consumed code).

I don't want people to think this is a big deal, however. It was just an
eh? that I thought must mean I was missing some way of suppressing the
additional traceback. Peter Otten has already provided a solution using
sys.except_hook().

regards
 Steve

-- 
Steve Holden   +1 571 484 6266   +1 800 494 3119
PyCon 2011 Atlanta March 9-17   http://us.pycon.org/
See Python Video!   http://python.mirocommunity.org/
Holden Web LLC http://www.holdenweb.com/

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exception Handling in Python 3

[Lie Ryan]

On 10/24/10 16:01, Steve Holden wrote:
 I was somewhat surprised to discover that Python 3 no longer allows an
 exception to be raised in an except clause (or rather that it reports it
 as a separate exception that occurred during the handling of the first).

FYI, Java has a similar behavior. In Java, however, after a certain
length, some of the older exceptions will be suppressed and will only
print message informing that there are more exceptions above it.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exception Handling in Python 3

[John Nagle]

On 10/23/2010 10:42 PM, Steve Holden wrote:

On 10/24/2010 1:26 AM, Chris Rebert wrote:

I was somewhat surprised to discover that Python 3 no longer
allows an

exception to be raised in an except clause (or rather that it
reports it as a separate exception that occurred during the
handling of the first).

snip

[snip]

What is the correct paradigm for this situation?

There doesn't seem to be one at the moment, although the issue
isn't very serious. Your Traceback is merely being made slightly
longer/more complicated than you'd prefer; however, conversely,
what if a bug was to be introduced into your exception handler?
Then you'd likely very much appreciate the superfluous Traceback
info.

Your quandary is due to the unresolved status of the Open Issue:
Suppressing Context in PEP 3141
(http://www.python.org/dev/peps/pep-3134/ ). I guess you could
start a discussion about closing that issue somehow.




   This is a traceback issue only, right?  The semantics of
the code below shouldn't change in Py3.x, I hope:

   try :
   ...
   try :
  x = 1/0 # in a real program, input data might cause a problem
   except ZeroDivisionError as msg:
  raise RuntimeError(Math error on problem:  + str(msg))
   except RuntimeError as msg :
   print(Trouble:  + str(msg))

I have code where I'm reading and parsing a web page,
a process which can produce a wide range of errors. A try-block
around the read and parse catches the various errors and creates a
single user-defined bad web page exception object, which is then
raised.   That gets caught further out, and is used to record the
troubled web page, schedule it for a retest, and such.  This is
normal program operation, indicative of external problems, not a
code error or cause for program termination with a traceback.

Are exception semantics changing in a way which would affect that?

John Nagle

--
http://mail.python.org/mailman/listinfo/python-list

Re: Exception Handling in Python 3

[Steve Holden]

On 10/24/2010 4:44 PM, John Nagle wrote:
 Are exception semantics changing in a way which would affect that?

No, I don't believe so. I simply felt that the traceback gives too much
information in the case where an exception is specifically being raised
to replace the one currently being handled.

regards
 Steve
-- 
Steve Holden   +1 571 484 6266   +1 800 494 3119
PyCon 2011 Atlanta March 9-17   http://us.pycon.org/
See Python Video!   http://python.mirocommunity.org/
Holden Web LLC http://www.holdenweb.com/

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exception Handling in Python 3

[Ben Finney]

Steve Holden st...@holdenweb.com writes:

 I simply felt that the traceback gives too much information in the
 case where an exception is specifically being raised to replace the
 one currently being handled.

Ideally, that description of the problem would suggest the obvious
solution: replace the class of the exception and allow the object to
continue up the exception handler stack.

But that doesn't work either::

 d = {}
 try:
... val = d['nosuch']
... except KeyError as exc:
... exc.__class__ = AttributeError
... raise exc
...
Traceback (most recent call last):
  File stdin, line 2, in module
KeyError: 'nosuch'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File stdin, line 4, in module
TypeError: __class__ assignment: only for heap types

which means, AFAICT, that re-binding ‘__class__’ is only allowed for
objects of a type defined in the Python run-time heap, not those defined
in C code (like the built-in-exception types).

-- 
 \ “I wish there was a knob on the TV to turn up the intelligence. |
  `\  There's a knob called ‘brightness’ but it doesn't work.” |
_o__) —Eugene P. Gallagher |
Ben Finney
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exception Handling in Python 3

[Lawrence D'Oliveiro]

In message mailman.190.1287924006.2218.python-l...@python.org, Steve 
Holden wrote:

 Yes, *if the exception is caught* then it doesn't make any difference.
 If the exception creates a traceback, however, I maintain that the
 additional information is confusing to the consumer (while helpful to
 the debugger of the consumed code).

Who needs the information more?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exception Handling in Python 3

[Steve Holden]

On 10/24/2010 7:51 PM, Ben Finney wrote:
 which means, AFAICT, that re-binding ‘__class__’ is only allowed for
 objects of a type defined in the Python run-time heap, not those defined
 in C code (like the built-in-exception types).

Yeah, that's a given. Ruby would probably let you do that, but Python
insists that you don't dick around with the built-in types. And roghtly
so, IMHO.

regards
 Steve
-- 
Steve Holden   +1 571 484 6266   +1 800 494 3119
PyCon 2011 Atlanta March 9-17   http://us.pycon.org/
See Python Video!   http://python.mirocommunity.org/
Holden Web LLC http://www.holdenweb.com/

-- 
http://mail.python.org/mailman/listinfo/python-list

Exception Handling in Python 3

[Steve Holden]

I was somewhat surprised to discover that Python 3 no longer allows an
exception to be raised in an except clause (or rather that it reports it
as a separate exception that occurred during the handling of the first).

So the following code:

 d = {}
 try:
... val = d['nosuch']
... except:
... raise AttributeError(No attribute 'nosuch')
...

Give the traceback I expected and wanted in Python 2:

Traceback (most recent call last):
  File stdin, line 4, in module
AttributeError: No attribute 'nosuch'

but in Python 3.1 the traceback looks like this:

Traceback (most recent call last):
  File stdin, line 2, in module
KeyError: 'nosuch'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File stdin, line 4, in module
AttributeError: No attribute 'nosuch'

Modifying the code a little allows me to change the error message, but
not much else:

 d = {}
 try:
... val = d['nosuch']
... except KeyError as e:
... raise AttributeError(No attribute 'nosuch') from e
...
Traceback (most recent call last):
  File stdin, line 2, in module
KeyError: 'nosuch'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File stdin, line 4, in module
AttributeError: No attribute 'nosuch'


In a class's __getattr__() method this means that instead of being able
to say

try:
value = _attrs[name]
except KeyError:
raise AttributeError ...

I am forced to write

if name not in _attrs:
raise AttributeError ...
value = _attrs[name]

which requires an unnecessary second lookup on the attribute name. What
is the correct paradigm for this situation?

regards
 Steve
-- 
Steve Holden   +1 571 484 6266   +1 800 494 3119
PyCon 2011 Atlanta March 9-17   http://us.pycon.org/
See Python Video!   http://python.mirocommunity.org/
Holden Web LLC http://www.holdenweb.com/

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exception Handling in Python 3

[Chris Rebert]

On Sat, Oct 23, 2010 at 10:01 PM, Steve Holden st...@holdenweb.com wrote:
 I was somewhat surprised to discover that Python 3 no longer allows an
 exception to be raised in an except clause (or rather that it reports it
 as a separate exception that occurred during the handling of the first).
snip
 Give the traceback I expected and wanted in Python 2:

 Traceback (most recent call last):
  File stdin, line 4, in module
 AttributeError: No attribute 'nosuch'

 but in Python 3.1 the traceback looks like this:

 Traceback (most recent call last):
  File stdin, line 2, in module
 KeyError: 'nosuch'

 During handling of the above exception, another exception occurred:

 Traceback (most recent call last):
  File stdin, line 4, in module
 AttributeError: No attribute 'nosuch'

 Modifying the code a little allows me to change the error message, but
 not much else:
snip
 What
 is the correct paradigm for this situation?

There doesn't seem to be one at the moment, although the issue isn't
very serious. Your Traceback is merely being made slightly longer/more
complicated than you'd prefer; however, conversely, what if a bug was
to be introduced into your exception handler? Then you'd likely very
much appreciate the superfluous Traceback info.

Your quandary is due to the unresolved status of the Open Issue:
Suppressing Context in PEP 3141
(http://www.python.org/dev/peps/pep-3134/ ). I guess you could start a
discussion about closing that issue somehow.

Cheers,
Chris
--
http://blog.rebertia.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exception Handling in Python 3

[Steve Holden]

On 10/24/2010 1:26 AM, Chris Rebert wrote:
 I was somewhat surprised to discover that Python 3 no longer allows an
  exception to be raised in an except clause (or rather that it reports it
  as a separate exception that occurred during the handling of the first).
 snip
[snip]
  What
  is the correct paradigm for this situation?
 There doesn't seem to be one at the moment, although the issue isn't
 very serious. Your Traceback is merely being made slightly longer/more
 complicated than you'd prefer; however, conversely, what if a bug was
 to be introduced into your exception handler? Then you'd likely very
 much appreciate the superfluous Traceback info.
 
 Your quandary is due to the unresolved status of the Open Issue:
 Suppressing Context in PEP 3141
 (http://www.python.org/dev/peps/pep-3134/ ). I guess you could start a
 discussion about closing that issue somehow.

You're right about the issue not being serious, (and about the possible
solution, though I don't relish a lengthy discussion on python-dev) but
it does seem that there ought to be some way to suppress that
__context__. From the user's point of view the fact that I am raising
AttributeError because of some implementation detail of __getattr__() is
exposing *way* too much information.

I even tried calling sys.exc_clear(), but alas that doesn't help :(

regards
 Steve
-- 
Steve Holden   +1 571 484 6266   +1 800 494 3119
PyCon 2011 Atlanta March 9-17   http://us.pycon.org/
See Python Video!   http://python.mirocommunity.org/
Holden Web LLC http://www.holdenweb.com/

-- 
http://mail.python.org/mailman/listinfo/python-list