On 2020-04-11 5:40 a.m., Stephen J. Turnbull wrote:
Soni L. writes:
> so, for starters, here's everything I'm worried about.
>
> in one of my libraries (yes this is real code. all of this is taken from
> stuff I'm deploying.) I have the following piece of code:
>
> def _extract(self, obj):
> try:
> yield (self.key, obj[self.key])
> except (TypeError, IndexError, KeyError):
> if not self.skippable:
> raise exceptions.ValidationError
> due to the wide range of supported objects, I can't expect the TypeError
> to always come from my attempt to index into a set, or the IndexError to
> always come from my attempt to index into a sequence, or the KeyError to
> always come from my attempt to index into a mapping. those could very
> well be coming from a bug in someone's weird sequence/mapping/set
> implementation. I have no way of knowing!
I hope that exclamation point doesn't mean you think anybody doesn't
understand the problem by now. I think most serious programmers have
run into "expected" Exceptions masking unexpected errors. What we
don't understand is (a) how your proposed fix is supposed to work
without a tremendous effort by third parties (and Python itself in the
stdlib), (b) if you do presume enormous amounts of effort, what is in
it for the third parties, and (c) why doesn't asking the third parties
to create finely graduated hierarchies of exceptions do as well (if
not better) than the syntax change?
The grungy details:
So as far as I can tell, your analysis above is exactly right. You
have no way of knowing, and with the syntax change *you* still will
have no way of knowing. You will be dependent on the implementer to
tell you, ie, the obj's getter must distinguish between an actual
"this key is not in the object" Error, and an implementation bug.
You're asking everybody else to consistently distinguish between
KeyErrors you think are KeyErrors and KeyErrors you think are actually
bugs. I don't see how they can know what you think, and of course if
you're right that they are actually bugs, by that very token the
relevant developer probably doesn't know they are there!
And I don't think they'll care enough to make the effort even if they
are aware of the difference. Because, if they did care, they already
can make the effort by using a wider variety of builtin exceptions, or
by adding attributes to the exceptions they raise, or deriving
subclasses of the various exceptions that might be raised, or of
RuntimeError (and they might disagree on what exception *should* be
the base exception!) But they don't do any of those very often. And
I've seen principled objections to the proliferation of specialized
exceptions (sorry, no cite; ignore if you like). I don't recall them
clearly but I suppose some of them would apply to this syntax.
> "exception spaces" would enable me to say "I want your
> (operator/function/whatnot) to raise some errors in my space, so I
> don't confuse them with bugs in your space instead". and they'd get
> me exactly that.
But how do you say that to me? I'm not your subcontractor, I'm an
independent developer. I write a library, I *don't know* what "your"
exception spaces are. It's the other way around, I guess? That is, I
raise exceptions in *my* space and some in builtin space and maybe even
some in imported spaces and you have to catch them explicitly to
distinguish. I think you're going to have to do a lot of studying and
a lot of extra work on exception handling to catch only the exceptions
you intend to. I'm not sure it's going to be worth it to do the
necessary research to be able to write
# these are KeyErrors you deem to be the real McCoy
except KeyError in SpaceX | SpaceY | SpaceZ: handle_spacey_errors()
# these are exceptions from spaces that you think are probably bugs
except KeyError: raise OutDamnSpotError
whenever you're handling potential exceptions from third-party code.
And I think you have to do it consistently, because otherwise you may
be missing lots of masked bugs. If it's not worth it for *you* to use
it *consistently*, why would the tremendous effort imposed on others to
lay the foundations for your "occasional" use be worth it?
One of the main points of exceptions as a mechanism is that they
"bubble up" until caught. If they occurred deep in the import
hierarchy, you may consider that a bug if middle layers don't
distinguish, but the developer-in-the-middle may disagree with you if
you're handing their library inputs they never intended to handle. I
guarantee you lose that argument, unless you are *very* nice to
Ms. DITM.
Bottom line: I just don't see how this can work in practice unless we
make the "in" clause mandatory, which I suspect is a Python 4.0
compatibility break of Python 3.0 annoyance. I know it would annoy
me: my exception handling is almost all of the "terminate processing,
discard pending input as possibly corrupt, reinitialize, restart
processing" variety. I just don't care about "masked bugs".
for starters: operator variants wherever exceptions are used as part of
API contract, and add a kwarg to functions that explicitly define
exceptions as part of their API contract.
if missing, defaults to None. all exceptions not raised in a specific
channel will not match a specific channel, but will match None.
so e.g. you do:
def foo(espace=None):
if thing():
raise Bar in espace
which means if the API user isn't expecting to have to handle an error,
they can just do:
x = foo()
and then they won't handle it. but if they want to handle it, they can do:
try:
x = foo(espace=my_espace)
except Bar in my_espace:
...
and then they'll handle it. alternatively, they can let their caller
handle it by having the espace come from the caller.
def bar(espace=None):
x = foo(espace=espace)
and then the caller is expected to handle it, *if they so choose*.
by default the None handler works even for exceptions in a channel:
try:
x = foo(espace=NotNone)
except Bar:
...
this'll also catch it. this is for backwards compatibility so existing
still code catches exceptions without caring where they come from, and
also so you can still have a "catches everything" construct.
we'll also need operator variants. I was thinking of the following:
x.in foo bar # for attribute access
x[in foo bar] # for item access
x +in foo bar # for math
# etc
(yes, these are very ugly. this is one of the unfortunate things of
trying to retrofit this into an existing language. but they're more
likely to catch bugs so I'm not worried.)
which would call new methods that take in an espace. (we could retrofit
existing methods with an optional espace but then we lose the ability to
catch everything and shove it in the appropriate espace anyway. this
would significantly reduce the effort required to make the stdlib work
with this new thing.)
maybe this should be python 4. I don't think any language's ever done
Rust-like explicit error handling (i.e. avoiding the issue of swallowing
bugs) using exceptions before. it could even be sugar for "except Foo:
if Foo.__channel__ != channel: raise" or something, I guess. but... it's
definitely worth it. and I don't think it'll be a python 3-level
annoyance, if implemented like this.
.
> it's also backwards compatible.
I don't think so, if it's to be useful to you or anyone else. See
discussion preceding "unless we make it mandatory," above.
> anyway, I'm gonna keep pushing for this because it's probably the
> easiest way to retrofix explicit error handling into python,
You're not going to get that, I'm pretty sure, except to the extent
that you already have it in the form of Exception subclasses. Python
is not intended to make defects hard to inject the way Rust is.
(Those are observations, not policy statements. I don't have much if
any influence over policy. ;-) I just know a lot about incentives in
my professional capacity.)
Steve
_______________________________________________
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/KBZLE4CJSRBJQVNUKL4JQYWB4KHPCXUM/
Code of Conduct: http://python.org/psf/codeofconduct/
