Re: Error handling in context managers
On Jan 16, 2017, at 11:34 PM, Peter Otten <__pete...@web.de> wrote:
>
> Gregory Ewing wrote:
>
>> Israel Brewster wrote:
>>> The problem is that, from time to time, I can't get a connection, the
>>> result being that cursor is None,
>>
>> That's your problem right there -- you want a better-behaved
>> version of psql_cursor().
>>
>> def get_psql_cursor():
>>c = psql_cursor()
>>if c is None:
>> raise CantGetAConnectionError()
>>return c
>>
>> with get_psql_cursor() as c:
>>...
>
> You still need to catch the error -- which leads to option (3) in my zoo,
> the only one that is actually usable. If one contextmanager cannot achieve
> what you want, use two:
>
> $ cat conditional_context_raise.py
> import sys
> from contextlib import contextmanager
>
> class NoConnection(Exception):
>pass
>
> class Cursor:
>def execute(self, sql):
>print("EXECUTING", sql)
>
> @contextmanager
> def cursor():
>if "--fail" in sys.argv:
>raise NoConnection
>yield Cursor()
>
> @contextmanager
> def no_connection():
>try:
>yield
>except NoConnection:
>print("no connection")
>
> with no_connection(), cursor() as cs:
>cs.execute("insert into...")
> $ python3 conditional_context_raise.py
> EXECUTING insert into...
> $ python3 conditional_context_raise.py --fail
> no connection
>
> If you want to ignore the no-connection case use
> contextlib.suppress(NoConnection) instead of the custom no_connection()
> manager.
Fun :-) I'll have to play around with that. Thanks! :-)
---
Israel Brewster
Systems Analyst II
Ravn Alaska
5245 Airport Industrial Rd
Fairbanks, AK 99709
(907) 450-7293
---
>
> --
> https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
Re: Error handling in context managers
On Jan 16, 2017, at 8:01 PM, Gregory Ewing wrote: > > Israel Brewster wrote: >> The problem is that, from time to time, I can't get a connection, the result >> being that cursor is None, > > That's your problem right there -- you want a better-behaved > version of psql_cursor(). > > def get_psql_cursor(): > c = psql_cursor() > if c is None: > raise CantGetAConnectionError() > return c > > with get_psql_cursor() as c: > ... > Ok, fair enough. So I get a better exception, raised at the proper time. This is, in fact, better - but doesn't actually change how I would *handle* the exception :-) --- Israel Brewster Systems Analyst II Ravn Alaska 5245 Airport Industrial Rd Fairbanks, AK 99709 (907) 450-7293 --- > -- > Greg > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Error handling in context managers
On Jan 16, 2017, at 1:27 PM, Terry Reedy wrote:
>
> On 1/16/2017 1:06 PM, Israel Brewster wrote:
>> I generally use context managers for my SQL database connections, so I can
>> just write code like:
>>
>> with psql_cursor() as cursor:
>>
>>
>> And the context manager takes care of making a connection (or getting a
>> connection from a pool, more likely), and cleaning up after the fact (such
>> as putting the connection back in the pool), even if something goes wrong.
>> Simple, elegant, and works well.
>>
>> The problem is that, from time to time, I can't get a connection, the result
>> being that cursor is None,
>
> This would be like open('bad file') returning None instead of raising
> FileNotFoundError.
>
>> and attempting to use it results in an AttributeError.
>
> Just as None.read would.
>
> Actually, I have to wonder about your claim. The with statement would look
> for cursor.__enter__ and then cursor.__exit__, and None does not have those
> methods. In other words, the expression following 'with' must evaluate to a
> context manager and None is not a context manager.
>
> >>> with None: pass
>
> Traceback (most recent call last):
> File "", line 1, in
>with None: pass
> AttributeError: __enter__
>
> Is psql_cursor() returning a fake None object with __enter__ and __exit__
> methods?
No, the *context manager*, which I call in the with *does* have __enter__ and
__exit__ methods. It's just that the __enter__ method returns None when it
can't get a connection. So the expression following with *does* evaluate to a
context manager, but the expression following as evaluates to None.
---
Israel Brewster
Systems Analyst II
Ravn Alaska
5245 Airport Industrial Rd
Fairbanks, AK 99709
(907) 450-7293
---
>
> --
> Terry Jan Reedy
>
> --
> https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
Re: Error handling in context managers
Gregory Ewing wrote:
> Israel Brewster wrote:
>> The problem is that, from time to time, I can't get a connection, the
>> result being that cursor is None,
>
> That's your problem right there -- you want a better-behaved
> version of psql_cursor().
>
> def get_psql_cursor():
> c = psql_cursor()
> if c is None:
>raise CantGetAConnectionError()
> return c
>
> with get_psql_cursor() as c:
> ...
You still need to catch the error -- which leads to option (3) in my zoo,
the only one that is actually usable. If one contextmanager cannot achieve
what you want, use two:
$ cat conditional_context_raise.py
import sys
from contextlib import contextmanager
class NoConnection(Exception):
pass
class Cursor:
def execute(self, sql):
print("EXECUTING", sql)
@contextmanager
def cursor():
if "--fail" in sys.argv:
raise NoConnection
yield Cursor()
@contextmanager
def no_connection():
try:
yield
except NoConnection:
print("no connection")
with no_connection(), cursor() as cs:
cs.execute("insert into...")
$ python3 conditional_context_raise.py
EXECUTING insert into...
$ python3 conditional_context_raise.py --fail
no connection
If you want to ignore the no-connection case use
contextlib.suppress(NoConnection) instead of the custom no_connection()
manager.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Error handling in context managers
Terry Reedy wrote: Traceback (most recent call last): File "", line 1, in with None: pass AttributeError: __enter__ Like he said, you get an AttributeError! -- Greg -- https://mail.python.org/mailman/listinfo/python-list
Re: Error handling in context managers
Israel Brewster wrote: The problem is that, from time to time, I can't get a connection, the result being that cursor is None, That's your problem right there -- you want a better-behaved version of psql_cursor(). def get_psql_cursor(): c = psql_cursor() if c is None: raise CantGetAConnectionError() return c with get_psql_cursor() as c: ... -- Greg -- https://mail.python.org/mailman/listinfo/python-list
Re: Error handling in context managers
On 1/16/2017 1:06 PM, Israel Brewster wrote:
I generally use context managers for my SQL database connections, so I can just
write code like:
with psql_cursor() as cursor:
And the context manager takes care of making a connection (or getting a
connection from a pool, more likely), and cleaning up after the fact (such as
putting the connection back in the pool), even if something goes wrong. Simple,
elegant, and works well.
The problem is that, from time to time, I can't get a connection, the result
being that cursor is None,
This would be like open('bad file') returning None instead of raising
FileNotFoundError.
and attempting to use it results in an AttributeError.
Just as None.read would.
Actually, I have to wonder about your claim. The with statement would
look for cursor.__enter__ and then cursor.__exit__, and None does not
have those methods. In other words, the expression following 'with'
must evaluate to a context manager and None is not a context manager.
>>> with None: pass
Traceback (most recent call last):
File "", line 1, in
with None: pass
AttributeError: __enter__
Is psql_cursor() returning a fake None object with __enter__ and
__exit__ methods?
--
Terry Jan Reedy
--
https://mail.python.org/mailman/listinfo/python-list
Re: Error handling in context managers
Steve D'Aprano : > or you can let the context manager do what it does, and write your own code > to do what you do: > > try: > with ...: >... > except: > ... Even better: try: a = open(...) except ...: ... with a: ... You want to catch exceptions immediately after (potentially) raising them. Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: Error handling in context managers
On Tue, 17 Jan 2017 05:06 am, Israel Brewster wrote:
> I generally use context managers for my SQL database connections, so I can
> just write code like:
>
> with psql_cursor() as cursor:
>
>
> And the context manager takes care of making a connection (or getting a
> connection from a pool, more likely), and cleaning up after the fact (such
> as putting the connection back in the pool), even if something goes wrong.
> Simple, elegant, and works well.
>
> The problem is that, from time to time, I can't get a connection, the
> result being that cursor is None,
Seriously? psql_cursor().__enter__ returns None instead of failing? That
sounds like a poor design to me.
Where is this psql_cursor coming from?
> and attempting to use it results in an
> AttributeError. So my instinctive reaction is to wrap the potentially
> offending code in a try block, such that if I get that AttributeError I
> can decide how I want to handle the "no connection" case. This, of course,
> results in code like:
>
> try:
> with psql_cursor() as cursor:
>
> except AttributeError as e:
>
Except that isn't necessarily the no-connection case. It could be *any*
AttributeError anywhere in the entire with block.
> I could also wrap the code within the context manager in an if block
> checking for if cursor is not None, but while perhaps a bit clearer as to
> the purpose, now I've got an extra check that will not be needed most of
> the time (albeit a quite inexpensive check).
It's cheap, it's only needed once (at the start of the block), it isn't
subject to capturing the wrong exception... I would definitely write:
with psql_cursor() as cursor:
if cursor is not None:
> The difficulty I have with either of these solutions, however, is that
> they feel ugly to me - and wrapping the context manager in a try block
> almost seems to defeat the purpose of the context manager in the first
> place - If I'm going to be catching errors anyway, why not just do the
> cleanup there rather than hiding it in the context manager?
Context managers don't necessarily swallow exceptions (although they can).
That's not what they're for. Context managers are intended to avoid:
try:
...
finally:
...
*not* try...except blocks. If you need a try...except, then you could avoid
using the context manager and re-invent the wheel:
try:
...
except:
...
finally:
# do whatever cleanup the context manager already defines
# but now you have to do it yourself
or you can let the context manager do what it does, and write your own code
to do what you do:
try:
with ...:
...
except:
...
[...]
> says "there should be a better way", so I figured I'd ask: *is* there a
> better way? Perhaps some way I could handle the error internally to the
> context manager, such that it just dumps me back out?
That's what's supposed to happen:
py> with open('foobarbaz') as f:
... pass
...
Traceback (most recent call last):
File "", line 1, in
FileNotFoundError: [Errno 2] No such file or directory: 'foobarbaz'
Notice that the context manager simply raises an exception on failure, which
I can catch or not as I so choose, rather than returning None.
I really think that the problem here is the design of psql_cursor().
> Of course, that
> might not work, given that I may need to do something different *after*
> the context manager, depending on if I was able to get a connection, but
> it's a thought. Options?
--
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Error handling in context managers
Israel Brewster wrote:
> I generally use context managers for my SQL database connections, so I can
> just write code like:
>
> with psql_cursor() as cursor:
>
>
> And the context manager takes care of making a connection (or getting a
> connection from a pool, more likely), and cleaning up after the fact (such
> as putting the connection back in the pool), even if something goes wrong.
> Simple, elegant, and works well.
>
> The problem is that, from time to time, I can't get a connection, the
> result being that cursor is None, and attempting to use it results in an
> AttributeError. So my instinctive reaction is to wrap the potentially
> offending code in a try block, such that if I get that AttributeError I
> can decide how I want to handle the "no connection" case. This, of course,
> results in code like:
>
> try:
> with psql_cursor() as cursor:
>
> except AttributeError as e:
>
>
> I could also wrap the code within the context manager in an if block
> checking for if cursor is not None, but while perhaps a bit clearer as to
> the purpose, now I've got an extra check that will not be needed most of
> the time (albeit a quite inexpensive check).
That seems to be the cleanest approach.
> The difficulty I have with either of these solutions, however, is that
> they feel ugly to me - and wrapping the context manager in a try block
> almost seems to defeat the purpose of the context manager in the first
> place - If I'm going to be catching errors anyway, why not just do the
> cleanup there rather than hiding it in the context manager?
>
> Now don't get me wrong - neither of these issues is terribly significant
> to me. I'll happily wrap all the context manager calls in a try block and
> move on with life if that it in fact the best option. It's just my gut
> says "there should be a better way", so I figured I'd ask: *is* there a
> better way? Perhaps some way I could handle the error internally to the
> context manager, such that it just dumps me back out? Of course, that
> might not work, given that I may need to do something different *after*
> the context manager, depending on if I was able to get a connection, but
> it's a thought. Options?
The problem is that there is no way to skip the with-block. It would be nice
if you could raise a special exception in the __enter__() method that would
achieve that.
For the time being you could
(1) abuse a for loop
def psql_cursor():
try:
yield make_cursor()
except NoConnection:
pass
for cursor in psql_cursor():
...
or you could
(2) return something that hopefully raises an exception soon, e. g.
$ cat conditional_context.py
import sys
from contextlib import contextmanager
class Cursor:
def execute(self, sql):
print("EXECUTING", sql)
class Exit(Exception):
pass
class FailingCursor:
def __getattr__(self, name):
raise Exit
@contextmanager
def cursor():
try:
yield FailingCursor() if "--fail" in sys.argv else Cursor()
except Exit:
print("no connection")
with cursor() as cs:
cs.execute("insert into...")
$ python3 conditional_context.py
EXECUTING insert into...
$ python3 conditional_context.py --fail
no connection
Option (1) works, but looks wrong while option (2) looks better, but does
not work reliably in the general case, e. g. when you perform some costly
calculation before the first attribute access.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Error handling in context managers
On Tue, Jan 17, 2017 at 5:06 AM, Israel Brewster wrote: > I generally use context managers for my SQL database connections, so I can > just write code like: > > with psql_cursor() as cursor: > > > And the context manager takes care of making a connection (or getting a > connection from a pool, more likely), and cleaning up after the fact (such as > putting the connection back in the pool), even if something goes wrong. > Simple, elegant, and works well. > > The problem is that, from time to time, I can't get a connection, the result > being that cursor is None, and attempting to use it results in an > AttributeError. My question is: If you can't get a connection, why do you get back None? Wouldn't it be better for psql_cursor() to raise an exception? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Error handling in context managers
I generally use context managers for my SQL database connections, so I can just write code like: with psql_cursor() as cursor: And the context manager takes care of making a connection (or getting a connection from a pool, more likely), and cleaning up after the fact (such as putting the connection back in the pool), even if something goes wrong. Simple, elegant, and works well. The problem is that, from time to time, I can't get a connection, the result being that cursor is None, and attempting to use it results in an AttributeError. So my instinctive reaction is to wrap the potentially offending code in a try block, such that if I get that AttributeError I can decide how I want to handle the "no connection" case. This, of course, results in code like: try: with psql_cursor() as cursor: except AttributeError as e: I could also wrap the code within the context manager in an if block checking for if cursor is not None, but while perhaps a bit clearer as to the purpose, now I've got an extra check that will not be needed most of the time (albeit a quite inexpensive check). The difficulty I have with either of these solutions, however, is that they feel ugly to me - and wrapping the context manager in a try block almost seems to defeat the purpose of the context manager in the first place - If I'm going to be catching errors anyway, why not just do the cleanup there rather than hiding it in the context manager? Now don't get me wrong - neither of these issues is terribly significant to me. I'll happily wrap all the context manager calls in a try block and move on with life if that it in fact the best option. It's just my gut says "there should be a better way", so I figured I'd ask: *is* there a better way? Perhaps some way I could handle the error internally to the context manager, such that it just dumps me back out? Of course, that might not work, given that I may need to do something different *after* the context manager, depending on if I was able to get a connection, but it's a thought. Options? --- Israel Brewster Systems Analyst II Ravn Alaska 5245 Airport Industrial Rd Fairbanks, AK 99709 (907) 450-7293 --- -- https://mail.python.org/mailman/listinfo/python-list
