Re: [Python-Dev] Simpler finalization semantics (was Re: PEP 343 - Abstract Block Redux)
Michael Chermside wrote: If I write with foo: BLOCK where I should have written with locked(foo): BLOCK ...it silently succeeds by doing nothing. I CLEARLY intended to do the appropriate cleanup (or locking, or whatever), but it doesn't happen. Ah, thanks. Like Guido, I had something in the back of my head saying it didn't like the idea, but I couldn't figure out the reason. I think you just nailed it. Plus, there is a nice alternative which is to provide a 'clean it up if it needs it' resource in the standard library: class resource(object): def __init__(self, obj): self.obj = obj self.enter = getattr(obj, __enter__, None) self.exit = getattr(obj, __exit__, None) def __enter__(self): if self.enter is not None: self.enter() # For consistency, always return the object return self.obj def __exit__(self, *exc_info): if self.exit is not None: self.exit(*exc_info) Then 'I don't know if this needs cleaning up or not' can be written: with resource(foo): # If foo needs cleaning up, it will be. A refinement would provide the option to specify the enter/exit methods directly: class resource(object): def __init__(self, obj, *other_args): self.obj = obj if other_args: if len(other_args) != 2: raise TypeError(need 1 or 3 arguments) self.enter = args[0] self.exit = None self.exit_no_args = args[1] else: self.enter = getattr(obj, __enter__, None) self.exit = getattr(obj, __exit__, None) self.exit_no_args = None def __enter__(self): if self.enter is not None: self.enter() # For consistency, always return the object return self.obj def __exit__(self, *exc_info): if self.exit is not None: self.exit(*exc_info) elif self.exit_no_args is not None: self.exit() That would let any object with a standard 'clean me up method' be easily used in a with statement: with resource(bar, None, bar.clear): # bar will be cleared when we're done Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.blogspot.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Simpler finalization semantics (was Re: PEP 343 -Abstract Block Redux)
[I apologize in advance if this sounds a bit disjointed... I started to argue one thing, but by the end had convinced myself of the opposite, and I re-wrote the email to match my final conclusion.] Guido writes: About deleting VAR I have mixed feelings. [...] I think that, given that we let the for-loop variable survive, we should treat the with-statement variable the same way. We said the same thing about the variable in list comprehensions and it's now obvious that it should NEVER have been allowed to escape it's scope. But the key difference is that list comprehensions are EXPRESSIONS, and for and 'with' are STATEMENTS. Expressions shouldn't modify the local environment, statements often do. Of course, that argument _permits_ not deleting VAR, but doesn't recomend in favor of it. My first thought for ideal behavior was that if VAR was previously defined (eg: a global, an existing attribute of some object, etc), then it should not be 'del''ed afterward. But VAR was newly created by the 'with' statement then we WOULD del it to keep the namespace neat. Trouble is, that's FAR too complex, and depends on a distinction Python has not used before (although it's nearly the same as the property that controls the meaning of globally declared variables). My next thought was to just allow 'with' statements to introduce their own scope... the meaning that the VAR variable takes on within a 'with' statement is not propogated outside the scope of the statement. But imagine trying to implement this in CPython... don't forget details like supporting locals(). If it's too hard to do, then it's probably not the right solution. So then I thought Well, what's the harm in letting the variable survive the 'with' statement? I'm a big fan of keeping namespaces clean, but it's just not important enough to incurr other penalties. So in this case, I (reluctantly, after giving myself quite a talking-to) favor having the 'with' statement with VAR create said variable in the appropriate scope as a side-effect, much like 'for'. -- Michael Chermside ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Simpler finalization semantics (was Re: PEP 343 - Abstract Block Redux)
Guido writes: [a rather silly objection to Phillip's proposal that 'with x:' is a no-op when x lacks __enter__ and __exit__] I know this is not a very strong argument, but my gut tells me this generalization of the with-statement is wrong, so I'll stick to it regardless of the strength of the argument. The real reason will come to me. Perhaps the real reason is that it allows errors to pass silently. If I write with foo: BLOCK where I should have written with locked(foo): BLOCK ...it silently succeeds by doing nothing. I CLEARLY intended to do the appropriate cleanup (or locking, or whatever), but it doesn't happen. -- Michael Chermside ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Simpler finalization semantics (was Re: PEP 343 - Abstract Block Redux)
Phillip J. Eby wrote: I'm suggesting that we simply take Nick's proposal to its logical conclusion, and allow any object to be usable under with, since it does not create any problems to do so. (At least, none that I can see.) A redundant 'with' does no harm; in the worst case it's just a hint to the reader about the scope within which an expression is used within the current fuction body. Do you mean translating this: with EXPR1 as VAR1: BLOCK1 To something along the lines of: the_stmt = EXPR1 stmt_enter = getattr(the_stmt, __enter__, None) stmt_exit = getattr(the_stmt, __exit__, None) if stmt_enter is None: VAR1 = the_stmt else: VAR1 = stmt_enter() if stmt_exit is None: BLOCK1 else: exc = (None, None, None) try: try: BLOCK1 except: exc = sys.exc_info() raise finally: stmt_exit(*exc) It has a certain elegance - you can switch from using an object that needs finalisation to one that doesn't without having to change your code. And library code can safely ensure finalisation without having to check whether the object needs it or not - that check becomes an inherent part of the with statement. I'm ambivalent about this one - I see some benefit to it, but there's something niggling at the back of my brain that doesn't like it (nothing I can point to in particular, though). Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.blogspot.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Simpler finalization semantics (was Re: PEP 343 - Abstract Block Redux)
[Nick Coghlan (replying to Phillip)] Do you mean translating this: with EXPR1 as VAR1: BLOCK1 To something along the lines of: the_stmt = EXPR1 stmt_enter = getattr(the_stmt, __enter__, None) stmt_exit = getattr(the_stmt, __exit__, None) if stmt_enter is None: VAR1 = the_stmt else: VAR1 = stmt_enter() if stmt_exit is None: BLOCK1 else: exc = (None, None, None) try: try: BLOCK1 except: exc = sys.exc_info() raise finally: stmt_exit(*exc) -1. The compiler must generate both code paths but one is wasted. I know this is not a very strong argument, but my gut tells me this generalization of the with-statement is wrong, so I'll stick to it regardless of the strength of the argument. The real reason will come to me. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Simpler finalization semantics (was Re: PEP 343 - Abstract Block Redux)
At 11:03 PM 5/17/2005 +1000, Nick Coghlan wrote: Do you mean translating this: with EXPR1 as VAR1: BLOCK1 To something along the lines of: the_stmt = EXPR1 stmt_enter = getattr(the_stmt, __enter__, None) stmt_exit = getattr(the_stmt, __exit__, None) if stmt_enter is None: VAR1 = the_stmt else: VAR1 = stmt_enter() if stmt_exit is None: BLOCK1 else: exc = (None, None, None) try: try: BLOCK1 except: exc = sys.exc_info() raise finally: stmt_exit(*exc) Essentially, yes; although I was actually suggesting that there be PyResource_Enter() and PyResource_Exit() C APIs that would supply the default behaviors if the tp_resource_enter and tp_resource_exit slots were missing from the object's type. But that's an implementation detail. It has a certain elegance - you can switch from using an object that needs finalisation to one that doesn't without having to change your code. And library code can safely ensure finalisation without having to check whether the object needs it or not - that check becomes an inherent part of the with statement. Precisely. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Simpler finalization semantics (was Re: PEP 343 - Abstract Block Redux)
Nick Coghlan wrote: the_stmt = EXPR1 stmt_enter = getattr(the_stmt, __enter__, None) stmt_exit = getattr(the_stmt, __exit__, None) if stmt_enter is None: VAR1 = the_stmt else: VAR1 = stmt_enter() If we're doing this, it might be better if VAR were simply bound to EXPR in all cases. Otherwise the ability to liberally sprinkle with-statements around will be hampered by uncertainty about what kind of object VAR will end up being. Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Simpler finalization semantics (was Re: PEP 343 - Abstract Block Redux)
And we're back at PEP 310 and you can't really write opening() as a generator. On 5/17/05, Greg Ewing [EMAIL PROTECTED] wrote: Nick Coghlan wrote: the_stmt = EXPR1 stmt_enter = getattr(the_stmt, __enter__, None) stmt_exit = getattr(the_stmt, __exit__, None) if stmt_enter is None: VAR1 = the_stmt else: VAR1 = stmt_enter() If we're doing this, it might be better if VAR were simply bound to EXPR in all cases. Otherwise the ability to liberally sprinkle with-statements around will be hampered by uncertainty about what kind of object VAR will end up being. Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Simpler finalization semantics (was Re: PEP 343 - Abstract Block Redux)
At 06:56 PM 5/16/2005 +1000, Nick Coghlan wrote: Anyway, I think it's stable enough now that I can submit it to be put up on www.python.org (I'll notify the PEP editors directly once I fix a couple of errors in the current version - like the missing 'raise' in the statement semantics. . .). If you have developer checkin privileges, it's best to get a PEP number sooner rather than later, if the PEP shows any signs of viability at all. Once you're in the PEP infrastructure people can subscribe to get notified when you change it, read the revision history, and so on. Anyway, I took a look at it, and I mostly like it. There appears to be an error in Deterministic generator finalisation (maybe you already know this): the _inject_exception() should be called with exc_info, not TerminateIteration, and it should swallow StopIteration instead of TerminateIteration. IOW, I think it should look like this: def __exit__(self, *exc_info): try: self._inject_exception(*exc_info) except StopIteration: pass Hm. Oh wait, I just realized - you don't mean this at all. You're describing a use of generators as non-templates. Ugh. I think that might lead to confusion about the semantics of 'with' and generators. I'll have to think about it some more, but my second impression after a little bit of thought is that if you're going to do this, then you should be allowed to use 'with' with any object, using the object as VAR if there's no __enter__. My reasoning here is that it then makes it possible for you to use arbitrary objects for 'with' without needing to know their implementation details. It should be harmless to use 'with' on objects that don't need it. This insight may actually be true regardless of what generators do or don't do; the point is that if you change from using a generator to a built-in iterator type, you shouldn't have to change every place you were using the 'with' blocks to work again. A further part of this insight: perhaps the 'with' block translation should include a 'del VAR' in its finally block, not to mention the equivalent of 'del stmt_enter,stmt_exit'. In other words, the binding of VAR should not escape the 'with' block. This would mean that for existing types that use __del__ for cleanup (e.g. files and sockets), then 'with open(file) as f' would automatically ensure closing under CPython (but other implementations would be allowed to wait for GC). In other words, I'm saying that this: with some_expr() as foo: # etc. should also be short for this (in the case where some_expr() has no __enter__ or __exit__ methods): foo = some_expr() try: # etc. finally: del foo And that could be a useful thing for many existing object types, without even updating them for PEP 34[0-9]. :) It wouldn't be *as* useful for non-CPython implementations, but presumably by the time those implementations catch up, more code will be out there with __enter__/__exit__ methods. Also, by allowing a default __enter__ to exist (that returns self), many objects need only implement an __exit__. (For example, I don't see a point to closed file objects raising an error when used in a 'with' block; if you're actually using the file you'll already get an error when you use its other methods, and if you're not actually using it, there's no point to the error, since close() is idempotent.) So, at the C API level, I'm thinking something like Py_EnterResource(ob), that returns ob if ob has no tp_resource_enter slot defined, otherwise it returns the result of calling the method. Similarly, some sort of Py_ExitResource() that guarantees an error return after invoking the tp_resource_exit slot (if any). Finally, note that this extension now makes 'with' seem more like 'with' in other languages, because it is now just a scoped variable definition, with hooks for the object being scoped to be notified about entry and exit from scope. It does mean that people encountering 'with some_expr()' (without an as) may wonder about whether names inside the scope are somehow relative to 'some_expr', but it will probably become clear from context, especially via appropriate names. For example 'with self.__locked' might provide that extra bit of clarity beyond 'with self.__lock'. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
