Re: [Python-Dev] Proposed changes to PEP 343
On 10/7/05, Fredrik Lundh [EMAIL PROTECTED] wrote: the whole concept might be perfectly fine on the this construct corre- sponds to this code level, but if you immediately end up with things that are not what they seem, and names that don't mean what the say, either the design or the description of it needs work. (yes, I know you can use this class to manage the context, but it's not really a context manager, because it's that method that's a manager, not the class itself. yes, all the information that belongs to the context are managed by the class, but that doesn't make... oh, shut up and read the PEP) Good points... Maybe it is the description that needs work. Here is a description of iterators, to illustrate the parallels: An object that has an __iter__ method is iterable. It can plug into the Python 'for' statement. obj.__iter__() returns an iterator. An iterator is a single-use, forward-only view of a sequence. 'for' calls __iter__() and uses the resulting iterator's next() method. (This is just as complicated as PEP343+changes, but not as mindboggling, because the terminology is better. Also because we're used to iterators.) Now contexts, per PEP 343 with Nick's proposed changes: An object that has a __with__ method is a context. It can plug into the Python 'with' statement. obj.__with__() returns a context manager. A context manager is a single-use object that manages a single visit into a context. 'with' calls __with__() and uses the resulting context manager's __enter__() and __exit__() methods. A contextmanager is a function that returns a new context manager. Okay, that last bit is weird. But note that PEP 343 has this oddness even without the proposed changes. Perhaps either context manager or contextmanager should be renamed, regardless of whether Nick's changes are accepted. With the changes, context managers will be (conceptually) single-use. So maybe a different term might be appropriate. Perhaps ticket. A ticket is a single-use object that manages a single visit into a context. -j ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed changes to PEP 343
On 10/9/05, Anders J. Munch [EMAIL PROTECTED] wrote: Nick Coghlan wrote: Anders J. Munch wrote: Note that __with__ and __enter__ could be combined into one with no loss of functionality: abc,VAR = (EXPR).__with__() They can't be combined, because they're invoked on different objects. Sure they can. The combined method first does what __with__ would have done to create abc, and then does whatever abc.__enter__ would have done. Since the type of 'abc' is always known to the author of __with__, this is trivial. I'm sure it can be done, but I find this ugly API design. While I'm not keen on complicating the API, the decimal context example has convinced me that it's necessary. The separation into __with__ which asks EXPR for a context manager and __enter__ / __exit__ which handle try/finally feels right. An API returning a tuple is asking for bugs. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Proposed changes to PEP 343
Based on Jason's comments regarding decimal.Context, and to explicitly cover the terminology agreed on during the documentation discussion back in July, I'm proposing a number of changes to PEP 343. I'll be updating the checked in PEP assuming there aren't any objections in the next week or so (and assuming I get CVS access sorted out ;). The idea of dropping __enter__/__exit__ and defining the with statement solely in terms of coroutines is *not* included in the suggested changes, but I added a new item under Resolved Open Issues to cover some of the reasons why. Cheers, Nick. 1. Amend the statement specification such that: with EXPR as VAR: BLOCK is translated as: abc = (EXPR).__with__() exc = (None, None, None) VAR = abc.__enter__() try: try: BLOCK except: exc = sys.exc_info() raise finally: abc.__exit__(*exc) 2. Add the following to the subsequent explanation: The call to the __with__ method serves a similar purpose to the __iter__ method for iterables and iterators. An object such as threading.Lock may provide its own __enter__ and __exit__ methods, and simply return 'self' from its __with__ method. A more complex object such as decimal.Context may return a distinct context manager which takes care of setting and restoring the appropriate decimal context in the thread. 3. Update ContextWrapper in the Generator Decorator section to include: def __with__(self): return self 4. Add a paragraph to the end of the Generator Decorator section: By applying the @contextmanager decorator to a context's __with__ method, it is as easy to write a generator-based context manager for the context as it is to write a generator-based iterator for an iterable (see the decimal.Context example below). 5. Add three items under Resolved Open Issues: 2. After this PEP was originally approved, a subsequent discussion on python-dev [4] settled on the term context manager for objects which provide __enter__ and __exit__ methods, and context management protocol for the protocol itself. With the addition of the __with__ method to the protocol, a natural extension is to call objects which provide only a __with__ method contexts (or manageable contexts in situations where the general term context would be ambiguous). The distinction between a context and a context manager is very similar to the distinction between an iterable and an iterator. 3. The originally approved version of this PEP did not include a __with__ method - the method was only added to the PEP after Jason Orendorff pointed out the difficulty of writing appropriate __enter__ and __exit__ methods for decimal.Context [5]. This approach allows a class to use the @contextmanager decorator to defines a native context manager using generator syntax. It also allows a class to use an existing independent context manager as its native context manager by applying the independent context manager to 'self' in its __with__ method. It even allows a class written in C to use a coroutine based context manager written in Python. The __with__ method parallels the __iter__ method which forms part of the iterator protocol. 4. The suggestion was made by Jason Orendorff that the __enter__ and __exit__ methods could be removed from the context management protocol, and the protocol instead defined directly in terms of the coroutine interface described in PEP 342 (or a cleaner version of that interface with start() and finish() convenience methods) [6]. Guido rejected this idea [7]. The following are some of benefits of keeping the __enter__ and __exit__ methods: - it makes it easy to implement a simple context manager in C without having to rely on a separate coroutine builder - it makes it easy to provide a low-overhead implementation for context managers which don't need to maintain any special state between the __enter__ and __exit__ methods (having to use a coroutine for these would impose unnecessary overhead without any compensating benefit) - it makes it possible to understand how the with statement works without having to first understand the concept of a coroutine 6. Add new references: [4] http://mail.python.org/pipermail/python-dev/2005-July/054658.html [5] http://mail.python.org/pipermail/python-dev/2005-October/056947.html [6] http://mail.python.org/pipermail/python-dev/2005-October/056969.html [7] http://mail.python.org/pipermail/python-dev/2005-October/057018.html 7. Update Example 4 to include a __with__ method: def __with__(self):
Re: [Python-Dev] Proposed changes to PEP 343
Nick Coghlan wrote: 9. Here's a proposed native context manager for decimal.Context: # This would be a new decimal.Context method @contextmanager def __with__(self): wouldn't it be better if the ContextWrapper class (or some variation thereof) could be used as a base class for the decimal.Context class? using decorators on methods to provide is a behaviour for the class doesn't really feel pythonic... /F ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed changes to PEP 343
Fredrik Lundh wrote: Nick Coghlan wrote: 9. Here's a proposed native context manager for decimal.Context: # This would be a new decimal.Context method @contextmanager def __with__(self): wouldn't it be better if the ContextWrapper class (or some variation thereof) could be used as a base class for the decimal.Context class? using decorators on methods to provide is a behaviour for the class doesn't really feel pythonic... That's not what the decorator is for - it's there to turn the generator used to implement the __with__ method into a context manager, rather than saying anything about decimal.Context as a whole. However, requiring a decorator to get a slot to work right looks pretty ugly to me, too. What if we simply special-cased the __with__ slot in type(), such that if it is populated with a generator object, that object is automatically wrapped using the @contextmanager decorator? (Jason actually suggested this idea previously) I initially didn't like the idea because of EIBTI, but I've realised that def __with__(self): is pretty darn explicit in its own right. I've also realised that defining __with__ using a generator, but forgetting to add the @contextmanager to the front would be a lovely source of bugs, particularly if generators are given a default __exit__() method that simply invokes self.close(). On the other hand, if __with__ is special-cased, then the slot definition wouldn't look ugly, and we'd still be free to define a generator's normal with statement semantics as: def __exit__(self, *exc): self.close() Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.blogspot.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed changes to PEP 343
Nick Coghlan [EMAIL PROTECTED] writes: What if we simply special-cased the __with__ slot in type(), such that if it is populated with a generator object, that object is automatically wrapped using the @contextmanager decorator? (Jason actually suggested this idea previously) nit You don't want to check if it's a generator, you want to check if it's a function whose func_code has the relavent bit set. /nit Seems a bit magical to me, but haven't thought about it hard. Cheers, mwh -- I think my standards have lowered enough that now I think ``good design'' is when the page doesn't irritate the living fuck out of me.-- http://www.jwz.org/gruntle/design.html ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed changes to PEP 343
Nick Coghlan did a +1 job to write: 1. Amend the statement specification such that: with EXPR as VAR: BLOCK is translated as: abc = (EXPR).__with__() exc = (None, None, None) VAR = abc.__enter__() try: try: BLOCK except: exc = sys.exc_info() raise finally: abc.__exit__(*exc) Note that __with__ and __enter__ could be combined into one with no loss of functionality: abc,VAR = (EXPR).__with__() exc = (None, None, None) try: try: BLOCK except: exc = sys.exc_info() raise finally: abc.__exit__(*exc) - Anders ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed changes to PEP 343
Nick Coghlan wrote: 1. Amend the statement specification such that: with EXPR as VAR: BLOCK is translated as: abc = (EXPR).__with__() exc = (None, None, None) VAR = abc.__enter__() try: try: BLOCK except: exc = sys.exc_info() raise finally: abc.__exit__(*exc) Is this correct? What happens to with 40*13+2 as X: print X ? --eric ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed changes to PEP 343
Eric Nieuwland wrote: What happens to with 40*13+2 as X: print X It would fail with a TypeError because the relevant slot in the type object was NULL - the TypeError checks aren't shown for simplicity's sake. This behaviour isn't really any different from the existing PEP 343 - the only difference is that the statement looks for a __with__ slot on the original EXPR, rather than looking directly for an __enter__ slot. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.blogspot.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed changes to PEP 343
Anders J. Munch wrote: Note that __with__ and __enter__ could be combined into one with no loss of functionality: abc,VAR = (EXPR).__with__() They can't be combined, because they're invoked on different objects. It would be like trying to combine __iter__() and next() into the same method for iterators. . . Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.blogspot.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed changes to PEP 343
Michael Hudson wrote: nit You don't want to check if it's a generator, you want to check if it's a function whose func_code has the relavent bit set. /nit Fair point :) Seems a bit magical to me, but haven't thought about it hard. Same here - I'm just starting to think that the alternative is worse, because it leaves open the nonsensical possibility of writing a __with__ method as a generator *without* applying the contextmanager decorator, and that would just be bizarre - if you want to get an iterable, why aren't you writing an __iter__ method instead? Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.blogspot.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed changes to PEP 343
Nick Coghlan wrote: Eric Nieuwland wrote: What happens to with 40*13+2 as X: print X It would fail with a TypeError because the relevant slot in the type object was NULL - the TypeError checks aren't shown for simplicity's sake. This behaviour isn't really any different from the existing PEP 343 - the only difference is that the statement looks for a __with__ slot on the original EXPR, rather than looking directly for an __enter__ slot. Hmmm I hadn't noticed that. In my memory a partial implementation of the protocol was possible. Thus, __enter__/__exit__ would only be called if they exist. Oh well, I'll just add some empty methods. --eric ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed changes to PEP 343
Nick Coghlan wrote: That's not what the decorator is for - it's there to turn the generator used to implement the __with__ method into a context manager, rather than saying anything about decimal.Context as a whole. possibly, but using a decorated __with__ method doesn't make much sense if the purpose isn't to turn the class into something that can be used with the with statement. However, requiring a decorator to get a slot to work right looks pretty ugly to me, too. the whole concept might be perfectly fine on the this construct corre- sponds to this code level, but if you immediately end up with things that are not what they seem, and names that don't mean what the say, either the design or the description of it needs work. (yes, I know you can use this class to manage the context, but it's not really a context manager, because it's that method that's a manager, not the class itself. yes, all the information that belongs to the context are managed by the class, but that doesn't make... oh, shut up and read the PEP) /F ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposed changes to PEP 343
Fredrik Lundh wrote: Nick Coghlan wrote: However, requiring a decorator to get a slot to work right looks pretty ugly to me, too. the whole concept might be perfectly fine on the this construct corre- sponds to this code level, but if you immediately end up with things that are not what they seem, and names that don't mean what the say, either the design or the description of it needs work. (yes, I know you can use this class to manage the context, but it's not really a context manager, because it's that method that's a manager, not the class itself. yes, all the information that belongs to the context are managed by the class, but that doesn't make... oh, shut up and read the PEP) Heh. OK, my current inclinitation is to make the new paragraph at the end of the Generator Decorator section read like this: 4. Add a paragraph to the end of the Generator Decorator section: If a generator is used to write a context's __with__ method, then Python's type machinery will automatically take care of applying this decorator. This means that it is just as easy to write a generator-based context manager for a context as it is to write a generator-based iterator for an iterable (see the decimal.Context example below). And then update the decimal.Context example to remove the @contextmanager decorator. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.blogspot.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
