Re: [Python-Dev] More on contextlib - adding back a contextmanager decorator
Guido van Rossum wrote: On 5/1/06, James Y Knight [EMAIL PROTECTED] wrote: Don't forget that the majority of users will never have heard any of these discussions nor have used 2.5a1 or 2.5a2. Choose the best term for them, not for the readers of python-dev. I couldn't agree more! (Another thought, occasionally useful,is to consider that surely the number of Python programs yet to be written, and the number of Python programmers who yet have to learn the language, must surely exceed the current count. At least, one would hope so -- if that's not true, we might as well stop now. :-) If reason 1 had been my only reason for agreeing with Greg, I wouldn't have said anything :) The conflict with 'managed code' and thinking about the number of objects named 'manager' I've seen in different programs were enough to give me pause, though. I've got no real idea as to how we can get a better feel for which terminology would be clearer to people that haven't already been exposed to this topic for months, though :( Nick, do you have it in you to fix PEP 343? Or at least come up with a draft patch? We can take this off-linel with all the +0's and +1's coming in I'm pretty comfortable with this change now, although we should probably wait until later today to commit. I can handle the PEP (just getting rid of __context__ for now). I'll willingly cede the joy of actually fixing SVN to someone else, though :) Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] More on contextlib - adding back a contextmanager decorator
On 5/2/06, Nick Coghlan [EMAIL PROTECTED] wrote: Nick, do you have it in you to fix PEP 343? Or at least come up with a draft patch? We can take this off-linel with all the +0's and +1's coming in I'm pretty comfortable with this change now, although we should probably wait until later today to commit. I can handle the PEP (just getting rid of __context__ for now). I'll willingly cede the joy of actually fixing SVN to someone else, though :) OK, if you fix the PEP, I'll fix the code to match; I added most of those __context__ methods so I can delete them. Unless someone else beats me to it. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] More on contextlib - adding back a contextmanager decorator
Phillip J. Eby wrote: At 08:08 PM 4/30/2006 -0700, Guido van Rossum wrote: If you object against the extra typing, we'll first laugh at you (proposals that *only* shave a few characters of a common idiom aren't all that popular in these parts), and then suggest that you can spell foo.some_method() as foo(). Okay, you've moved me to at least +0 for dropping __context__. I have only one object myself that has a non-self __context__, and it doesn't have a __call__, so none of my code breaks beyond the need to add parentheses in a few places. ;) At least +0 here, too. I've just been so deep in this lately that it is taking a while to wind my thinking back a year or so. Still, far better to be having this discussion now than in 6 months time :) It sure has been a long and winding road back to Guido's original version of PEP 343, though! As for decimal contexts, I'm thinking maybe we should have a decimal.using(ctx=None, **kw) function, where ctx defaults to the current decimal context, and the keyword arguments are used to make a modified copy, seems like a reasonable best way to implement the behavior that __context__ was added for. And then all of the existing special machinery can go away and be replaced with a single @contextfactory. 'localcontext' would probably work as at least an interim name for such a function. with decimal.localcontext() as ctx: # use the new context here This is really an all-round improvement over the current SVN approach, where the fact that a new decimal context object is being created by the existing decimal context object is thoroughly implicit and unobvious. (I think we should stick with @contextfactory as the decorator name, btw, even if we go back to calling __enter__/__exit__ things context managers.) Agreed. And that decorator will still be useful for defining methods as well as functions. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] More on contextlib - adding back a contextmanager decorator
Greg Ewing wrote: I've been thinking about the terms guarded context and context guard. We could say that the with-statement executes its body in a guarded context (an abstract notion, not a concrete object). To do this, it creates a context guard (a concrete object) with __enter__ and __exit__ methods that set up and tear down the guarded context. This seems clearer to me, since I can more readily visualise a guard object being specially commissioned to deal with one particular job (guarding a particular invocation of a context). With only one object, there wouldn't be a need for any more terms. contrast and compare: http://pyref.infogami.com/with http://pyref.infogami.com/with-alt http://pyref.infogami.com/with-guard a distinct term for whatever the __enter__ method returns (i.e. the thing assigned to the target list) would still be nice. /F ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] More on contextlib - adding back a contextmanager decorator
Greg Ewing wrote: Also a thought on terminology. Even though it seems I may have been the person who thought it up originally, I'm not sure I like the term manager. It seems rather wooly, and it's not clear whether a context manager is supposed to manage just one context or multiple contexts. I think getting rid of __context__ should clear up most of this confusion (which is further evidence that Guido is making the right call). Once that change is made, the context expression in the with statement produces a context manager with __enter__ and __exit__ methods which set up and tear down a managed context for the body of the with statement. This is very similar to your later suggestion of context guard and guarded context. I believe this is actually going back to using the terminology as you originally suggested it (one concrete object, one abstract concept). We really only got into trouble once we tried to add a second kind of concrete object into the mix (objects with only a __context__ method). Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] More on contextlib - adding back a contextmanager decorator
Nick Coghlan wrote: the context expression in the with statement produces a context manager with __enter__ and __exit__ methods which set up and tear down a managed context for the body of the with statement. This is very similar to your later suggestion of context guard and guarded context. Currently I think I still prefer the term guard, since it does a better job of conjuring up the same sort of idea as a try-finally. There's also one other issue, what to call the decorator. I don't like @contextfactory, because it sounds like something that produces contexts, yet we have no such object. With only one object, it should probably be named after that object, i.e. @contextmanager or @contextguard. That's if we think it's really worth making it easy to abuse a generator in this way -- which I'm not convinced about. It's not as if people are going to be implementing context managers/guards every day. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] More on contextlib - adding back a contextmanager decorator
Nick Coghlan wrote: Greg Ewing wrote: Also a thought on terminology. Even though it seems I may have been the person who thought it up originally, I'm not sure I like the term manager. It seems rather wooly, and it's not clear whether a context manager is supposed to manage just one context or multiple contexts. I think getting rid of __context__ should clear up most of this confusion (which is further evidence that Guido is making the right call). Once that change is made, the context expression in the with statement produces a context manager with __enter__ and __exit__ methods which set up and tear down a managed context for the body of the with statement. This is very similar to your later suggestion of context guard and guarded context. Thinking about it a bit further. . . 1. PEP 343, 2.5 alpha 1, 2.5 alpha 2 and the discussions here have no doubt seriously confused the meaning of the term 'context manager' for a lot of people (you can certainly put me down as one such person). Anyone not already confused is likely to *become* confused if we subtly change the meaning in alpha 3. 2. The phrase managed context is unfortunately close to .NET's term managed code, and would likely lead to confusion for IronPython folks (and other programmers with .NET experience) 3. manager is an extremely generic term that is already used in a lot of different ways in various programming contexts Switching to Greg's suggestion of context guard and guarded context as the terms would allow us to hit the reset button and start the documentation afresh without terminology confusion resulting from the evolution of PEP 343 and its implementation and documentation. I think context guard also works better in terms of guarding entry to and exit from the guarded context, whereas I always wanted to call those operations set up and tear down for context managers. The current @contextfactory decorator could be renamed to @guardfactory to make it explicit that it results in a factory function for context guards. Cheers, Nick. P.S. I think I can hear anguished howls coming from the offices of various book publishers around the world ;) -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] More on contextlib - adding back a contextmanager decorator
Greg Ewing wrote: Nick Coghlan wrote: the context expression in the with statement produces a context manager with __enter__ and __exit__ methods which set up and tear down a managed context for the body of the with statement. This is very similar to your later suggestion of context guard and guarded context. Currently I think I still prefer the term guard, since it does a better job of conjuring up the same sort of idea as a try-finally. See the other message I wrote while you were writing this one for the various reasons I now agree with you :) There's also one other issue, what to call the decorator. I don't like @contextfactory, because it sounds like something that produces contexts, yet we have no such object. Agreed. With only one object, it should probably be named after that object, i.e. @contextmanager or @contextguard. That's if we think it's really worth making it easy to abuse a generator in this way -- which I'm not convinced about. It's not as if people are going to be implementing context managers/guards every day. I suggested renaming it to guardfactory in my other message. Keeping the term 'factory' in the name emphasises that the decorator results in a callable that returns a context guard, rather than producing a context guard directly. As for whether or not we should provide this ability, I think we definitely should. It allows try/finally boilerplate code to be replaced with a guarded context almost mechanically, whereas converting the same code to a manually written context guard could involve significant effort in changing from local variable based storage to instance attribute based storage. IOW, the feature is provided for the same reason that generator functions are provided: it is typically *much* easier to write a generator function than it is to write the same iterator manually. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] More on contextlib - adding back a contextmanager decorator
Fredrik Lundh wrote: a distinct term for whatever the __enter__ method returns (i.e. the thing assigned to the target list) would still be nice. I've called that the context entry value in a few places (I don't think any of them were in the actual documentation though). A sample modification to the reference page: -- Here's a more detailed description: 1. The context expression is evaluated, to obtain a context guard. 2. The guard object's __enter__ method is invoked to obtain the context entry value. 3. If a target list was included in the with statement, the context entry value is assigned to it. 4. The suite is executed. 5. The guard object's __exit__ method is invoked. If an exception caused the suite to be exited, its type, value, and traceback are passed as arguments to __exit__. Otherwise, three None arguments are supplied. -- Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] More on contextlib - adding back a contextmanager decorator
On May 1, 2006, at 8:15 AM, Nick Coghlan wrote: 1. PEP 343, 2.5 alpha 1, 2.5 alpha 2 and the discussions here have no doubt seriously confused the meaning of the term 'context manager' for a lot of people (you can certainly put me down as one such person). Anyone not already confused is likely to *become* confused if we subtly change the meaning in alpha 3. Don't forget that the majority of users will never have heard any of these discussions nor have used 2.5a1 or 2.5a2. Choose the best term for them, not for the readers of python-dev. James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] More on contextlib - adding back a contextmanager decorator
On 5/1/06, James Y Knight [EMAIL PROTECTED] wrote: Don't forget that the majority of users will never have heard any of these discussions nor have used 2.5a1 or 2.5a2. Choose the best term for them, not for the readers of python-dev. I couldn't agree more! (Another thought, occasionally useful,is to consider that surely the number of Python programs yet to be written, and the number of Python programmers who yet have to learn the language, must surely exceed the current count. At least, one would hope so -- if that's not true, we might as well stop now. :-) Nick, do you have it in you to fix PEP 343? Or at least come up with a draft patch? We can take this off-linel with all the +0's and +1's coming in I'm pretty comfortable with this change now, although we should probably wait until later today to commit. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] More on contextlib - adding back a contextmanager decorator
At 08:29 PM 5/1/2006 +1000, Nick Coghlan wrote: 'localcontext' would probably work as at least an interim name for such a function. with decimal.localcontext() as ctx: # use the new context here And the as ctx should be unnecessary for most use cases, if localcontext has an appropriately designed API. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] More on contextlib - adding back a contextmanager decorator
On Tue, May 02, 2006, Greg Ewing wrote: Nick Coghlan wrote: the context expression in the with statement produces a context manager with __enter__ and __exit__ methods which set up and tear down a managed context for the body of the with statement. This is very similar to your later suggestion of context guard and guarded context. Currently I think I still prefer the term guard, since it does a better job of conjuring up the same sort of idea as a try-finally. Guard really doesn't work for me. It seems clear (especially in light of Fredrik's docs) that wrapper comes much closer to what's going on. -- Aahz ([EMAIL PROTECTED]) * http://www.pythoncraft.com/ Argue for your limitations, and sure enough they're yours. --Richard Bach ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] More on contextlib - adding back a contextmanager decorator
A few things from the pre-alpha2 context management terminology review have
had a chance to run around in the back of my head for a while now, and I'd
like to return to a topic Paul Moore brought up during that discussion.
Paul had a feeling there should be two generator decorators in contextlib -
one for __context__ methods and one for standalone generator functions.
However, contextfactory seemed to meet both needs, so we didn't follow the
question up for alpha 2.
The second link in this chain is the subsequent discussion with Guido about
making the context manager and managed context protocols orthogonal. With this
clearer separation of the two terms to happen in alpha 3, it becomes more
reasonable to have two different generator decorators, one for defining
managed contexts and one for defining context managers.
The final link is a use case for such a context manager decorator, in the form
of a couple of HTML tag example contexts in the contextlib documentation.
(contextlib.nested is also a good use case, where caching can be used to
ensure certain resources are always acquired in the same order, but the issue
is easier to demonstrate using the HTML tag examples)
Firstly, the class-based HTML tag context manager example:
class TagClass:
def __init__(self, name):
self.name = name
@contextfactory
def __context__(self):
print %s % self.name
yield self
print /%s % self.name
h1_cls = TagClass('h1')
with h1_cls:
... print Header A
...
h1
Header A
/h1
with h1_cls:
... print Header B
...
h1
Header B
/h1
Each with statement creates a new context object, so caching the tag object
itself works just as you would expect. Unfortunately, the same cannot be said
for the generator based version:
@contextfactory
def tag_gen(name):
print %s % name
yield
print /%s % name
h1_gen = tag_gen('h1')
with h1_gen:
... print Header A
...
h1
Header A
/h1
with h1_gen:
... print Header B
...
Traceback (most recent call last):
...
RuntimeError: generator didn't yield
The managed contexts produced by the context factory aren't reusable, so
caching them doesn't work properly - they need to be created afresh for each
with statement.
Adding another decorator to define context managers, as Paul suggested, solves
this problem neatly. Here's a possible implementation:
def managerfactory(gen_func):
# Create a context manager factory from a generator function
context_factory = contextfactory(gen_func)
def wrapper(*args, **kwds):
class ContextManager(object):
def __context__(self):
return context_factory(*args, **kwds)
mgr = ContextManager()
mgr.__context__() # Throwaway context to check arguments
return mgr
# Having @functools.decorator would eliminate the next 4 lines
wrapper.__name__ = context_factory.__name__
wrapper.__module__ = context_factory.__module__
wrapper.__doc__ = context_factory.__doc__
wrapper.__dict__.update(context_factory.__dict__)
return wrapper
@managerfactory
def tag_gen2(name):
print %s % name
yield
print /%s % name
h1_gen2 = tag_gen2('h1')
with h1_gen2:
... print Header A
...
h1
Header A
/h1
with h1_gen2:
... print Header B
...
h1
Header B
/h1
--
Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia
---
http://www.boredomandlaziness.org
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] More on contextlib - adding back a contextmanager decorator
On 4/30/06, Nick Coghlan [EMAIL PROTECTED] wrote: A few things from the pre-alpha2 context management terminology review have had a chance to run around in the back of my head for a while now, and I'd like to return to a topic Paul Moore brought up during that discussion. I believe the context API design has gotten totally out of hand. Regardless of the merits of the with approach to HTML generation (which I personally believe to be an abomination), I don't see why the standard library should support every possible use case with a custom-made decorator. Let the author of that tag library provide the decorator. I have a counter-proposal: let's drop __context__. Nearly all use cases have __context__ return self. In the remaining cases, would it really be such a big deal to let the user make an explicit call to some appropriately named method? The only example that I know of where __context__ doesn't return self is the decimal module. So the decimal users would have to type with mycontext.some_method() as ctx:# ctx is a clone of mycontext ctx.prec += 2 BODY The implementation of some_method() could be exactly what we currently have as the __context__ method on the decimal.Context object. Its return value is a decimal.WithStatementContext() instance, whose __enter__() method returns a clone of the original context object which is assigned to the variable in the with-statement (here 'ctx'). This even has an additional advantage -- some_method() could have keyword parameters to set the precision and various other context parameters, so we could write this: with mycontext.some_method(prec=mycontext.prec+2): BODY Note that we can drop the variable too now (unless we have another need to reference it). An API tweak for certain attributes that are often incremented or decremented could reduce writing: with mycontext.some_method(prec_incr=2): BODY -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] More on contextlib - adding back a contextmanager decorator
At 09:53 AM 4/30/2006 -0700, Guido van Rossum wrote: I have a counter-proposal: let's drop __context__. Nearly all use cases have __context__ return self. In the remaining cases, would it really be such a big deal to let the user make an explicit call to some appropriately named method? The only example that I know of where __context__ doesn't return self is the decimal module. So the decimal users would have to type with mycontext.some_method() as ctx:# ctx is a clone of mycontext ctx.prec += 2 BODY The implementation of some_method() could be exactly what we currently have as the __context__ method on the decimal.Context object. Its return value is a decimal.WithStatementContext() instance, whose __enter__() method returns a clone of the original context object which is assigned to the variable in the with-statement (here 'ctx'). This even has an additional advantage -- some_method() could have keyword parameters to set the precision and various other context parameters, so we could write this: with mycontext.some_method(prec=mycontext.prec+2): BODY Note that we can drop the variable too now (unless we have another need to reference it). An API tweak for certain attributes that are often incremented or decremented could reduce writing: with mycontext.some_method(prec_incr=2): BODY But what's an appropriate name for some_method? Given that documentation is the sore spot that keeps us circling around this point, doesn't this just push the problem to finding a name to use in place of __context__? And not only for this use case, but for others? After all, for any library that has a notion of the current X, it seems reasonable to want to be able to say with some_X to mean use some_X as the current X for this block. And it thus seems to me that people will want to have something like: def using(obj): if hasattr(obj,'__context__'): obj = obj.__context__() return obj so they can do with using(some_X), because with some_X.using() or with some_X.as_current() is awkward. If you can solve the naming issue for these use cases (and I notice you punted on that issue by calling it some_method), then +1 on removing __context__. Otherwise, I'm -0; we're just fixing one documentation/explanation problem (that only people writing contexts will care about) by creating others (that will affect the people *using* contexts too). ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] More on contextlib - adding back a contextmanager decorator
On 4/30/06, Guido van Rossum [EMAIL PROTECTED] wrote: On 4/30/06, Nick Coghlan [EMAIL PROTECTED] wrote: A few things from the pre-alpha2 context management terminology review have had a chance to run around in the back of my head for a while now, and I'd like to return to a topic Paul Moore brought up during that discussion. I believe the context API design has gotten totally out of hand. Regardless of the merits of the with approach to HTML generation (which I personally believe to be an abomination), I don't see why the standard library should support every possible use case with a custom-made decorator. Let the author of that tag library provide the decorator. I have a counter-proposal: let's drop __context__. Nearly all use cases have __context__ return self. In the remaining cases, would it really be such a big deal to let the user make an explicit call to some appropriately named method? The only example that I know of where __context__ doesn't return self is the decimal module. So the decimal users would have to type +1. -Brett with mycontext.some_method() as ctx:# ctx is a clone of mycontext ctx.prec += 2 BODY The implementation of some_method() could be exactly what we currently have as the __context__ method on the decimal.Context object. Its return value is a decimal.WithStatementContext() instance, whose __enter__() method returns a clone of the original context object which is assigned to the variable in the with-statement (here 'ctx'). This even has an additional advantage -- some_method() could have keyword parameters to set the precision and various other context parameters, so we could write this: with mycontext.some_method(prec=mycontext.prec+2): BODY Note that we can drop the variable too now (unless we have another need to reference it). An API tweak for certain attributes that are often incremented or decremented could reduce writing: with mycontext.some_method(prec_incr=2): BODY -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] More on contextlib - adding back a contextmanager decorator
Guido van Rossum wrote: On 4/30/06, Nick Coghlan [EMAIL PROTECTED] wrote: A few things from the pre-alpha2 context management terminology review have had a chance to run around in the back of my head for a while now, and I'd like to return to a topic Paul Moore brought up during that discussion. I believe the context API design has gotten totally out of hand. Regardless of the merits of the with approach to HTML generation (which I personally believe to be an abomination), The example is tempting because it's easy to follow. I agree actually doing it in real code would almost certainly be nuts :) I don't see why the standard library should support every possible use case with a custom-made decorator. Let the author of that tag library provide the decorator. The HTML tag was just an example. The underlying idea is being able to easily create a re-usable object that can be passed to multiple with statements (potentially nested within each other or within distinct threads). Without the __context__ method, the naive version of such an object looks like: class reusable(object): def __init__(self, factory): self.factory = factory factory() # Check the factory works at definition time def __enter__(self): current = self.current = factory() return current.__enter__() def __exit__(self, *exc_info): return self.current.__exit__(*exc_info) The downside of this over the __context__ method is that it is neither nesting nor thread-safe. Because the storage is on the object rather than in the execution frame, sharing such objects between threads or using one for nested with statements will break (as self.current gets overwritten). I have a counter-proposal: let's drop __context__. Nearly all use cases have __context__ return self. In the remaining cases, would it really be such a big deal to let the user make an explicit call to some appropriately named method? The only example that I know of where __context__ doesn't return self is the decimal module. It would also prevent threading.Condition from using its underlying lock object as the managed context. The real problem I have with removing __context__() is that it pushes the burden of handling thread-safety and nesting-safety issues onto the developers of context managers without giving them any additional tools beyond threading.locals(). This was the problem Jason brought up for decimal.Context that lead to the introduction of __context__ in the first place. Without the __context__() method, *users* of the with statement will be forced to create a new object with __enter__()/__exit__() methods every time, either by invoking a method (whose name will vary from object to object, depending on the whim of the designer) or by calling a factory function (which is likely to be created either as a zero-argument lambda returning an object with enter/exit methods, or else by using PEP 309's partial function). So if you see a with statement with a bare variable name as the context expression, it will probably be wrong, unless: a) the implementor of that type provided thread-safety and nesting-safety; or b) the object is known to be neither thread-safe nor nesting-safe The synchronisation objects in threading being examples of category a, file objects being examples of category b. In this scenario, generator contexts defined using @contextfactory should always be invoked directly in the context expression, as attempting to cache them in order to be reused won't work (you would need to put them in a zero-argument lambda and call it in the context expression, so that you get a new generator object each time). Documenting all of the thread-safety and nesting-safety issues and how to deal with them would be a serious pain. I consider it much easier to provide the __context__() method and explain how to use that as the one obvious way to deal with such problems. Then only implementors need to care about it - from a user's point of view, you just provide a context expression that resolves to a context manager, and everything works as intended, including being able to cache that expression in a local variable and use it multiple times. (That last point obviously not applying to context managers like files that leave themselves in an unusable state after __exit__, and don't restore themselves to a usable state in __enter__). Essentially, I don't think dropping __context__ would gain us anything - the complexity associated with it is real, and including that method in the API let's us deal with that complexity in one place, once and for all. Removing the method from the statement definition just pushes the documentation burden out to all of the context managers where it matters (like decimal.Context, the documentation for which would get stuck with trying to explain why you have to call a method in order to get a usable context manager). Cheers, Nick.
Re: [Python-Dev] More on contextlib - adding back a contextmanager decorator
[I'm cutting straight to the chase here] On 4/30/06, Nick Coghlan [EMAIL PROTECTED] wrote: The downside of this over the __context__ method is that it is neither nesting nor thread-safe. This argument is bogus. We currently have two types of objects involved in with-statements: those whose __context__ returns self and those whose __context__ returns some other object. In all of the latter cases, the original object doesn't have __enter__ or __exit__ methods, so using it in the reduced-with-statement will be an immediate run-time error. Writing a method that returns the correct type of object with the correct behavior (thread-safe, or nesting, or whatever is required by that specific object) is no harder whether the method name is __context__ or not. The real problem I have with removing __context__() is that it pushes the burden of handling thread-safety and nesting-safety issues onto the developers of context managers without giving them any additional tools beyond threading.locals(). This was the problem Jason brought up for decimal.Context that lead to the introduction of __context__ in the first place. Again, I don't see how writing the thread-safe version is easier when the method is called __context__. Without the __context__() method, *users* of the with statement will be forced to create a new object with __enter__()/__exit__() methods every time, You seem to be missing the evidence that 9 out of 10 objects currently have a __context__ that returns self. In all those cases the user of the with-statement won't have to make any changes at all compared to code that works with 2.5a2. either by invoking a method (whose name will vary from object to object, depending on the whim of the designer) or by calling a factory function (which is likely to be created either as a zero-argument lambda returning an object with enter/exit methods, or else by using PEP 309's partial function). Having the name being different in each situation may actually be an advantage -- it will give an additional clue as to what is happening, and it will let us design different APIs for use in a with-statement (just like dicts have iterkeys() and iteritems()). So if you see a with statement with a bare variable name as the context expression, it will probably be wrong, unless: a) the implementor of that type provided thread-safety and nesting-safety; or b) the object is known to be neither thread-safe nor nesting-safe The synchronisation objects in threading being examples of category a, file objects being examples of category b. In this scenario, generator contexts defined using @contextfactory should always be invoked directly in the context expression, as attempting to cache them in order to be reused won't work (you would need to put them in a zero-argument lambda and call it in the context expression, so that you get a new generator object each time). Now we get to the crux of the matter. When I recommend that people write with foo.some_method(): BLOCK you are worried that if they need two separate blocks like that, they are tempted to write x = foo.some_method() with x: BLOCK1 with x: BLOCK2 But there's an obvious solution for that: make sure that the object returned by foo.some_method() can only be used once. The second with x will raise an exception explaining what went wrong. (We could even rig it so that nesting the second with x inside the first one will produce a different error message.) So the with foo.some_method() idiom is the only one that works, and that is just as thread-safe and nesting-resilient as is writing __context__() today. If you object against the extra typing, we'll first laugh at you (proposals that *only* shave a few characters of a common idiom aren't all that popular in these parts), and then suggest that you can spell foo.some_method() as foo(). Essentially, I don't think dropping __context__ would gain us anything - the complexity associated with it is real, and including that method in the API let's us deal with that complexity in one place, once and for all. It's not real in 9 out of 10 current cases. (And the Condition variable could easily be restored to its previous case where it had its own __enter__ and __exit__ methods that called the corresponding methods of the underlying lock. The code currently in svn is merely an optimization.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] More on contextlib - adding back a contextmanager decorator
At 08:08 PM 4/30/2006 -0700, Guido van Rossum wrote: If you object against the extra typing, we'll first laugh at you (proposals that *only* shave a few characters of a common idiom aren't all that popular in these parts), and then suggest that you can spell foo.some_method() as foo(). Okay, you've moved me to at least +0 for dropping __context__. I have only one object myself that has a non-self __context__, and it doesn't have a __call__, so none of my code breaks beyond the need to add parentheses in a few places. ;) As for decimal contexts, I'm thinking maybe we should have a decimal.using(ctx=None, **kw) function, where ctx defaults to the current decimal context, and the keyword arguments are used to make a modified copy, seems like a reasonable best way to implement the behavior that __context__ was added for. And then all of the existing special machinery can go away and be replaced with a single @contextfactory. (I think we should stick with @contextfactory as the decorator name, btw, even if we go back to calling __enter__/__exit__ things context managers.) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
