Re: [Python-Dev] PEP 3103: A Switch/Case Statement
On 6/29/06, Nick Coghlan [EMAIL PROTECTED] wrote: You mean something like this?: switch x in colours: case RED: # whatever case GREEN: # whatever case BLUE: # whatever I think Guido's right. It doesn't solve the underlying problem because the compiler still has to figure out how to build a dispatch table from the possible values in colours to the actual bytecode offsets of the cases. To implement this, you actually need two lookup tables: one particular to the switch that maps labels to bytecode offsets, and one in the dispatch table to map values to labels. The former is built when the switch is compiled, and the latter is built wherever the dispatch table is defined. Each lookup is still O(1), so the whole operation remains O(1). It is O(n) or worse to check that all of the cases in the switch are defined in the dispatch table, but that only has to be done once per dispatch table/switch statement pair, and can then be stred in one or the other (probably the dispatch table, as that will be a proper object). -- Eric Sumner ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3103: A Switch/Case Statement
You mean something like this?: switch x in colours: case RED: # whatever case GREEN: # whatever case BLUE: # whatever I think Guido's right. It doesn't solve the underlying problem because the compiler still has to figure out how to build a dispatch table from the possible values in colours to the actual bytecode offsets of the cases. To implement this, you actually need two lookup tables: one particular to the switch that maps labels to bytecode offsets, and one in the dispatch table to map values to labels. The former is built when the switch is compiled, and the latter is built wherever the dispatch table is defined. Each lookup is still O(1), so the whole operation remains O(1). what's a label ? In your example, RED, GREEN, and BLUE. colours provides a mapping from values to labels/cases, and the switch statement provides a mapping from labels/cases to code. Sorry about introducing a new term without saying anything about it. -- Eric Sumner ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3103: A Switch/Case Statement
yeah, but what are they? integers? strings? names without an associated value? how do you create new labels? where are they stored? who keeps track of them? In this scheme, dispatch tables can be considered to be reverse-lookup namespaces. Where a regular namespace is used to look up a value given its name, a dispatch table is used to look up a name given its value. The switch statement then lets you actually do something based on which name is returned. -- Eric Sumner ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3103: A Switch/Case Statement
Forget subroutines for a moment - the main point of the thread was the idea that the dispatch table was built explicitly rather than automatically - that instead of arguing over first-use vs. function-definition, we let the user decide. I'm sure that my specific proposal isn't the only way that this could be done. But anything that makes the build explicit is going to be so much more ugly. And I still think you're trying to solve the wrong problem. Only if the programmer has to see it. The dispatch table need not include the behaviors of each of the cases; it only needs to define what the cases are. In most of the use cases I've seen, switch is used to define behavior for different values of an enumeration. The dispatch table for an enumeration can be built wherever the values for the enumeration are defined (such as in a module). Programmers don't need to bother with making a dispatch table unless they are defining enumeration values themselves. -- Eric Sumner Note: I sent an email yesterday with a proposal to this effect, but it seems to have been lost. If anybody wants, I can resend it. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Split switch statement
One of the big problems here seems to be that an optimized switch statement requires some code to be evaluated fewer times than the rest of the switch statement, and there isn't any good way to make that happen with a single statement. Thus, I propose two statements: a 'dispatcher' statement and a 'switch' statement. The dispatcher statement defines the available cases, and generates a dispatcher object, and the switch statement specifies code to be run for each case. - #Sample 1: Primary dispatcher syntax dispatcher myEvents on e: on e: case None: Idle on getattr(e, 'type', None): from globalEvents: * case 42: myEvent # or whatever the user event code is #EOF A dispatcher statement contains a sequence of 'on' blocks. Each 'on' block specifies an expression and a set of cases. The expression is stored as a lambda inside the dispatcher which is applied whenever the switch is run. Inside a 'on' block, there are two kinds of statements. 'case' evaluates its expression immediately, and associates it with a label; 'from' imports tests and labels from another dispatcher. If the result of any case expression is unhashable, an exception is raised. -- #Sample 2: Shorthand dispatcher syntax dispatcher globalEvents: case pygame.KEYDOWN: KEYDOWN case pygame.KEYUP: KEYUP ... #EOF Because dispatching on the switched value directly is so common, any 'from' or 'case' statements outside an 'on' block are considered to be applied to be inside an on switched_value block. The name for the switched value can be omitted if it's not needed. -- #Sample 3: Using a dispatcher while True: ... switch events on pygame.event.poll(): case KEYUP, KEYDOWN: ... case myEvent: ... case Idle: ... else: ... #EOF Internally, each switch statement has some unique identifier. Each dispatcher object maintains a list of the switch statements it has previously serviced. If this switch statement is new to this dispatcher, the dispatcher verifies that it might generate all of the cases that are specified in the switch. Otherwise, an exception is raised. If the test passed (or was skipped due to previous experience), each of the 'on' expressions in the dispatcher is executed (in order) and their results are checked against the stored values. If no case (from the switch, not the dispatcher) matches, the switch's 'else' block is executed, if present. If more than one case (from the switch) matches, an exception is raised. Otherwise, the code from associated case block is executed. -- Eric Sumner PS. Yes, I know that's not how pygame handles idle events; it makes a better sample this way. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Temporary Constantification
It seems to me that the main reason to provide constantification (in a switch statement or anywhere else) is to be able to optimize execution by caching the result locally for later use. The problem comes from trying to determine exactly when and how a value gets calculated, as most values in Python are subject to change. If, however, there was a mechanism by which a cache could be invalidated when its value changes, the only remaining difference between cached and non-cached values becomes their execution speed and memory usage (and possibly impact on the execution speed of other code). Thus, I propose a 'cached' keyword much like the static and const proposed keywords. In general, a cached value can be used (rather than re-evaluating the expression) if: - The expression has no side effects, - The result of all operations is deterministic, and - None of the expression parameters have changed since the cached value was generated The first two can be determined at compile-time without too much difficulty (except possibly function calls, I'll get to those in a minute). The hard issue here is knowing when parameters have changed. These fall into two different categories: literals and name lookups. Immutable literals don't cause a problem, and mutable literals always have a side-effect of generating a new object. There are two ways to determine if name lookups have changed: 1) Cache all the parameters, and check them against the current values, or 2) Be notified whenever one of the parameters changes. The first option requires a bunch of name lookups whenever the cached value is considered, which is exactly the problem that caching is supposed to fix. To implement the second, each name in each namespace needs a list of caches that depend on the name, and all name binding operations need to check the list and mark all dependent caches invalid. This is a small performance penalty whenever any name is rebound, and a large penalty whenever a watched name is rebound. Function calls can safely be considered volatile, but this would invalidate many of the uses for caching. Instead, each function has an immutable property of being either volatile or deterministic. Each deterministic function call maintains its own cache which is invalidated if the name to which the associated function (or any of its parameters) is rebound. Thus, if a function is rebound to something volatile, it does not force continual re-evaluation of other sub-expressions. Functions should be assumed to be volatile unless specified otherwise (probably via a decorator). I'm not particularly familiar with the internals of Python, so I'm not able to actually assess the feasability or performance implications of this proposal, but I think it basically covers the problem. -- Eric Sumner ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Temporary Constantification
On 6/25/06, Guido van Rossum [EMAIL PROTECTED] wrote: Unfortunately, a mechanism that would let you register a callback for when a particular variable or attribute used in a cached expression is used, is pretty hard to implement without affecting the performance of code that doesn't use it. I'm afraid this is not a very likely path towards a solution. I could make a strong argument that it is actually impossible to implement without affecting the performance of other code; the only issue is whether or not the impact is acceptable. I may be wrong, but I think that this particular scheme minimizes the impact: - There is a bit more data to store in every namespace - There is no change to dereferencing names; no test is required, no callback is generated - Binding to a name that currently has no binding simply requires allocating the extra memory and clearing it. - Binding to a name that is bound and does have callbacks is slow, but those are supposed to be constant *in practice* anyway. - Binding to a name that is already bound, but has no callbacks requires a test on a single variable against a constant. Without knowing more about the internals of Python (such as how long a check of a single variable takes relative to binding a new value to a name), I can't properly evaluate how much of a problem this would be. -- Eric Sumner ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Switch statement
On 6/22/06, Guido van Rossum [EMAIL PROTECTED] wrote: (3) A switch is implemented using a dict which is precomputed at the same time its static expressions are precomputed. The switch expression must be hashable. Overlap between different cases will raise an exception at precomputation time. How does this interact with __contains__, __len__, and __iter__ for the 'case in S' statement? Would it work with a class that only implements __contains__, such as a continuous range class? -- Eric Sumner ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Switch statement
On 6/23/06, Guido van Rossum [EMAIL PROTECTED] wrote: No; in order to make it possible to use a single dict lookup for dispatch, the set members are expanded into the dict key. If you have a large contiguous range, you'll be better off (sometimes *much* better) doing an explicit if/elif check before entering the switch. In that case, I would argue that the proposed syntax is misleading. Regardless of how it is implemented, a switch statement is conceptually a chain of if/elif statements. As such, the 'in' keyword, if it is allowed at all, should behave like it does in if statements, rather than it does in loops. If, for implementation reasons, you want to ensure that all of the sets are enumerable, I would recommend a syntax like this: case [*] expression (, [*] expression)* : suite This is consistent with parameter lists, which emphasizes that the sequences are being enumerated instead of simply tested against. -- Eric Sumner ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Switch statement
In that case, I would argue that the proposed syntax is misleading. Regardless of how it is implemented, a switch statement is conceptually a chain of if/elif statements. As such, the 'in' keyword, if it is allowed at all, should behave like it does in if statements, rather than it does in loops. If, for implementation reasons, you want to ensure that all of the sets are enumerable, I would recommend a syntax like this: case [*] expression (, [*] expression)* : suite This is consistent with parameter lists, which emphasizes that the sequences are being enumerated instead of simply tested against. You apparently missed the post where Guido expressed that he believes that one of the valid motivators for the switch statement and the dict-based dispatching was for that of speed improvements. He also already stated that cases could essentially only be examples for which Python does pre-computation and the storing of constants (he didn't use those words, and there are caveats with regards to module.attr and global 'constants', but that was the gist I got from it). I admit that I came into this discussion in the middle, and my initial post was for informational (to me) purposes only. I did not mean to imply by that post that the proposal was flawed in any way, just to verify that I properly understood the proposal. I am sorry if I was unclear about this. As such, because any explicit range object is neither dict-accessable as the values in the range would be, nor are they generally precomputed (or precomputable) as constants (like (1,2,3) is and 1+1 should be), your particular use-case (range objects that may implement __contains__ fast, but whose __iter__ returns a huge number of values if it were implemented as such) is not covered under switch/case, and we would likely point you back off to if/elif/else. I concur. I actually suspected as much prior to my original message on this topic, but I wanted to make sure I was understanding things correctly before attempting to make a suggestion. This is a good thing, because if switch/case ends up functionally identical to if/elif/else, then it has no purpose as a construct. On the other hand, because it is different from if/elif/else, and it is different in such a way to make certain blocks of code (arguably) easier to read or understand, (likely provably) faster, then it actually has a purpose and use. Again, I concur. My point was not that the mechanics of the construct were incorrect, but that the proposed syntax misrepresented its function. Again, I am sorry if I was unclear about this. -- Eric Sumner ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343: Context managers a superset of decorators?
On 2/12/06, Josiah Carlson [EMAIL PROTECTED] wrote:
[paragraphs swapped]
The desire for context managers to have access to its enclosing scope is
another discussion entirely, though it may do so without express
permission via stack frame manipulation.
My main point was that, with relatively small changes to 343, it can
replace the decorator syntax with a more general solution that matches
the style of the rest of the language better. The main change (access
to scopes) makes this possible, and the secondary change (altering the
block syntax) mitigates (but does not remove) the syntax difficulties
presented. I realize that I made an assumption that may not be valid;
namely, that a new scope is generated by the 'with' statement. Stack
frame manipulation would not be able to provide access to a scope that
no longer exists.
Re-read the decorator PEP: http://www.python.org/peps/pep-0318.html to
understand why both of these options (indentation and prefix notation)
are undesireable for a general decorator syntax.
With the changes that I propose, both syntaxes are equivalent and can
be used interchangeably. While each of them has problems, I believe
that in situations where one has a problem, the other usually does
not.
From this point on, I provide a point-by-point reaction to the most
applicable syntax objections listed in PEP 318. If you're not
interested in this, bail out now.
In the PEP, there is no discussion of a prefix notation in which the
decorator is placed before the 'def' on the same line. The most
similar example has the decorator between the 'def' and the parameter
list. It mentions two problems:
There are a couple of objections to this form. The first is that it breaks
easily 'greppability' of the source -- you can no longer search for 'def foo('
and find the definition of the function. The second, more serious, objection
is that in the case of multiple decorators, the syntax would be extremely
unwieldy.
The first does not apply, as this syntax does not separate 'def' and
the function name. The second is still a valid concern, but the
decorator list can easily be broken across multiple lines.
The main objection to an indented syntax seems to be that it requires
decorated functions to be indented an extra level. For simple
decorators, the compacted syntax could be used to sidestep this
problem. The main complaints about the J2 proposal don't quite apply:
the code in the block is a sequence of statements and 'with' is
already going to be added to the language as a compound statement.
-- Eric
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343: Context managers a superset of decorators?
On 2/13/06, Nick Coghlan [EMAIL PROTECTED] wrote: Eric Sumner wrote: I realize that I made an assumption that may not be valid; namely, that a new scope is generated by the 'with' statement. The with statement uses the existing scope - its just a way of factoring out try/finally boilerplate code. No more, and, in fact, fractionally less (the 'less' being the fact that just like any other Python function, you only get to supply one value to be bound to a name in the invoking scope). Ok. These changes are more substantial than I thought, then. Trying to link this with the function definition pipelining provided by decorators seems like a bit of a stretch. It certainly isn't a superset of the decorator functionality - if you want a statement that manipulates the namespace it contains, that's what class statements are for :) Several examples of how the 'with' block would be used involve transactions which are either rolled back or confirmed. All of these use the transaction capabilities of some external database. With separate scopes, the '__exit__' function can decide which names to export outwards to the containing scope. Unlike class statements, the contained scope is used temporarily and can be discarded when the 'with' statement is completed. This would allow a context manager to provide a local transaction handler. To me, it is not much of a leap from copying data between scopes to modifying it as it goes through, which is exactly what decorators do. The syntax that this provides for decorators seems reasonable enough (to me) to make the '@' syntax redundant. However, this is a larger change than I thought, and maybe not worth the effort to implement. -- Eric ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 343: Context managers a superset of decorators?
Forgive me if someone has already come up with this; I know I am coming to the party several months late. All of the proposals for decorators (including the accepted one) seemed a bit kludgey to me, and I couldn't figure out why. When I read PEP 343, I realized that they all provide a solution for an edge case without addressing the larger problem. If context managers are provided access to the contained and containing namespaces of their with statement, they can perform the same function that decorators do now. A transforming class could be implemented as: ## Code Start - class DecoratorContext(object): def __init__(self, func): self.func = func def __context__(self): return self def __enter__(self, contained, containing): pass def __exit__(self, contained, containing): for k,v in contained.iteritems(): containing[k] = self.func(v) ## Code End --- With this in place, decorators can be used with the with statement: ## Code Start - classmethod = DecoratorContext(classmethod) class foo: def __init__(self, ...): pass with classmethod: def method1(cls, ...): pass def method2(cls, ...): pass ## Code End --- The extra level of indention could be avoided by dealing with multiple block-starting statements on a line by stating that all except the last block contain only one statement: ## Code Start - classmethod = DecoratorContext(classmethod) class foo: def __init__(self, ...): pass with classmethod: def method1(cls, ...): pass with classmethod: def method2(cls, ...): pass ## Code End --- I will readily admit that I have no idea how difficult either of these suggestions would be to implement, or if it would be a good idea to do so. At this point, they are just something to think about -- Eric Sumner ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
