Re: [Python-Dev] Son of PEP 246, redux
On Thu, 13 Jan 2005 01:04:01 -0500, Phillip J. Eby [EMAIL PROTECTED] wrote: At 12:01 AM 1/13/05 -0500, Michael Walter wrote: What am I missing? The fact that this is a type-declaration issue, and has nothing to do with *how* types are checked. I was talking about how you declare such types, sir :] (see the interface pseudo code sample -- maybe my reference to type inference lead you to think the opposite.) In other words, compared to the previous state of things, this should actually require *fewer* interfaces to accomplish the same use cases, and it doesn't require Python to have a built-in notion of interface, because the primitive notion is an operation, not an interface. Yepyep, but *how* you declare types now? Can you quickly type the function def f(x): x.read()? without needing an interface interface x_of_f: def read(): pass or a decorator like @foo(x.read)? I've no idea what you mean, really :o) Michael ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Son of PEP 246, redux
Michael Walter wrote: Yepyep, but *how* you declare types now? Can you quickly type the function def f(x): x.read()? without needing an interface interface x_of_f: def read(): pass or a decorator like @foo(x.read)? I've no idea what you mean, really :o) Why would something like def f(x): x.read() do any type checking at all? Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Son of PEP 246, redux
At 08:50 AM 1/13/05 +0100, Alex Martelli wrote: Your proposals are novel and interesting. They also go WAY deeper into a critical reappraisal of the whole object model of Python, which has always been quite reasonably close to the above-mentioned canon and indeed has been getting _more_ so, rather than less, since 2.2 (albeit in a uniquely Pythonical way, as is Python's wont -- but not conceptually, nor, mostly, practically, all that VERY far from canonic OOP). Actually, the whole generic function thing was just a way to break out of the typing problems the Python community has struggled with for years. Every attempt to bring typing or interfaces to Python has run aground on simple, practical concepts like, what is the abstract type of dict? file? In essence, the answer is that Python's object model *already* has a concept of type... duck typing! Basically, I'm proposing a way to formalize duck typing... when you say you want a 'file', then when you call the object's read method, it reads like a file. This isn't a critical reappraisal of Python's current object model at all! It's a reappraisal of the ways we've been trying (and largely failing) to make it fit into the mold of other languages' object models. Moreover, your proposals are at a very early stage and no doubt need a lot more experience, discussion, maturation, and give-and-take. Agreed. In particular, operations on numeric types are the main area that needs conceptual work; most of the rest is implementation details. I hope to spend some time prototyping an implementation this weekend (I start a new contract today, so I'll be busy with real work till then). Nonetheless, the idea is so exciting I could barely sleep, and even as I woke this morning my head was spinning with comparisons of adaptation networks and operation networks, finding them isomorphic no matter what I tried. One really exciting part is that this concept basically allows you to write good adapters, while making it very difficult or impossible to write bad ones! Since it forces adapters to have no per-adapter state, it almost literally forces you to only create as-a adapters. For example, if you try to define file operations on a string, you're dead in the water before you even start: strings have no place to *store* any state. So, you can't adapt an immutable into a mutable. You *can*, however, add extra state to a mutable, but it has to be per-object state, not per-adapter state. (Coincidentally, this eliminates the need for PyProtocols' somewhat kludgy concept of sticky adapters.) As a result of this, this adapter model is shaped like a superset of COM - you can adapt an object as many times as you like, and the adapters you get are basically pointers to interfaces on the same object, with no adapter composition. And adapt(x,object) can always give you back the original object. This model also has some potential to improve performance: adapter classes have all their methods in one dictionary, so there's no __mro__ scan, and they have no instance dictionary, so there's no __dict__ lookup. This should mean faster lookups of methods that would otherwise be inherited, even without any special interpreter support. And it also allows a possible fast-path opcode to be used for calling methods on a type-declared parameter or variable, possibly eventually streamlined to a vtable-like structure eliminating any dictionary lookups at all (except at function definition time, to bind names in the code object to vtable offsets obtained from the types being bound to the function signature). But of course all that is much further down the road. Anyway, I suppose the *really* exciting thing about all this is how *many* different problems the approach seems to address. :) (Like automatically detecting certain classes of Liskov violations, for example. And not having to create adapter classes by hand. And being able to support Ping's abstract operations. Etc., etc., etc.) So, I think the best course of action at this time might be for me to edit PEP 246 to reflect some of this enormously voluminous discussion, including points of contention (it's part of a PEP's job to also indicate points of dissent, after all); and I think you should get a new PEP number to use for your new ideas, and develop them on that separate PEP, say PEP XYZ. Knowing that a rethink of the whole object-model and related canon is going on at the same time should help me keep PEP 246 reasonably minimal and spare, very much in the spirit of YAGNI -- as few features as possible, for now. Sounds good to me. If Guido, in consequence, decides to completely block 246's progress while waiting for the Copernican Revolution of your new PEP XYZ to mature, so be it -- his ``nose'' will no doubt be the best guide to him on the matter. It's not that big of a revolution, really. This should make Python *more* like Python, not less. But I
Re: [Python-Dev] Son of PEP 246, redux
At 11:26 PM 1/12/05 -0500, Clark C. Evans wrote: Regardless, getting back to the main goal I had when writing PEP 246 -- your alternative proposal still doesn't seem to provide a mechanism for component developers to have a dialogue with one another to connect components without involving the application programmer. Eh? You still have adapt(); you still have adapters. The only difference is that I've specified a way to not need interfaces - instead interfaces can be defined in terms of individual operations, and those operations can be initially defined by an abstract base, concrete class, or an interface object. Oh, and you don't have to write adapter *classes* - you write adapting *methods* for individual operations. This can be done by the original author of a class or by a third party -- just like with PEP 246. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Son of PEP 246, redux
instead interfaces can be defined in terms of individual operations, and those operations can be initially defined by an abstract base, concrete class, or an interface object. I think this is quite problematic in the sense that it will force many dummy interfaces to be created. At least without type inference, this is a no-no. Consider: In order to type a function like: def f(x): # ... x.foo() # ... ...so that type violations can be detected before the real action takes place, you would need to create a dummy interface as in: interface XAsFUsesIt: def foo(): pass def f(x : XAsFUsesIt): # ... ...or you would want type inference (which at compile time types x as a thing which has a 'nullary' foo() function) and a type system like System CT. Former appears cumbersome (as it should really be done for every function), latter too NIMPY-ish wink. What am I missing? Sleepingly yours, Michael On Wed, 12 Jan 2005 23:48:47 -0500, Phillip J. Eby [EMAIL PROTECTED] wrote: At 11:26 PM 1/12/05 -0500, Clark C. Evans wrote: Regardless, getting back to the main goal I had when writing PEP 246 -- your alternative proposal still doesn't seem to provide a mechanism for component developers to have a dialogue with one another to connect components without involving the application programmer. Eh? You still have adapt(); you still have adapters. The only difference is that I've specified a way to not need interfaces - instead interfaces can be defined in terms of individual operations, and those operations can be initially defined by an abstract base, concrete class, or an interface object. Oh, and you don't have to write adapter *classes* - you write adapting *methods* for individual operations. This can be done by the original author of a class or by a third party -- just like with PEP 246. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/michael.walter%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Son of PEP 246, redux
At 09:57 PM 1/12/05 -0500, Phillip J. Eby wrote: class StringIO: def read(self, bytes) implements file.read: # etc... could be used to indicate the simple case where you are conforming to an existing operation definition. A third-party definition, of the same thing might look like this: def file.read(self: StringIO, bytes): return self.read(bytes) Assuming, of course, that that's the syntax for adding an implementation to an existing operation. After some more thought, I think this approach: 1. Might not actually need generic functions to be implemented. I need to think some more about properties and Ka-Ping Yee's abstract method idea, to make sure they can be made to work without real generic functions, but a basic version of this approach should be implementable with just a handful of dictionaries and decorators. 2. Can be prototyped in today's Python, whether generic functions are used or not (but the decorator syntax might be ugly, and the decorator implementations might be hacky) 3. May still have some rough bits with respect to subclassing Liskov; I need to work through that part some more. My preliminary impression is that it might be safe to consider inherited (but not overridden) methods as being the same logical operation. That imposes some burden on subclassers to redeclare compatibility on overridden methods, but OTOH would be typesafe by default. 4. Might be somewhat more tedious to declare adaptations with, than it currently is with tools like PyProtocols. Anyway, the non-generic-function implementation would be to have 'adapt()' generate (and cache!) an adapter class by going through all the methods of the target class and then looking them up in its 'implements' registry, while walking up the source class' __mro__ to find the most-specific implementation for that type (while checking for overridden-but-not-declared methods along the way). There would be no __conform__ or __adapt__ hooks needed. Interestingly, C# requires you to declare when you are intentionally overriding a base class method, in order to avoid accidentally overriding a new method added to a base class later. This concept actually contains a germ of the same idea, requiring overrides to specify that they still conform to the base class' operations. Maybe this weekend I'll be able to spend some time on whipping up some sort of prototype, and hopefully that will answer some of my open questions. It'll also be interesting to see if I can actually use the technique directly on existing interfaces and adaptation, i.e. get some degree of PyProtocols backward-compatibility. It might also be possible to get backward-compatibility for Zope too. In each case, the backward compatibility mechanism would be to change the adapter/interface declaration APIs to be equivalent to assertions about all the operations defined in a particular interface, against the concrete class you're claiming implements the interface. However, for both PEAK and Zope, it would likely be desirable to migrate any interfaces like mapping object to be based off of operations in e.g. the 'dict' type rather than rolling their own IReadMapping and such. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Son of PEP 246, redux
At 12:01 AM 1/13/05 -0500, Michael Walter wrote: What am I missing? The fact that this is a type-declaration issue, and has nothing to do with *how* types are checked. Note that I'm only proposing: 1) a possible replacement for PEP 246 that leaves 'adapt()' as a function, but uses a different internal implementation, 2) a very specific notion of what an operation is, that doesn't require an interface to exist if there is already some concrete type that the interface would be an abstraction of, 3) a strawman syntax for declaring the relationship between operations In other words, compared to the previous state of things, this should actually require *fewer* interfaces to accomplish the same use cases, and it doesn't require Python to have a built-in notion of interface, because the primitive notion is an operation, not an interface. Oh, and I think I've now figured out how to define a type-safe version of Ping's abstract operations concept that can play in the non-generic-function implementation, but I really need some sleep, so I might be hallucinating the solution. :) Anyway, so far it seems like it can all be done with a handful of decorators: @implements(base_operation, for_type=None) (for_type is the adapt *from* type, defaulting to the enclosing class if used inside a class body) @override (means the method is overriding the one in a base class, keeping the same operation correspondence(s) defined for the method in the base class) @abstract(base_operation, *required_operations) (indicates that this implementation of base_operation requires the ability to use the specified required_operations on a target instance. The adapter machinery can then safely fail if the operations aren't available, or if it detects a cycle between mutually-recursive abstract operations that don't have a non-abstract implementation. An abstract method can be used to perform the operation on any object that provides the required operations, however.) Anyway, from the information provided by these decorators, you can generate adapter classes for any operation-based interfaces. I don't have a planned syntax or API for defining attribute correspondences as yet, but it should be possible to treat them internally as a get/set/del operation triplet, and then just wrap them in a descriptor on the adapter class. By the way, saying generate makes it sound more complex than it is: just a subclass of 'object' with a single slot that points to the wrapped source object, and contains simple descriptors for each available operation of the protocol type that call the method implementations, passing in the wrapped object. So really generate means, populate a dictionary with descriptors and then call 'type(name,(object,),theDict)'. A side effect of this approach, by the way, is that since adapters are *never* composed (transitively or otherwise), we can *always* get back to the original object. So, in theory we could actually have 'adapt(x,object)' always convert back to the original unwrapped object, if we needed it. Likewise, adapting an already-adapted object can be safe because the adapter machinery knows when it's dealing with one of its own adapters, and unwrap it before rewrapping it with a new adapter. Oh, btw, it should at least produce a warning to declare multiple implementations for the same operation and source type, if not an outright error. Since there's no implicit transitivity in this system (either there's a registered implementation for something or there isn't), there's no other form of ambiguity besides dual declarations of a point-to-point adaptation. Hm. You know, this also solves the interface inheritance problem; under this scheme, if you inherit an operation from a base interface, it doesn't mean that you provide the base interface. Oh, actually, you can still also do interface adaptation in a somewhat more restrictive form; you can declare abstract operations for the target interface in terms of operations in the base interface. But it's much more controlled because you never stack adapters on adapters, and the system can tell at adaptation time what operations are and aren't actually available. Even more interesting: Alex's loss of middle name example can't be recreated in this system as a problem, at least if I'm still thinking clearly. But I'm probably not, so I'm going to bed now. :) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
