At 07:42 PM 1/10/05 +0100, Alex Martelli wrote:
On 2005 Jan 10, at 18:43, Phillip J. Eby wrote: ...At 03:42 PM 1/10/05 +0100, Alex Martelli wrote:The fourth case above is subtle. A break of substitutability can occur when a subclass changes a method's signature, or restricts the domains accepted for a method's argument ("co-variance" on arguments types), or extends the co-domain to include return values which the base class may never produce ("contra-variance" on return types). While compliance based on class inheritance _should_ be automatic, this proposal allows an object to signal that it is not compliant with a base class protocol.
-1 if this introduces a performance penalty to a wide range of adaptations (i.e. those using abstract base classes), just to support people who want to create deliberate Liskov violations. I personally don't think that we should pander to Liskov violators, especially since Guido seems to be saying that there will be some kind of interface objects available in future Pythons.
If interfaces can ensure against Liskov violations in instances of their subclasses, then they can follow the "case (a)" fast path, sure.
Inheriting from an interface (in Guido's current proposal, as per his Artima blog) is a serious commitment from the inheritor's part; inheriting from an ordinary type, in real-world current practice, need not be -- too many cases of assumed covariance, for example, are around in the wild, to leave NO recourse in such cases and just assume compliance.
I understand that, sure. But I don't understand why we should add complexity to PEP 246 to support not one but *two* bad practices: 1) implementing Liskov violations and 2) adapting to concrete classes. It is only if you are doing *both* of these that this extra feature is needed.
If it were to support some kind of backward compatibility, that would be understandable. However, in practice, I don't know of anybody using adapt(x,ConcreteClass), and even if they did, the person subclassing ConcreteClass will need to change their subclass to raise LiskovViolation, so why not just switch to delegation?
Anyway, it seems to me a bad idea to add complexity to support this case. Do you have a more specific example of a situation in which a Liskov violation coupled to concrete class adaptation is a good idea? Or am I missing something here?
I am not saying we shouldn't have a tp_conform; just suggesting that it may be appropriate for functions and modules (as well as classic classes) to have their tp_conform delegate back to self.__dict__['__conform__'] instead of a null implementation.
I have not considered conformance of such objects as functions or modules; if that is important,
It's used in at least Zope and PEAK; I don't know if it's in use in Twisted.
I need to add it to the reference implementation in the PEP. I'm reluctant to just get __conform__ from the object, though; it leads to all sort of issues with a *class* conforming vs its *instances*, etc. Maybe Guido can Pronounce a little on this sub-issue...
Actually, if you looked at the field-tested implementations of the old PEP 246, they actually have code that deals with this issue effectively, by recognizing TypeError when raised by attempting to invoke __adapt__ or __conform__ with the wrong number of arguments or argument types. (The traceback for such errors does not include a frame for the called method, versus a TypeError raised *within* the function, which does have such a frame. AFAIK, this technique should be compatible with any Python implementation that has traceback objects and does signature validation in its "native" code rather than in a new Python frame.)
I don't see the benefit of LiskovViolation, or of doing the exact type check vs. the loose check. What is the use case for these? Is it to allow subclasses to say, "Hey I'm not my superclass?" It's also a bit confusing to say that if the routines "raise any other exceptions" they're propagated. Are you saying that LiskovViolation is *not* propagated?
Indeed I am -- I thought that was very clearly expressed!
The PEP just said that it would be raised by __conform__ or __adapt__, not that it would be caught by adapt() or that it would be used to control the behavior in that way. Re-reading, I see that you do mention it much farther down. But at the point where __conform__ and __adapt__ are explained, it has not been explained that adapt() should catch the error or do anything special with it. It is simply implied by the "to prevent this default behavior" at the end of the section. If this approach is accepted, the description should be made explicit, becausse for me at least it required a retroactive re-interpretation of the earlier part of the spec.
The previous version treated TypeError specially, but I think (on the basis of just playing around a bit, admittedly) that offers no real added value and sometimes will hide bugs.
See http://peak.telecommunity.com/protocol_ref/node9.html for an analysis of the old PEP 246 TypeError behavior, and the changes made the by PyProtocols and Zope to deal with the situation better, while still respecting the fact that __conform__ and __adapt__ may be retrieved from the wrong "meta level" of descriptor.
Your new proposal does not actually fix this problem in the absence of tp_conform/tp_adapt slots; it merely substitutes possible confusion at the metaclass/class level for confusion at the class/instance level. The only way to actually fix this is to detect when you have called the wrong level, and that is what the PyProtocols and Zope implementations of "old PEP 246" do. (PyProtocols also introduces a special descriptor for methods defined on metaclasses, to help avoid creating this possible confusion in the first place, but that is a separate issue.)
If none of the first four mechanisms worked, as a last-ditch attempt, 'adapt' falls back to checking a registry of adapter factories, indexed by the protocol and the type of `obj', to meet the fifth case. Adapter factories may be dynamically registered and removed from that registry to provide "third party adaptation" of objects and protocols that have no knowledge of each other, in a way that is not invasive to either the object or the protocols.
This should either be fleshed out to a concrete proposal, or dropped.
There are many details that would need to be answered, such as whether "type" includes subtypes and whether it really means type or __class__. (Note that isinstance() now uses __class__, allowing proxy objects to lie about their class; the adaptation system should support this too, and both the Zope and PyProtocols interface systems and PyProtocols' generic functions support it.)
I disagree: I think the strawman-level proposal as fleshed out in the pep's reference implementation is far better than nothing.
I'm not proposing to flesh out the functionality, just the specification; it should not be necessary to read the reference implementation and try to infer intent from it. What part is implementation accident, and what is supposed to be the specification? That's all I'm talking about here. As currently written, the proposal is just, "we should have a registry", and is not precise enough to allow someone to implement it based strictly on the specification.
I mention the issue of subtypes explicitly later, including why the pep does NOT do anything special with them -- the reference implementation deals with specific types. And I use type(X) consistently, explicitly mentioning in the reference implementation that old-style classes are not covered.
As a practical matter, classic classes exist and are useful, and PEP 246 implementations already exist that work with them. Dropping that functionality is a major step backward for PEP 246, IMO.
I didn't know about the "let the object lie" quirk in isinstance. If that quirk is indeed an intended design feature,
It is; it's in one of the "what's new" feature highlights for either 2.3 or 2.4, I forget which. It was intended to allow proxy objects (like security proxies in Zope 3) to pretend to be an instance of the class they are proxying.
One other issue: it's not possible to have standalone interoperable PEP 246 implementations using a registry, unless there's a standardized place to put it, and a specification for how it gets there. Otherwise, if someone is using both say Zope and PEAK in the same application, they would have to take care to register adaptations in both places. This is actually a pretty minor issue since in practice both frameworks' interfaces handle adaptation, so there is no *need* for this extra registry in such cases.
I'm not sure I understand this issue, so I'm sure glad it's "pretty minor".
All I was saying is that if you have two 'adapt()' implementations, each using its own registry, you have a possible interoperability problem. Two 'adapt()' implementations that conform strictly to PEP 246 *without* a registry are interoperable because their behavior is the same.
As a practical matter, all this means is that standalone PEP 246 implementations for older versions of Python either shouldn't implement a registry, or they need to have a standard place to put it that they can share with each other, and it needs to be implemented the same way. (This is one reason I think the registry specification should be more formal; it may be necessary for existing PEP 246 implementations to be forward-compatible with the spec as implemented in later Python versions.)
The issue isn't that adaptation isn't casting; why would casting a string to a file mean that you should open that filename?
Because, in most contexts, "casting" object X to type Y means calling Y(X).
Ah; I had not seen that called "casting" in Python, at least not to my immediate recollection. However, if that is what you mean, then why not say it? :)
Maybe we're using different definitions of "casting"?
I'm most accustomed to the C and Java definitions of casting, so that's probably why I can't see how it relates at all. :)
If I were going to say anything about that case, I'd say that adaptation should not be "lossy"; adapting from a designator to a file loses information like what mode the file should be opened in.
(Similarly, I don't see adapting from float to int; if you want a cast to int, cast it.) Or to put it another way, adaptability should imply substitutability: a string may be used as a filename, a filename may be used to designate a file. But a filename cannot be used as a file; that makes no sense.
I don't understand this "other way" -- nor, to be honest, what you "would say" earlier, either. I think it's pretty normal for adaptation to be "lossy" -- to rely on some but not all of the information in the original object: that's the "facade" design pattern, after all. It doesn't mean that some info in the original object is lost forever, since the original object need not be altered; it just means that not ALL of the info that's in the original object used in the adapter -- and, what's wrong with that?!
I think we're using different definitions of "lossy", too. I mean that defining an adaptation relationship between two types when there is more than one "sensible" way to get from one to the other is "lossy" of semantics/user choice. If I have a file designator (such as a filename), I can choose how to open it. If I adapt directly from string to file by way of filename, I lose this choice (it is "lossy" adaptation).
Here's a better way of phrasing it (I hope): adaptation should be unambiguous. There should only be one sensible way to interpret a thing as implementing a particular interface, otherwise, adaptation itself has no meaning. Whether an adaptation adds or subtracts behavior, it does not really change the underlying *intended* meaning of a thing, or else it is not really adaptation. Adapting 12.0 to 12 does not change the meaning of the value, but adapting from 12.1 to 12 does.
Does that make more sense? I think that some people start using adaptation and want to use it for all kinds of crazy things because it seems cool. However, it takes a while to see that adaptation is just about removing unnecessary accidents-of-incompatibility; it's not a license to transform arbitrary things into arbitrary things. There has to be some *meaning* to a particular adaptation, or the whole concept rapidly degenerates into an undifferentiated mess.
(Or else, you decide to "fix" it by disallowing transitive adaptation, which IMO is like cutting off your hand because it hurts when you punch a brick wall. Stop punching brick walls (i.e. using semantic-lossy adaptations), and the problem goes away. But I realize that I'm in the minority here with regards to this opinion.)
For example, say that I have some immutable "record" types. One, type Person, defined in some framework X, has a huge lot of immutable data fields, including firstName, middleName, lastName, and many, many others. Another, type Employee, defines in some separate framework Y (that has no knowlege of X, and viceversa), has fewer data fields, and in particular one called 'fullName' which is supposed to be a string such as 'Firstname M. Lastname'. I would like to register an adapter factory from type Person to protocol Employeee. Since we said Person has many more data fields, adaptation will be "lossy" -- it will look upon Employee essentially as a "facade" (a simplified-interface) for Person.
But it doesn't change the *meaning*. I realize that "meaning" is not an easy concept to pin down into a nice formal definition. I'm just saying that adaptation is about semantics-preserving transformations, otherwise you could just tack an arbitrary object on to something and call it an adapter. Adapters should be about exposing an object's *existing semantics* in terms of a different interface, whether the interface is a subset or superset of the original object's interface. However, they should not add or remove arbitrary semantics that are not part of the difference in interfaces.
For example, adding a "current position" to a string to get a StringIO is a difference that is reflected in the difference in interface: a StringIO *is* just a string of characters with a current position that can be used in place of slicing.
But interpreting a string as a *file* doesn't make sense because of added semantics that have to be "made up", and are not merely interpreting the string's semantics "as a" file. I suppose you could say that this is "noisy" adaptation rather than "lossy". That is, to treat a string as a file by using it as a filename, you have to make up things that aren't present in the string. (Versus the StringIO, where there's a sensible interpretation of a string "as a" StringIO.)
IOW, adaptation is all about "as a" relationships from concrete objects to abstract roles, and between abstract roles. Although one may colloquially speak of using a screwdriver "as a" hammer, this is not the case in adaptation. One may use a screwdriver "as a" pounder-of-nails. The difference is that a hammer might also be usable "as a" remover-of-nails. Therefore, there is no general "as a" relationship between pounder-of-nails and remover-of-nails, even though a hammer is usable "as" either one. Thus, it does not make sense to say that a screwdriver is usable "as a" hammer, because this would imply it's also usable to remove nails.
This is why I don't believe it makes sense in the general case to adapt to concrete classes; such classes usually have many roles where they are usable. I think the main difference in your position and mine is that I think one should adapt primarily to interfaces, and interface-to-interface adaptation should be reserved for non-lossy, non-noisy adapters. Where if I understand the opposing position correctly, it is instead that one should avoid transitivity so that loss and noise do not accumulate too badly.
So, can you please explain your objections to what I said about adapting vs casting in terms of this example? Do you think the example, or some variation thereof, should go in the PEP?
I'm not sure I see how that helps. I think it might be more useful to say that adaptation is not *conversion*, which is not the same thing (IME) as casting. Casting in C and Java does not actually "convert" anything; it simply treats a value or object as if it were of a different type. ISTM that bringing casting into the terminology just complicates the picture, because e.g. casting in Java actually corresponds to the subset of PEP 246 adaptation for cases where adapt() returns the original object or raises an error. (That is, if adapt() could only ever return the original object or raise an error, it would be precisely equivalent to Java casting, if I understand it correctly.) Thus, at least with regard to object casting in Java, adaptation is a superset, and saying that it's not casting is just confusing.
Reference Implementation and Test Cases
The following reference implementation does not deal with classic classes: it consider only new-style classes. If classic classes need to be supported, the additions should be pretty clear, though a bit messy (x.__class__ vs type(x), getting boundmethods directly from the object rather than from the type, and so on).
Please base a reference implementation off of either Zope or PyProtocols' field-tested implementations which deal correctly with __class__ vs. type(), and can detect whether they're calling a __conform__ or __adapt__ at the wrong metaclass level, etc. Then, if there is a reasonable use case for LiskovViolation and the new type checking rules that justifies adding them, let's do so.
I think that if a PEP includes a reference implementation, it should be self-contained rather than require some other huge package. If you can critique specific problems in the reference implementation, I'll be very grateful and eager to correct them.
Sure, I've got some above (e.g. your implementation will raise a spurious TypeError if it calls an __adapt__ or __conform__ at the wrong metaclass level, and getting them from the type does *not* fix this issue, it just bumps it up by one metalevel). I wasn't proposing you pull in either whole package, though; just adapt() itself. Here's the existing Python one from PyProtocols (there's also a more low-level one using the Python/C API, but it's probably not appropriate for the spec):
from sys import exc_info from types import ClassType ClassTypes = type, ClassType
def adapt(obj, protocol, default=_marker):
"""PEP 246-alike: Adapt 'obj' to 'protocol', return 'default'
If 'default' is not supplied and no implementation is found,
the result of 'factory(obj,protocol)' is returned. If 'factory'
is also not supplied, 'NotImplementedError' is then raised.""" if isinstance(protocol,ClassTypes) and isinstance(obj,protocol):
return obj try:
_conform = obj.__conform__
except AttributeError:
pass
else:
try:
result = _conform(protocol)
if result is not None:
return result
except TypeError:
if exc_info()[2].tb_next is not None:
raise
try:
_adapt = protocol.__adapt__
except AttributeError:
pass
else:
try:
result = _adapt(obj)
if result is not None:
return result
except TypeError:
if exc_info()[2].tb_next is not None:
raise if default is _marker:
raise AdaptationFailure("Can't adapt", obj, protocol)return default
Obviously, some changes would need to be made to implement your newly proposed functionality, but this one does support classic classes, modules, and functions, and it has neither the TypeError-hiding problem of the original PEP 246 nor the TypeError-raising problem of your new version.
Transitivity of adaptation is in fact somewhat controversial, as is the relationship (if any) between adaptation and inheritance.
The issue is simply this: what is substitutability? If you say that interface B is substitutable for A, and C is substitutable for B, then C *must* be substitutable for A, or we have inadequately defined "substitutability".
Not necessarily, depending on the pragmatics involved.
In that case, I generally prefer to be explicit and use conversion rather than using adaptation. For example, if I really mean to truncate the fractional part of a number, I believe it's then appropriate to use 'int(someNumber)' and make it clear that I'm intentionally using a lossy conversion rather than simply treating a number "as an" integer without changing its meaning.
Thanks, BTW, for your highly detailed feedback.
No problem; talking this out helps me clarify my own thoughts on these matters. I haven't had much occasion to clarify these matters, and when they come up, it's usually in the context of arguing some specific inappropriate use of adaptation, so I can easily present an alternative that makes sense in that context. This discussion is helping me clarify the general principle, since I have to try to argue the general case, not just N specific cases. :)
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
