Re: [Zope-dev] New-style ExtensionClasses (Zope 2.8) -- MRO issue
On Sat, Nov 01, 2003 at 12:18:47PM +0100, Stefan H. Holek wrote: | I am also very, very worried about breaking b/w compatibility in Zope 2. | | I am responsible for about 15 sites, with say 10 distinct products | each. Are you saying I have to evaluate/upgrade 150 products because I | want to go to Zope 2.8(9)? No customer is going to pay for that effort | (and I will likely not find the time to do it anyway) so I will be | locked in 2.7-land forever. Bah! | | So yes, option 3 it must be. Let's save the fundamental changes for | Zope 3. As far as I understand, even if we dont go with option 2 there will be *some* effort on migrating from 2.7 to 2.8. Adding the update of a few products may not be as bad as you painted here. I've analised the changes needed to make the current Plone from CVS to work with option 2 and they are not that intrusive. Its basically change the order of one or two base classes and nothing more than that. Now I have a interesting question to ask: *If* we go for option 3, and we fix some products to work with option 2, would that cause any incompatibility? -- Sidnei da Silva <[EMAIL PROTECTED]> dreamcatching :: making your dreams come true http://awkly.org The first version always gets thrown away. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] New-style ExtensionClasses (Zope 2.8) -- MRO issue
I am also very, very worried about breaking b/w compatibility in Zope 2. I am responsible for about 15 sites, with say 10 distinct products each. Are you saying I have to evaluate/upgrade 150 products because I want to go to Zope 2.8(9)? No customer is going to pay for that effort (and I will likely not find the time to do it anyway) so I will be locked in 2.7-land forever. Bah! So yes, option 3 it must be. Let's save the fundamental changes for Zope 3. IMHO, Stefan On Freitag, Okt 31, 2003, at 18:14 Europe/Vienna, Jim Fulton wrote: Thoughts? I am worried enough about breaking products that I'm inclined to go with option 3. Does anybody think we ought to use the new algorithm (option 2)? Jim -- The time has come to start talking about whether the emperor is as well dressed as we are supposed to think he is. /Pete McBreen/ ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] New-style ExtensionClasses (Zope 2.8) -- MRO issue
Jim Fulton wrote: 3. Use a hybrid schema. I'll call this the "encapsulated base" scheme. Is it possible for the hybrid schema to generate a 'deprecation' warning for each instance of a class that doesn't match the requirements of the C3 resolution order when Zope is first started, and thus increase the chances that product authors will modify their products (or be coerced to do so :)), without actually breaking them? --Richard ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] New-style ExtensionClasses (Zope 2.8) -- MRO issue
Jim Fulton wrote at 2003-10-31 12:14 -0500: > ... > Thoughts? > > I am worried enough about breaking products that I'm inclined to go > with option 3. > > Does anybody think we ought to use the new algorithm (option 2)? Please do not use the new algorithm (option 2). I like (multiple) inheritance and often used it to the extreme. I fear lots of my products will break when there are class definitions which fails due to MRO problems. -- Dieter ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] New-style ExtensionClasses (Zope 2.8) -- MRO issue
On Fri, Oct 31, 2003 at 02:46:51PM -0500, Jim Fulton wrote: | >Most of the time, this is the same people who can't update their | >products, so if they *do* want a new zope, they will need to get a new | >product as well. I think its safe to make that assumption. | | I don't follow this. What do you mean be "Can't update their products"? | Are you saying that to upgrade to a new Zope, they should actually have to | replace the products they are using with different ones? I meant, people that 'are afraid' of updating their existing products to new versions, or just can't because they would break compatibility. | >Considering that 2.8 may be at least 6-8 months in the future (is that | >a nice assumption?) | | I hope that 2.8 will come out much sooner than that. I hope to see 2.9 | in that timeframe. Remember that 2.8 will have just nsEC, ZODB 3.3, and | any other stuff that's piled up in the interim. :) Yay! Thats very nice to hear. | >Now *that* is a good point for going with the new algorithm. We don't | >want to have even another round later on to make it compatible with | >Zope 3. If we could do it at the same time it would be a | >bonus. Though, from what I understand, your custom algorithm would | >provide that. | | Huh? That doesn't make sense. As long as people use EC, they will use | whatever algorthm we pick now. If people use ExtensionClasses in Zope 3, | then those classes will use the algorithm we pick now. People would | switch to the new algorithm when they convert (or mor likely rewrite) | their products to be ExtensionClass-less and Zope3-ish. If they | make their products Zope 3-ish, the lookup algorithm won't matter much. Ok, I got it wrong then. Sorry. -- Sidnei da Silva <[EMAIL PROTECTED]> dreamcatching :: making your dreams come true http://awkly.org The value of a program is proportional to the weight of its output. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] New-style ExtensionClasses (Zope 2.8) -- MRO issue
Sidnei da Silva wrote: On Fri, Oct 31, 2003 at 12:14:27PM -0500, Jim Fulton wrote: | Thoughts? | | I am worried enough about breaking products that I'm inclined to go | with option 3. | | Does anybody think we ought to use the new algorithm (option 2)? I'm for option 2. Given that a huge amount of software that is based on CMF is in the Collective these days, we could probably convert all of them in a bug day, and in the same sweep, remove the ones that are not being used anymore. The same is true for Plone. Before you commit to this, perhaps you should try figuring out how to get some products working on the mro-advanture branch. (See my other note.) Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (540) 361-1714http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] New-style ExtensionClasses (Zope 2.8) -- MRO issue
Sidnei da Silva wrote: | Ohkay. If anyone has a problem, I'll send them to you and Sidnei. :) /me takes his iBook and a plane to The Amazon | Note that, in many cases, the people having problems will *not* be the | product authors. They'll be people who just want to upgrade thier Zope | installationa that use third-party products. Most of the time, this is the same people who can't update their products, so if they *do* want a new zope, they will need to get a new product as well. I think its safe to make that assumption. I don't follow this. What do you mean be "Can't update their products"? Are you saying that to upgrade to a new Zope, they should actually have to replace the products they are using with different ones? .. | > as opposed to a | >short-term problem of updating existing code. | | It won't be that short term. It will come up each time someone wants to | use a product that hasn't been converted before. Considering that 2.8 may be at least 6-8 months in the future (is that a nice assumption?) I hope that 2.8 will come out much sooner than that. I hope to see 2.9 in that timeframe. Remember that 2.8 will have just nsEC, ZODB 3.3, and any other stuff that's piled up in the interim. :) ... | I don't think it makes the code any clearer. Zope 2 has an extremely | complex inheritence graph. Changing the mro algorithm won't change that. | | Zope 3, of course, makes much less use of inheritence and has clearer | code. It also uses new-style classes and, thus, uses the new algorithm. Now *that* is a good point for going with the new algorithm. We don't want to have even another round later on to make it compatible with Zope 3. If we could do it at the same time it would be a bonus. Though, from what I understand, your custom algorithm would provide that. Huh? That doesn't make sense. As long as people use EC, they will use whatever algorthm we pick now. If people use ExtensionClasses in Zope 3, then those classes will use the algorithm we pick now. People would switch to the new algorithm when they convert (or mor likely rewrite) their products to be ExtensionClass-less and Zope3-ish. If they make their products Zope 3-ish, the lookup algorithm won't matter much. Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (540) 361-1714http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] New-style ExtensionClasses (Zope 2.8) -- MRO issue
On Fri, 2003-10-31 at 13:52, Sidnei da Silva wrote: > | I personally don't like the new algorithm, but I don't really care > | in the long run. One should avoid inheritence complex enough to show > | a difference. > > I hearthly agree here. Home domestic of you . I do like the new algorithm, but we can all agree to avoid complex uses of inheritance. Jeremy ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] New-style ExtensionClasses (Zope 2.8) -- MRO issue
| Ohkay. If anyone has a problem, I'll send them to you and Sidnei. :) /me takes his iBook and a plane to The Amazon | Note that, in many cases, the people having problems will *not* be the | product authors. They'll be people who just want to upgrade thier Zope | installationa that use third-party products. Most of the time, this is the same people who can't update their products, so if they *do* want a new zope, they will need to get a new product as well. I think its safe to make that assumption. | Well, except that they are the same rules used for classic classes, which | probably still predominate in the Python world. That may be worth considering. | > as opposed to a | >short-term problem of updating existing code. | | It won't be that short term. It will come up each time someone wants to | use a product that hasn't been converted before. Considering that 2.8 may be at least 6-8 months in the future (is that a nice assumption?) I would consider that a just-long-enough term. | I don't think it makes the code any clearer. Zope 2 has an extremely | complex inheritence graph. Changing the mro algorithm won't change that. | | Zope 3, of course, makes much less use of inheritence and has clearer | code. It also uses new-style classes and, thus, uses the new algorithm. Now *that* is a good point for going with the new algorithm. We don't want to have even another round later on to make it compatible with Zope 3. If we could do it at the same time it would be a bonus. Though, from what I understand, your custom algorithm would provide that. | I personally don't like the new algorithm, but I don't really care | in the long run. One should avoid inheritence complex enough to show | a difference. I hearthly agree here. -- Sidnei da Silva <[EMAIL PROTECTED]> dreamcatching :: making your dreams come true http://awkly.org The trouble with computers is that they do what you tell them, not what you want. -- D. Cohen ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] New-style ExtensionClasses (Zope 2.8) -- MRO issue
Jeremy Hylton wrote: I am worried enough about breaking products that I'm inclined to go with option 3. Does anybody think we ought to use the new algorithm (option 2)? I think we should use the new algorithm. However, I don't have any products that I would be responsible for updating and maintaining. I'd be happy to help people work out problems, though. Ohkay. If anyone has a problem, I'll send them to you and Sidnei. :) Note that, in many cases, the people having problems will *not* be the product authors. They'll be people who just want to upgrade thier Zope installationa that use third-party products. Oh, and as you help people fix these products, you'll want to redistribute the fixed products so as to avoid having to fix them over and over. ... If we stick with a non-standard MRO, then Zope programmers will have to learn special rules for dealing with multiple inheritance with Zope code that don't apply to Python programming in general. Well, except that they are the same rules used for classic classes, which probably still predominate in the Python world. > I don't see much value in creating such a special case here. We'd also need to implement our own MRO, maintain it, make sure it's correct, etc. That's a long-term maintenance burden, That's not a big deal. Options 1 and 3 both have simple algorithms that are easy to implement and won't really need much maintanance. Fortunately, type actually provides a hook for overriding the mro computation that makes this easy. > as opposed to a short-term problem of updating existing code. It won't be that short term. It will come up each time someone wants to use a product that hasn't been converted before. > We can write tools to help people update their code if it ends up being a real problem, I think you are underestimating the complexity of the analysis. > and they get clearer code as an added bonus. I don't think it makes the code any clearer. Zope 2 has an extremely complex inheritence graph. Changing the mro algorithm won't change that. Zope 3, of course, makes much less use of inheritence and has clearer code. It also uses new-style classes and, thus, uses the new algorithm. I personally don't like the new algorithm, but I don't really care in the long run. One should avoid inheritence complex enough to show a difference. Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (540) 361-1714http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] New-style ExtensionClasses (Zope 2.8) -- MRO issue
> I am worried enough about breaking products that I'm inclined to go > with option 3. > > Does anybody think we ought to use the new algorithm (option 2)? I think we should use the new algorithm. However, I don't have any products that I would be responsible for updating and maintaining. I'd be happy to help people work out problems, though. If we were starting from scratch, the C3 algorithm would be the obvious choice. The Dylan linearization paper makes a pretty good argument for it: The first reason for the change is that a monotonic linearization seems a better match for users' intuitions about how inheritance and linearizations work. With a monotonic linearization, it is easier to understand the behavior of classes when multiple inheritance is used, largely because behavior of instances of a class can be explained in terms of the direct superclasses. The depth-first resolution order can produce un-intuitive results because it doesn't honor local precedence order. It also leads to sloppy code where the same base class is mixed into the inheritance hierarchy at multiple levels. If we stick with a non-standard MRO, then Zope programmers will have to learn special rules for dealing with multiple inheritance with Zope code that don't apply to Python programming in general. I don't see much value in creating such a special case here. We'd also need to implement our own MRO, maintain it, make sure it's correct, etc. That's a long-term maintenance burden, as opposed to a short-term problem of updating existing code. We can write tools to help people update their code if it ends up being a real problem, and they get clearer code as an added bonus. Jeremy ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] New-style ExtensionClasses (Zope 2.8) -- MRO issue
On Fri, Oct 31, 2003 at 12:14:27PM -0500, Jim Fulton wrote: | Thoughts? | | I am worried enough about breaking products that I'm inclined to go | with option 3. | | Does anybody think we ought to use the new algorithm (option 2)? I'm for option 2. Given that a huge amount of software that is based on CMF is in the Collective these days, we could probably convert all of them in a bug day, and in the same sweep, remove the ones that are not being used anymore. The same is true for Plone. -- Sidnei da Silva <[EMAIL PROTECTED]> dreamcatching :: making your dreams come true http://awkly.org My apologies if I sound angry. I feel like I'm talking to a void. -- Avery Pennarun ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] New-style ExtensionClasses (Zope 2.8) -- MRO issue
I've been working on a "new-style" ExtensionClass (nsEC) project to reimplement ExtensionClass as a new-style meta class. This will allow ExtensionClasses (i.e. most Zope 2 classes) to be able to use features of new-style Python classes, including: - New protocols, - Descriptors, - Garbage Collection, ... I think I've got this largely working, but have encountered a significant issue that I'd like some input on. New-style classes and classic classes use different algorithms for looking up inherited attributes (e.g. methods). Classic classes use a left-to-right depth-first algorithm. Python 2.3 uses a "C3 Method-Resolution Order" algorithm. I won't try to explain how this algorithm works. If you are curious, see: http://www.python.org/2.3/mro.html This new algorithm has a couple of noteworthy properties: - It yields different lookup results than the left-to-right depth-first algorithm used for classic classes. - It is sometimes impossible to compute a "method resolution order" for a given class and base classes. >>> class A(object): ... pass >>> class B(A): ... pass >>> class C(A, B): ... pass Traceback (most recent call last): File "", line 1, in ? TypeError: Cannot create a consistent method resolution order (MRO) for bases B, A The original ExtensionClass used the left-to-right depth-first algorithm used for classic classes. Currently, nsEC inherits the new method lookup algorithm from type (the standard Python 2.3 metaclass used for new-style classes). This has run into significant difficulties with the Zope 2 classes, which have a highly complex inheritence graph. An initial analysis showed that for more than half of the classes in Zope 2, it was impossible to compute a method-resolution order using the new algorithm. After about a day of analysis (and development of a tool to aid in the analysis) I've been able to adjust the Zope classes so that they will work with the new algorithm. I also analyzed the CMF head. To use the new algorithm with the CMF, it was necessary to make some changes to the CMF and to Zope. In one case, it was necessary to copy some methods around. An interesting result of the analysis is that it identified several methods that were being inherited incorrectly. The new inheritence algorithm yeilded better results for these cases, once the classes were modified enough to use the algorithm at all. It is very likely that, if we use the new algorithm for nsEC, many products will not work anymore (meaning with Zope 2.8). Modifying the products to work will probably be a non-trivial undertaking. We have three alternatives: 1. Use the old method-lookup algorithm for nsEC. Pro: backward compatible Con: may produce odd results if a new-style class with a complex arrangement of base classes is mixed with an ExtensionClass. 2. Use the new-style method-lookup algithm Pro: Most consistent with new-style classes. Cons: Likely that many products will be broken and require change to work with nsEC/Zope 2.8. Analysis of the changes needed to fix method-resolution-order problems is extremely complex. 3. Use a hybrid schema. I'll call this the "encapsulated base" scheme. Consider: >>> class C(X, Y, Z): ...def foo(self): ... ...... When we look up an attribute, we do the following: - Look in C's dictionary first. All three algorithms agree on this. :) - Look up the attribute in X. We don't care how we het the attribute from X. If X is a new-style-class, we use the new algithm. If X is a classic class, we use left-to-right depth-first. If X is an nsEC, use the "encapsulated base" algithm. If we don't find the attribute in X, look in Y and then in Z, using the same approach. This algithm will produce backward compatible results, providing the equivalent of left-to-right depth-first for nsECs and classic classes. (Note for the more advanced reader: we'll actually do something less abstract. We'll use a simple algorthm to merge the __mro__ of the base classes, computing an __mro__ for classes classes using the left-to-right depth-first algorithm. We'll basically lay the mros end-to-end left-to-right and remove repeats, keeping the first occurence of each class.) I find this approach appealing for reasons of encapsulation. When reasoning about C, I can reason from known behavior of X, Y, and Z, without considering their respective inherietnce hierarchies, which I consider to be an implementation detail. Pros: backward compatible Less likely than option 1 to produce odd results if a new-style class with a complex arrangement of base classes is mixed with an ExtensionClass. Con: may produce odd results if a new-style class with a complex arrangement of base classes is mixed with an ExtensionClass.
