Overriding methods inherited from a superclass with methods from a mixin
Hello there. I'm trying to override methods inherited from a superclass by methods defined in a mixin class. Here's an sscce: https://bpaste.net/show/6c7d8d590658 (never expires) I've had problems finding the proper way to do that, since at first the base class wasn't to the right and I've assumed the correct order was from left to right. It was previously suggested on IRC that the mixin should be a subclass of "Base"; that worked but I wasn't happy with it because the B class basically serves the purpose of "holding" a list of methods to be inherited and that should override the methods inherited from the "A" classes, so I didn't see why it should derive from "Base". I eventually found in an article that the problem was the ordering of the superclasses I was deriving from, which should be from right to left, the only article I could find that states that is this one: https://www.ianlewis.org/en/mixins-and-python Every single example of mixins in Python that I've read -except that one- (and I've seen literally dozens) has the base class to the left, although the other classes aren't overriding any methods (at least in most of them). That bpaste code is working perfectly for me and makes sense, but I don't really like it, and the people on IRC couldn't convince me the code is fine. I haven't used Python for some time so I don't feel confident to judge that code, and perhaps there's a better way to achieve that result. However, what really scared me is the obscurity of the mixins usage in Python, and the fact that every example except that single one gets it "wrong", including from notable pythonistas. Perhaps you guys could help me either convincing me that the bpaste code is OK, or perhaps coming up with a better solution for that problem. What matters to me is code re-usability in this case. I surely could re-implement the overrides in all "Z" classes separately, but that's what I'm trying to avoid. The requirements are: 1. I can't touch the "classes I don't have control over" (as per comment in code). 2. I don't want to pass the superclasses as parameters in the constructor. I see how you could solve the problem that way, but that would only increase the complexity of the code (PEP20). 3. I absolutely need to override methodX, I can't use composition and access the members another way unless I override methodX and access them there. This is to interface properly with other modules. 4. I need to be able to access A#.methodX in the "Z" classes methods. 5. I want to avoid using a factory. It's one of the most over-used patterns in my opinion, and I really don't like that. Please note that the bpaste code is just an example. The real problem is much bigger and involves multiple methods to override and more classes, so the solution has to scale accordingly. Thank you in advance. -- https://mail.python.org/mailman/listinfo/python-list
Re: Overriding methods inherited from a superclass with methods from a mixin
Thank you for your replies. I don't know if I'm quoting you correctly, I'm
quite confused with Google Groups... not sure if it's a "forum", something like
a mailing list, or both... or neither.
> The class' MRO ("Method Resolution Order") determines in which
> order attributes are looked up.
> Either, you must order your base classes in such a way that the MRO
> controlled lookup finds the methods you want to be used or
> you must explicitely put a definition for those methods in your
> derived class (it may have the form "overridden_method =
> .overridden_method").
>
> The rules to determine the MRO are complex. The "inspect" module contains
> a function ("get_mro") to show the MRO of a given class. Use it
> to get your inheritance order right.
OK, I'm going to read about the MRO rules. Thanks for the heads up on the
inspect module.
> But the code is fine. Write unit tests to ensure that the correct method is
> called.
>
> Usually you use mixins to add methods. In that case the order of base
> classes doesn't matter.
Yeah, I see that in most cases the order doesn't matter, but still I would
think that since the correct order is from right to left, that should be the
common practice. What if later on you want to expand a mixin and actually
override a method? That could cause a situation that's a hell to debug, so
people should just get used to keeping their (most) "base" classes to the right
when doing multiple inheritance in my opinion.
> That looks like you are actively excluding most alternatives ;)
>
> Here's one option that should also work (though I'm not sure if that is what
> you mean with point 2 of your list):
>
> def subclass(base):
> class Z(base):
> def methodX(self):
> print "overridden methodX"
> return Z
>
> Z = subclass(A)
> Z2 = subclass(A2)
>
Yes, that violates the second item :). That would increase the complexity of
the code because I would need to know which A to use when instantiating the
classes, and what I'm doing is exactly to simplify the usage of those classes.
Of course some docs and perhaps a well tailored enum (or alike) to be passed as
the parameter could help, but a factory could do just as well, however, that
isn't going to fulfill my needs at all, and since I'm the one who's going to
use the code most of the time, I don't care much about using something somewhat
"unconventional".
> The details are complex, but there are two fairly simple principles
> that can be relied upon:
>
> 1) Subclasses always appear before their superclasses in the MRO.
> 2) When a class inherits from multiple base classes, they will always
> appear in the MRO in the order they appear in the class statement from
> left-to-right.
>
> Note that neither of these rules guarantee that the classes will
> appear *sequentially* in the MRO.
>
> So in the case of mixins, listing the mixin class first is absolutely correct:
>
> class Derived(Mixin, Base): pass
>
> This ensures that Mixin will always appear before Base in the MRO (and
> thus override Base's methods) for Derived and for any subclass of
> Derived.
>
> It is possible to concoct elaborate inheritance hierarchies in which
> it is not possible to come up with an MRO that will satisfy both 1)
> and 2) above. In such cases, it is also useful to know what Python
> will do. Fortunately, the answer to that is also simple: the type
> constructor will throw an exception. So this isn't something that
> really needs to be worried about.
That's some very nice information you got there. I also like how Ian Lewis
explained that, it may not be very technical and perhaps that "logic" is
accidental, but it surely helps you remember the correct order when doing
multiple inheritance. (That part about forgetting the brackets and thinking of
the code as a lined-up hierarchy, so MyClass => Mixin2 => Mixin1 => BaseClass).
In my opinion Python users should get used to that order. It doesn't matter if
*atm* you're not overriding, if that's the correct way to do it, you should get
used to it in my opinion because who know how you may need/want to expand your
code in the future. Still, pretty much *no one* uses that order. A quick Google
search returns (at least in my "bubble") many blog articles from "notable"
Python users with that order wrong.
As long as I can rely on that it's OK for me. I couldn't find anything
"official" on that subject though (mixins that override).
> Yet, you are clearly judging the code you pasted as not OK.
Actually, I didn't say (hard and fast) the code wasn't OK (if memory serves), I
said I didn't like it -on a first superficial analysis-. Especially due the
lack of recent experience with Python I was afraid I wasn't seeing some obvious
caveat, thus I've basically asked you guys to tell me if the code is or isn't
OK.
> It would be helpful for you to explain why you think the code you pasted is
> not OK. To me it makes sense. English reads
Re: Overriding methods inherited from a superclass with methods from a mixin
On Monday, June 13, 2016 at 7:29:05 PM UTC-3, Gregory Ewing wrote: > > I see that in most cases the order doesn't matter, but still I would > > think that since the correct order is from right to left, that should be the > > common practice. > > This order is only "correct" if overriding is what you want. > That's not always going to be the case. The mixin might be > intended to supply default functionality that can be > overridden by the classes it's being mixed into. You > can't say that one order is more correct than the other > in general. Yes, but (I presume) ordering it rtl doesn't have any caveat, and the code gets 'future-proof' in case you want to override a method in a class that's intended to be used as a mixin. I for instance am making an effort to get used to the rtl order as quickly as possible. But sure, in most cases it's probably never going to make a difference. > > Basically, I > > think of the mixins like plugins of kinds, I'm "adding" functionality to > > that > > "base" class I'm inheriting from. Having them to the left sounds like a > > sandwich recipe that tells you to "slice the ham, put mayonnaise on it, and > > put it on the bread". > > To me it's more like adding seasoning to a dish. Normally > you put the main ingredient in first, then add the salt > and pepper. This is probably the way most people are > thinking when they write the mixins after the main base > class. Exactly. On Monday, June 13, 2016 at 7:51:14 PM UTC-3, Michael Selik wrote: > > Thank you for your replies. I don't know if I'm quoting you correctly, I'm > > quite confused with Google Groups... not sure if it's a "forum", something > > like a mailing list, or both... or neither. > > > > Mailing list. A "forum" in the metaphorical sense, not the sense of phpBB. > You're doing fine with quoting, though it would help to add who said what > when quoting multiple people. > > > since the correct order is from right to left, > > > Did you mean left-to-right? > Are you thinking of Java where there's only one parent class allowed and > you specify any extra interfaces you're implementing afterwards? Because > that's not what your code is doing. Nope, I'm certainly not thinking of interfaces of abstract classes, I'm literally thinking about mixins. > > Still, pretty much *no one* uses that order [mixin1, mixin2, base]. A quick > > Google search returns (at least in my "bubble") many blog articles from > > "notable" Python users with that order wrong. > > > > Do you mind providing links? I haven't seen anyone "notable" make this > mistake. Not at all. Of course the snippets work because they're not overriding, but they all use an 'ltr' order. I feel somewhat bad about posting these links because I don't want to imply the code is wrong or something, since apparently there's no convention on that. So I'm publishing this list with all due respect to the programmers, and this only proves that it's probably more intuitive for most people to put is as [base,mixin]: class Request(BaseRequest, AcceptMixin, ETagRequestMixin, UserAgentMixin, AuthorizationMixin): http://stackoverflow.com/a/547714 class RealTestCase(BaseTestCase, MyMixin): http://nedbatchelder.com/blog/201210/multiple_inheritance_is_hard.html class TextBook(Book, IndexMixin): https://ahal.ca/blog/2014/when-would-you-use-python-mixin/ > > > To me it makes sense. English reads left-to-right, so method > > > > lookups go left-to-right (and children before parents) in the inheritance > > > list. > > > > Basically I wasn't very confident the code was OK because my intuition > > said the right way to order the classes I was inheriting from was from left > > to right. > > That is correct! Write them in the order you are prioritizing the > definitions. Overrides before bases, left to right. I'm sorry but it doesn't seem that obvious to me... It feels like building a house starting from the roof to me. Of course that's entirely subjective. > > I don't see any sense in that analogy you made... sure English and most > > occidental languages read ltr, but I fail to see how it makes more sense to > > have the base class on the right and the mixins on the left. Basically, I > > think of the mixins like plugins of kinds, I'm "adding" functionality to > > that "base" class I'm inheriting from. > > > That's not a good metaphor for what's actually happening. For more detail, > check out Raymond's "super considered super" video ( > https://www.youtube.com/watch?v=EiOglTERPEo). I'm going to take a look. Thank you for the link. > > Don't take my word for it, Google multi inheritance or mixin in Python and > > let me know if you find someone who thinks it's "natural" to do it that way. > > > This might be a bit of selection bias. People who think it makes sense > might not have an incentive to write a blog post about it. People who don't > like it will thus be more vocal. Yes, or maybe it's just my Google results that are biased. > Ian Lewi
