On Mon, Jul 26, 2010 at 3:36 PM, Nicolas M. Thiery <nicolas.thi...@u-psud.fr> wrote: > William, > > <grumpy o' pa> > There is a strong technical issue that needs to be resolved, and for > which I explicitly take the blame. And I understand your frustration > when fighting with zillions of technical points. However I perceived > your e-mail as unfairly bashing the sage-combinat community, which > is just putting me off. Now, I should just step back and answer in a > couple hours, but on the other hand this discussion needs to get > going on the technical side. > > Deep breath. > </grumpy o' pa>
<humble servant> I'm very sorry to have unfairly bashed the sage-combinat community, and I'm glad you've jumped to their defense to clarify matters. I was exhausted after a very long, sleepless coding sprint (days and days) trying to speed things up, and very surprised to discover this source of a speed issue. My email was certainly very unfairly rewritten. I hope you'll accept my apology. </humble servant> > > On Mon, Jul 26, 2010 at 11:53:59AM -0700, William Stein wrote: >> I'm talking about the __getattr__ that you added. Simply commenting >> out the __getattr__ function leads to a significant speedup. This >> has been replicated by several other people, and this thread >> contains a deep investigation into why this is happening (though >> things still aren't clear). > > Then, do you have an explanation for why this does not show up above? > Can you reproduce my timings? > >> >> A few months ago some code was added to Sage by the combinat folks >> >> called "categories". I recently learned ObjectiveC, and found out >> >> that this idea of "categories" goes back to smalltalk at least, and >> >> was fully implemented in ObjectiveC in the mid 1980s. It has little >> >> to do with category theory in the sense of mathematics, though is >> >> related. Basically, it is a way to attach a bunch of generic >> >> methods to a class without using inheritance. >> > >> > That's a very reductive view. And actually quite inaccurate: most of >> > Sage's parents and elements *do* get their generic methods from the >> > categories, through class inheritance. There is indeed one exception: >> > extension types (i.e. Cython classes), which currently use the getattr >> > hack (because Cython does not support yet any form of multiple >> > inheritance). >> >> Do you know anything about Objective C or smalltalk or "categories" in >> those languages? > > I haven't played with them directly. But from looking around this very > much looks like what "extends" do in other languages I played with > (Aldor, ...). And which I would love to have in Python as well (I am > actually planning to make a crude implementation of it, for plain > Python classes, for use with my external experimental code; it should > just be a couple lines of code). > > So yes, the getattr hack is some sort of dynamic implementation of > categories (in smalltalk sense), which is a building block for the > implementation of "real" categories (in the Sage sense) for extension > types. > >> I don't really care about blame; I just want to find a way to >> resolve this problem. I've slowed down/messed up etc., so many >> things in Sage over the years, it isn't even funny. > > <grumpy o' pa> > I am glad to hear that, because that did not show up in your e-mail. > </grumpy o' pa> > >> I think the impact depends a lot on how many python def'd methods the >> class has. E.g., I was testing with the huge number field classes, >> and the slowdown is much greater than with testing with a naked class >> that just derives from Element. I did not notice any real difference >> between otherwise identical Cython and Python classes. > > I just had a look at the Python sources (in Object/typeobject.c), and > it's all about the difference between slot_tp_getattro and > slot_tp_getattr_hook, the later being a bit more complicated than the > former. I did not see obvious reasons for having it slower for large > classes, except for the lookup of the __getattr__ method. That makes a lot of sense. > From > glancing at it, it sounds like this could be optimized, and in > particular that the getattr lookup could be delayed until it is really > used (if at all). This does not tell anything about Cython I guess. > > Also it sounds like __getattr__ could be set to None if it is not > really needed, and then on the next call getattro_hook would switch > tp_getattro to the fast simple implementation slot_tp_getattroback. At > least this could solve the issue for Python classes. I'll give it a > shot. Thanks!! > >> > 2 Since only extension types need the getattr hack, make sure that >> > __getattr__ is only defined for those. I would love it. Anyone a >> > good idea on how to achieve this? The problem is that all parents >> > and elements classes derive from Parent and Element, which are the >> > natural places to put the getattr hack. >> >> I'm dubious that this is not an issue for Cython classes. > > Could someone lookup the sources to know this for sure? Maybe I'm just wrong. > >> > 3 Implement partial multiple inheritance in Cython >> > >> > We discussed Option 3 quite some with Robert during the categories >> > brainstorm in Nancy (October 2008). As far as I remember from the >> > discussions, full multiple inheritance (including merging of data >> > structure and method lookup table) was impossible; on the other hand, >> > there was no technical obstructions for allowing a Cython class to >> > inherit from multiple purely abstract classes besides the main base >> > class branch, if one accepts the price of a virtual lookup for the >> > methods defined there. Essentially, this would boil down to >> > implementing a variant of the getattr trick, but within the Cython >> > compiler, and with full control so as to make sure that standard >> > attribute and method lookups would not be affected. > >> That sounds like implementing what is called "categories" in >> ObjectiveC. ObjectiveC is an extension to C that was done in the >> late 1980s to add features like one has in Smalltalk. It's built on >> C structs, like Cython is, and does *not* have multiple inheritance. >> Categories make up for this. > > Yeah, with the extra thing that this would have to be done > dynamically, and for parents possibly on a parent by parent basis, > since in many cases one does not know statically the category. > > Cheers, > Nicolas > -- > Nicolas M. Thiéry "Isil" <nthi...@users.sf.net> > http://Nicolas.Thiery.name/ > > -- > You received this message because you are subscribed to the Google Groups > "sage-combinat-devel" group. > To post to this group, send email to sage-combinat-de...@googlegroups.com. > To unsubscribe from this group, send email to > sage-combinat-devel+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/sage-combinat-devel?hl=en. > > -- William Stein Professor of Mathematics University of Washington http://wstein.org -- You received this message because you are subscribed to the Google Groups "sage-combinat-devel" group. To post to this group, send email to sage-combinat-de...@googlegroups.com. To unsubscribe from this group, send email to sage-combinat-devel+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sage-combinat-devel?hl=en.
