On Mar 8, 2008, at 4:52 PM, Dag Sverre Seljebotn wrote: >> It would be nice to figure out a way that pluggins could be written >> to be less brittle (i.e. not so dependent on the inner workings and >> names of the Cython compiler) too, eventually. > > Totally agreed (and I've written about that too some places, > especially > that the -T switch will only ever be a developer tool). It's > important for > me that this transform framework isn't as much a plugin > architecture as a > change in the modularity of the compiler core itself.
This is a worthy goal. > Currently (if I dare) the ambitions seems to be too high for the > current code architecture > and (in all modesty :-) ) I believe this approach goes a long way > solving > that problem. Yep. > So this is the way I see it (and I'd guess you agree but I want it > on the > mailing list): > - In an ideal world, it is simply wrong to have the methods acting > on the > parse tree implemented as methods on the parse tree itself. (See > example > below). Rather you want the parse tree to be "data" and seperate, > componentized and well-defined transforms to be the vehicles of > making the > compilation happen. So ideally, the parse tree shouldn't be that > brittle > because there should be "nothing there" but information that is > spec-ed > according to the phases one is in. (However for plugin use it is > definetely too complex still). > > - Working towards such an ideal code world is not the business one > is in > and is also very boring, so one makes compromises: Whenever new > functionality is introduced, write it as transforms, whenever > something > new is impossible to write as transforms, make the minimal necesarry > modifications towards making it possible to write is as transforms, > but > leave it at that. > > - I would however consider that type coercion should be split into a > seperate CoercionTransform descending from the Transform class, rather > than making a new coercion phase within the existing code-in-nodes > paradigm. This makes a lot of sense. I would like to move information about when the coercions are possible to the types themselves too. > - At some point a new layer should definitely be written that allows > simpler types of plugins. However (but I'm less sure of this than > other > things) I believe NumPy support and C++ template support can be > written as > transforms directly, because they are high priorities and to gain > experience with what kind of plugin support would be most beneficial. I think parameterized and intelligent types will play a key role here as well, and seem more modular than transforms looking for types with the string "numpy" in them. > - We should keep in the back of our heads that the transform > problem has > been solved in many ways in varying degrees of generalness > (ick..too late > in the night to look up the proper phrase). For instance it would be > perfectly possible to export the tree to XML, run an XSLT > transform, and > parse the XML back, and then one could use XSLT for writing all the > transforms (which would be much better in some cases than Python, > and much > worse in others). (I'm personally not that happy about XSLT in this > particular case, though for many other uses it is perfect, but I'd > do some > research and look for libraries in general that can help write > transforms. > Something like doing XPath on this thing would help. Hmm...in fact > there > probably are Python XPath implementations that one could fit on our > DOM, > only question is whether it is worth the effort. If you don't know > these > just look them up). Speaking specifically of XSLT, in your framework it would be possible to write a Transform that dumps out the tree as XML (like your printer, but fancier), runs some XSLT on it, then reads it back in. Most of my personal experience with XML has been unimpressive (usually 'cause it's way to bloated of a tool than needed for the task at hand, and slow). > The important thing is that having a basic support for > transforms doesn't say anything about different creative ways the > transforms can be implemented in. > > Example for demonstration only: What if you want to switch your > output to > Java for Jython support? A lot of work, sure, but there's no reason it > should be impossible *in principle* to fit something like that into > Cython > (there would be a lot of overlap in functionality, increasing as we > get > smarter with what Cython does). If C code generation is moved to a > tree > transform instead, it is simply a matter of replacing that one (big!) > component, instead of a matter of breaking the code all over the > place. > (Not saying we ever attempt that though :-) better not, but > contemplating > it kind of makes me decide for which direction I like and don't > like...) That would be one massively crazy transform to try and understand (especially because many relationships are very non-local), but in theory it could be viewed that way. I made a couple of changes to get it to run on Python 2.3 and merged it into the development repo. Something else I noticed was that you are looking at parse trees of single functions, rather than the module as a whole. (I'm guessing this is because the processing phases aren't in sync). It'd be good to change this. - Robert _______________________________________________ Cython-dev mailing list [email protected] http://codespeak.net/mailman/listinfo/cython-dev
