> It would be nice to figure out a way that pluggins could be written > to be less brittle (i.e. not so dependent on the inner workings and > names of the Cython compiler) too, eventually.
Totally agreed (and I've written about that too some places, especially that the -T switch will only ever be a developer tool). It's important for me that this transform framework isn't as much a plugin architecture as a change in the modularity of the compiler core itself. Currently (if I dare) the ambitions seems to be too high for the current code architecture and (in all modesty :-) ) I believe this approach goes a long way solving that problem. So this is the way I see it (and I'd guess you agree but I want it on the mailing list): - In an ideal world, it is simply wrong to have the methods acting on the parse tree implemented as methods on the parse tree itself. (See example below). Rather you want the parse tree to be "data" and seperate, componentized and well-defined transforms to be the vehicles of making the compilation happen. So ideally, the parse tree shouldn't be that brittle because there should be "nothing there" but information that is spec-ed according to the phases one is in. (However for plugin use it is definetely too complex still). - Working towards such an ideal code world is not the business one is in and is also very boring, so one makes compromises: Whenever new functionality is introduced, write it as transforms, whenever something new is impossible to write as transforms, make the minimal necesarry modifications towards making it possible to write is as transforms, but leave it at that. - I would however consider that type coercion should be split into a seperate CoercionTransform descending from the Transform class, rather than making a new coercion phase within the existing code-in-nodes paradigm. - At some point a new layer should definitely be written that allows simpler types of plugins. However (but I'm less sure of this than other things) I believe NumPy support and C++ template support can be written as transforms directly, because they are high priorities and to gain experience with what kind of plugin support would be most beneficial. - We should keep in the back of our heads that the transform problem has been solved in many ways in varying degrees of generalness (ick..too late in the night to look up the proper phrase). For instance it would be perfectly possible to export the tree to XML, run an XSLT transform, and parse the XML back, and then one could use XSLT for writing all the transforms (which would be much better in some cases than Python, and much worse in others). (I'm personally not that happy about XSLT in this particular case, though for many other uses it is perfect, but I'd do some research and look for libraries in general that can help write transforms. Something like doing XPath on this thing would help. Hmm...in fact there probably are Python XPath implementations that one could fit on our DOM, only question is whether it is worth the effort. If you don't know these just look them up). The important thing is that having a basic support for transforms doesn't say anything about different creative ways the transforms can be implemented in. Example for demonstration only: What if you want to switch your output to Java for Jython support? A lot of work, sure, but there's no reason it should be impossible *in principle* to fit something like that into Cython (there would be a lot of overlap in functionality, increasing as we get smarter with what Cython does). If C code generation is moved to a tree transform instead, it is simply a matter of replacing that one (big!) component, instead of a matter of breaking the code all over the place. (Not saying we ever attempt that though :-) better not, but contemplating it kind of makes me decide for which direction I like and don't like...) Dag Sverre _______________________________________________ Cython-dev mailing list [email protected] http://codespeak.net/mailman/listinfo/cython-dev
