Hello Cython-World, some time ago, I found Cython and wanted to use it. I even came up with a couple of patches, which I believe didn't get merged, mostly because it soon became apparent that we have not the same goals completely.
My interest and goal was to use Cython for my pure Python code. I believe that Python is the language and that using language extensions that turn it to another language is not what I want. Obviously, this is not exactly what Cython is about. So I thought of having my own "toy compiler" which translates pure Python into C or another language which is then compiled. Time is short for me, I have a day job and everything, but this is to inform you that I believe I have succeeded with it. That is, I can take pure Python 2.5 syntax and compile it to C++ and it does pass most of the CPython tests. It is capable of generator expressions and list comprehensions, selection or/and, etc. in all its beauty. It does lambda expressions, decorators for classes and functions. It does exception handling, args with "**" and "*", conditional expressions, and most interestingly, it is capable of generator functions (yield). Well, just everything. You can imagine, I learned a lot about Python in the process. My initial approach was to start from CPython bytecode. (Stefan, you likely recall that discussion.) Turned out that bytecode got me very far, also in understanding how CPython works, but for some cases, I just wasn't happy. I was just almost everything. I wanted everything though. Then I used the parser module of Python itself, and that went much better. Of course the semantics was often new to me, and there is always somebody who uses a syntax in just one test. But I managed to pull through that. BTW: Did you know [ x,y for x in range(4) for y in range(5) ] and could you predict its value? How often is each iterated being created? Deeply confusing stuff. I also absolutely enjoy these strangenesses. Currently, I have for the first release 4 goals. a) Pass the CPython tests even more completely. Some decorators e.g. use the function decorated also as a dictionary to remember things. My "compiled function" type doesn't allow that yet. I do handle eval, exec, locals, globals, execfile, etc. correctly, but there is stuff I will never have, e.g. a "func_code" object, or an execution frame. I find it very convenient to have the CPython tests, I also have some of my own, but being able use "unittest" was a real break through, you can surely imagine. b) I never compiled the compiler with the compiler. I surely want to do that. c) Whole program support. I currently compile module by module from ".py" to ".so" or ".exe". I can recursively compile into os.path, but currently don't have the code to make it actually run. d) Detect dead code. Really, the compiler is a result of solo extreme programming. I at least want the compiler to tell me about unused code and test coverage. That would be milestone 1 and address primarily correctness. How fast it is, I didn't care much yet. In some cases faster than CPython and Cython, but that is not the point yet. Currently libpython does almost all the hard work, and what I mostly do, is to eliminate the bytecode generation and interpretation, and replace it with C++. Depending on the program, that is close to nothing. The idea though, is clearly to get to the point, where I have it optimize the C type cases. I expect that a release achieving these 3 goals will happen at the end of August. I hope to be able to build on your experiences as well, and I also hope to convince you to accept my idea of a "hints" module with decorators that should at least optionally replace the cdef syntax. My core idea there, is that if I say "@hints.returns( int )" or something like this, it would decorate the function into something that asserts that this is really true, where for the compiler or Cython, the use of hints.returns() could be more in place. Probably also interesting will be an exchange of optimization methods, although I have not ventured there much already. I did e.g. compile pystone only to find how much faster CPython was. Then I stole the specialization from them. And I guess, I can learn a lot more from Cython too. I am sure, I have a couple of tricks now, that I will gladly share after my release. You clearly invested a lot of effort to see what's fast and what is not, if only because to me the C-API looks alike and you have tested those already. Regarding newer CPython, I already use Python 2.6 syntax, but not the "with" statement, and I didn't look at 2.7 at all yet. I will probably only make the jump to there after the release. I do not care much about Python 3.x at this point. Best regards, Kay Hayen _______________________________________________ Cython-dev mailing list [email protected] http://codespeak.net/mailman/listinfo/cython-dev
