> On Fri, Sep 11, 2009 at 1:36 PM, Dag Sverre Seljebotn > <[email protected]> wrote: >>> On Fri, Sep 11, 2009 at 12:26 PM, Dag Sverre Seljebotn >>> <[email protected]> wrote: >>>> Hi Darren, >>>> >>>>> Hello, >>>>> >>>>> I am just learning cython, please bear with me. This is maybe a >>>>> common >>>>> question, but I didn't recognize it in the documentation or the FAQs. >>>>> How do you make cython definitions available to external C code? For >>>>> example, converting some of numpy's code in numpy/core/src/multiarray >>>>> to cython without affecting the C API? >>>> >>>> Doing just this is something I've been eager to try myself, please >>>> don't >>>> hesitate to ask any questions you might have in the process. >>>> >>>> I hope it's OK that I write down some thoughts I have about this, even >>>> if >>>> they're not really related to your question. >>>> >>>> One thing that complicates this process is that NumPy has a >>>> requirement >>>> that both modes of building the multiarray module works: >>>> - Building all C files seperately, then linking >>>> -- But this won't work with Cython out of the box because the module >>>> initialization code won't get called, so that globals etc. are not >>>> initialized properly. >>>> - Including all C files in a single master C file, then compile only >>>> that >>>> one file >>>> -- This won't work with Cython as it inserts module initialization >>>> code >>>> which will conflict with the module initialization code already >>>> present >>>> in multiarray. >>> >>> Where can I learn more about these requirements? >> >> It's just how the build works, there's two seperate ways of building >> things; one "old" and one new which David introduced recently. I asked >> whether the old one would be dropped, but it appears not: >> >> http://thread.gmane.org/gmane.comp.python.numeric.general/32385 >> >> I'm not sure if this is all that well documented anywhere, I think the >> information has to be extracted on the mailing lists. Please ask again >> whenever you're stuck. >> >> If you can get Cython code into the multiarray module using any of the >> build modes it would be a great first step -- I don't expect it to be >> easy >> (because Cython is made for writing entire modules, not for mixing with >> other C source also making up the module), but I think it is doable. > > So let me see if I understood you: Cython is not currently designed to > let you build a module up from several submodule sources, be they C or
Correct this far. (But see below.) > Cython, because it will not expose the symbols in those submodules. Not quite. The problem is that each Cython-generated source file will always contain a module initialization function, which it is expected (and required) that Python call. multiarray also contains one such function already, and so the two are in conflict. > Isn't it possible to make these available by treating them as external > C sources, using the "cdef extern from "spam.h":" discussed at > http://docs.cython.org/docs/external_C_code.html#referencing-c-header-files > ? Yes. Basically, what you can do is: a) Make a Cython .pyx the "top-level" multiarray module b) Drop the module initialization code currently written in C (that is, turn it from a function called by Python, to a C function called by the Cython module in a)). c) Call into the existing C source by using "cdef extern" etc. When it comes to the build, decide which of the two mechanisms you want to support first. It is probably easiest to tackle the multiple-compiled-sources build first; that basically only requires to declare functions in Cython code "public", and import functions from C through "cdef extern". Single-unit-compilation requires, I think, that you include all the relevant C files into the master top-level pyx file. Then all bets are off concerning whether things will work regarding inclusion order of API header files etc., but it might work. > > Although I am very interested in diving into cython, if it is not > currently a good fit for helping with the numpy py3 transition then I > should probably focus on getting up to speed on what needs to be done > with the existing numpy C code. There are obstacles, but we're talking about things which can be fixed (if necesarry in Cython itself) in a day or two of work. NumPy developers estimate that porting NumPy to Python 3 is an effort for several full-time months by the people who are already up to speed on the codebase -- in comparison with that, the time it would take to deal with this particular issue with Cython (i.e. mixing C and Cython into one module) fades. However, these are complicated issues, and the learning curve might be a bit high, as it requires knowing the internals of both the NumPy build system and the Cython-generated C source. Dag Sverre _______________________________________________ Cython-dev mailing list [email protected] http://codespeak.net/mailman/listinfo/cython-dev
