On Wed, Jun 10, 2009 at 10:01 AM, Robert Bradshaw<[email protected]> wrote: > On Jun 10, 2009, at 8:35 AM, Ondrej Certik wrote: > >> On Wed, Jun 10, 2009 at 12:13 AM, Robert >> Bradshaw<[email protected]> wrote: >>> On Jun 9, 2009, at 11:03 AM, Ondrej Certik wrote: >>> >>>> Hi, >>>> >>>> I have the following problem -- I have a package hermes2d, with the >>>> structure: >>>> >>>> hermes2d/__init__.py >>>> hermes2d/_hermes2d.pyx + .pxd >>>> hermes2d/_forms.pyx >>>> >>>> and sometimes (depending on what the users tells cmake) I want to >>>> build the .so libraries out of the tree and put their paths to >>>> PYTHONPATH. Things just work as long as remove the file >>>> hermes2d/__init__.py during the cython compilation. The reason is >>>> that >>>> if I do: >>>> >>>> cd hermes2d >>>> cython _forms.pyx >>>> >>>> it produces _forms.c with lines like: >>>> >>>> __pyx_ptype_8hermes2d_9_hermes2d_LinSystem = >>>> __Pyx_ImportType("hermes2d._hermes2d", "LinSystem", sizeof(struct >>>> __pyx_obj_8hermes2d_9_hermes2d_LinSystem)); if >>>> (unlikely(!__pyx_ptype_8hermes2d_9_hermes2d_LinSystem)) >>>> {__pyx_filename = __pyx_f[1]; __pyx_lineno = 292; __pyx_clineno = >>>> __LINE__; goto __pyx_L1_error;} >>>> >>>> Notice the import "hermes2d._hermes2d", which fails if >>>> _hermes2d.so is >>>> out of the tree (e..g somewhere else on the filesystem, even if it's >>>> in the search path!). If, however I remove the __init__.py, then >>>> cython generates: >>>> >>>> __pyx_ptype_9_hermes2d_LinSystem = __Pyx_ImportType("_hermes2d", >>>> "LinSystem", sizeof(struct __pyx_obj_9_hermes2d_LinSystem)); if >>>> (unlikely(!__pyx_ptype_9_hermes2d_LinSystem)) {__pyx_filename = >>>> __pyx_f[1]; __pyx_lineno = 292; __pyx_clineno = __LINE__; goto >>>> __pyx_L1_error;} >>>> >>>> and everything just works, no matter where the file _hermes2d.so >>>> actually is. >>>> >>>> Now my question is -- if I want those .so libraries not to be in the >>>> tree, am I doing things the right way? E.g. I just have a line: >>>> >>>> from _forms import * >>>> >>>> in the hermes2d/forms.py >>>> >>>> and then people just do "from hermes2d import forms" and things >>>> work, >>>> no matter where the .so files are. But I need some way to force >>>> cython >>>> to just pretend there is no __init__.py in there. Why is cython >>>> doing >>>> this check at all? Does it have any advantages? >>> >>> One advantage I see is that you may have two modules named _hermes2d >>> in different packages that it might have to distinguish. (OK, rare >>> with that name, but it could happen with more common names). Also, >>> this way the module path is fully qualified as one would expect. >> >> Ok. So how do you propose I should fix the problem, if I need those >> .so libraries at different place than the rest of the python source >> files? > > I don't know. A hack is to have your setup.py actually move the > __init__.py temporarily--I'm sure there's a better longterm fix > though. Just out of curiosity, why do you need to segregate the .so > files?
Because we use cmake to build the project (C++ and cython) and cmake allows to build the project out of the tree, e.g. it leaves the .cpp and .py files in the source dir and creates a build dir, where it stores all the .o and .so files. If the project is just C++, it is then possible to use the build directory as is, e.g. you don't have to install it and it still works (of course if you want, you can also install it). With Cython, it doesn't work, because the .so files don't find each other. One solution (besides moving __init__.py files away at each build) is to just create an option to cython, that will disable this feature (or pretend the __init__.py files are not there). I am not convinced it's worth the pain though yet. So you think one should never need to segregate the .so file in a Python + C++ project? Ondrej _______________________________________________ Cython-dev mailing list [email protected] http://codespeak.net/mailman/listinfo/cython-dev
