On Mon, Jul 25, 2011 at 11:50 PM, Stefan Behnel <stefan...@behnel.de> wrote: > Robert Bradshaw, 26.07.2011 07:00: >> >> On Mon, Jul 25, 2011 at 9:48 PM, Vitja Makarov wrote: >>> >>> I think that would seriously help with moving shared C-code into >>> cython library (.h and .so). >>> >>> Some things like generator class implementation and cyfunction could >>> be move into .so file. >> >> Yes, that would be good. > > Well, *if* we find a way to make sure the additional modules get properly > distributed. It's not obvious to me how to do that, certainly not in a > cross-package way, and not even in a per-package way, as that would still > require some kind of interaction with distutils and maybe even changes to > the users' setup.py scripts (hopefully in a way that doesn't require them to > know what exact additional modules will be generated).
This is where the ext_modules = Cython.Build.cythonize(...) pattern could come in handy. We'd just add an extra module to the list. (The generated .c file could also be shipped, but might have to be listed explicitly.) >> Of course we don't want a single .so file for >> every tiny chunk of utility code, but one needs to be careful lumping >> them together as well (e.g. things may compile differently depending >> on the set of headers included in the .pyx file). To do this right I >> we need to compile things at a higher level than module-by-module with >> Cython.Build.cythonize or similar. >> >> Certainly CyFunction and the generator class are easy and high-benefit >> targets (compared to, e.g. tuple indexing utility code). > > Now that we have a way to group utility code in a single file, including > automatically resolved dependencies to other files, it should be trivial to > take such a file and to compile it into a separate module. Cython modules > would then trivially export their public names anyway, and utility code in C > files could have an additional Cython wrapper file that would simply depend > on the C file and that properly exports the C functions through a Python API > and inter-module C-API. > > We could even write empty utility files that only contain dependencies, thus > grouping together multiple utility files to one larger module. I was thinking about stuff like how we handle complex numbers where the pre-processor has a large role to play. Low-level stuff like this probably belongs in a .h file rather than a .so file though. I'm also wondering if there are any efficiency gains to be had from actually linking the resulting .so file at compile time rather than using the runtime inter-module C-API stuff. - Robert _______________________________________________ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
