Hi, > I'm currently looking at integrating bgen with distutils. > > Bgen is a little-known part of the Python core distribution: it is > similar to swig, in that it generates C extension modules. In some > respects it is more powerful than swig, the main one being that it > reads standard C .h files in stead of adorned .i files. > (Incidentally, this is also the reason it's part of core python, it > is used to generate the MacOS API modules, so these are (almost) > automatically updated when Apple adds new functionality. At least, > that was true under MacOS9 and will be again when I get my act > together:-). But bgen has a lot of disadvantages when compared to > swig, the main one being that it is a rather fearsome tool to try and > master. Integration with distutils is one of the things I want to do > to lower that barrier. > > But now I'm at a loss as to how to proceed. I had a look at how swig > is integrated into distutils, and I don't really like it, it smells > like a hack. And, according to the comments in the source and the > manual, the author agrees with me:-) Swig support is basically done > in the build_ext command, by filtering out all ".i" files in the > source file list very early in the process, running swig on them, and > replacing them by the .c or .cpp equivalents.
I've read distutils SWIG support, and I would (politely) say it is an afterthought. The main problem is that distutils is fixed on a two-stage compilation process: compile and link. Additional processing steps are just not supported. > I can see various ways of adding bgen support, but I'm not sure which > one is the best one, and/or whether there are other options. So I'd > be interested in hearing what other people think, and how other > packages have added a preprocessor to distutils. > > There's a fair amount of Python code needed to drive bgen, at least > for interfaces to complex APIs (bridging C types to Python, handling > callbacks, how to parse the specific .h files for this API, etc). > Currently that code is in two .py files but it will be put in a > class, probably modeled somewhat after Extension (but having C/C++ > source files as output in stead of dynamic extension modules). > > What I don't know is how I'd connect this to the Extension object > that will create the extension module. Ideally I'd like the bgen > process to be optional. In other words, the distribution packager has > three options: (a) include the bgen C output in the distribution and > don't run bgen unless the end users specifically asks for it; (b) > include the bgen C output but only run bgen if the normal timestamp > dependencies require it; or (c) always run bgen. > > But it doesn't seem the Extension object currently has any support > for such make-like chaining, and I'm not sure how to add it. One way > would be to allow non-strings in the sources argument, and do > something smart there. A similar mod could be used for libraries and > extra_objects to allow chaining there too. Another way would be to > add a "dependencies" argument, where those dependencies are objects > that get run early, and can add their results to sources, libraries > and extra_objects. I think this latter solution is probably better, > as such a dependency object could modify multiple arguments of > Extension in one fell swoop. As a somewhat contrived example, an > "OptionalJPEGSupport" dependency could check whether the relevant > libraries and include files are available to enable JPEG support in > an imaging package, and then add the right source files, libraries, > defines, library paths and include paths to the relevant Extension > arguments. > > But all of this is made quite a bit more difficult (I think) by the > fact the Extension doesn't really do anything, it's only a container > and all the logic is in build_ext. Maybe I should follow the paradigm > set by "build_clib", and add a "build_bgen" command with build_ext > picking up the results? And maybe there are better solutions that I > haven't thought of yet? I'd like full blown support for more compilation stages in distutils. I tried to come up with something nonobtrusive and failed. Also, the current stated goal of distutils maintenance seems to be: don't break existing installers. That flatly rules out any sort of rewrite. - Lars _______________________________________________ Distutils-SIG maillist - [email protected] http://mail.python.org/mailman/listinfo/distutils-sig
