Watch out, there are are dragons ... :-)
Let me try to give you some answers nevertheless (which might be neither correct nor complete). Essentially, Sage uses the classical Python setuptools/distutils, with some proprietary enhancements for Cython support. Classically, you have a "python setup.py build" part, and a "python setup.py install" part, much like "make" versus "make install". The "build" part is done in some subdirectory (created/named again "build") of the directory where the setup.py script sits, and where under "sage" all the sources are. Then, in "build/sage/ temp.macosx-10.3-i386-2.6" (or the like) all the generated *.o files from C extensions land (and nothing else), in "build/sage/ lib.macosx-10.3-i386-2.6" (or the like) all the .so files (generated from the *.o files) land, plus the *.py source files, and in "build/ sage" the generated *.pyc files (together with copies of the .so files and again, copies of the *.py files). In the end, instead of doing some real "install" step, a symlink is created under "python/ sitepackages" to this last directory (essentially, putting it into the PYTHONPATH should do the same trick). So far so good, up to now all this can be read up e.g. in the documentation at www.python.org. But for Sage, things are more complicated, there are (at least) two more features involved: interactive lookup, and doctesting. The first means that you can do tab-completion from the Sage command line, or look up functions and code with "foo?" and "foo??". For this, not only all the *.py files are necessary under "build/sage" (if you wondered before, why they are copied there *again*), but to make this also work with Cython sources, too, the Sage build system has to manually copy also all the *.pyx files under "build/sage". (Since we know this, we do it right at the beginning, even before the *.c files are compiled by the Cython compiler out of the *.pyx files, and profit later from it.) Maybe it is this part confusing you --- in this manual step, new directories are created, if necessary. And currently, this manual step is using SITE_PACKAGES as anchor (see in the Sage 4.6 setup.py the function "compile_command0()", around lines 772 - 787). To adapt this for "build/sage" as anchor instead might mean only a handful of changes, but these should be thoroughly reviewed ... On the one hand, the need to do this copying manually is a (well known, sigh) shortcoming of the available general setuptools/distutils framework (I think even the adapted version of it coming with Cython does not do this, but I might be wrong here). On the other hand, currently it is done incompletely --- all the Cython (header-like) *.pyd files should also be copied, but aren't. I consider this a bug (for which I should have made a trac ticket a log time ago). It would be needed e.g. for being able to doctest correctly out of "build/ sage" (so you have to doctest out of "devel/sage/sage" instead). I skipped a bit through #9967, one more point that comes to my mind is that we currently have nothing like a CYTHONPATH or such. Remember, that to compile the *.pyx files, the entire source dir tree is seen from a different angle, namely from that of a Cython compiler instead that of Python interpreter. (Often, in foo.pyx, it does not suffice to do "cimport bar.pyx", one also has to do "import bar.pyx" in addition to that --- neither forms are subsets of the other.) Currently, one just adds manually to the Cython compile command the necessary magic (both during really building, and also during doctesting), but all of this seems to me ad-hoc, rather than a general framework. After having compiled the Cython *.pyx files to C files (or C++ files), then a C (or C++) compiler is to be invoked to generate the *.o files for the respective extensions. Again, one has to pay attention with having the necessary include paths etc. to be set --- but as said before, this one step is handled more or less gracefully by the classical setuptools/distutils machinery. I hope that helped! Cheers, Georg -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org