Watch out,

there are are dragons ... :-)

Let me try to give you some answers nevertheless (which might be
neither correct nor complete). Essentially, Sage uses the classical
Python setuptools/distutils, with some proprietary enhancements for
Cython support. Classically, you have a "python setup.py build" part,
and a "python setup.py install" part, much like "make" versus "make
install".

The "build" part is done in some subdirectory (created/named again
"build") of the directory where the setup.py script sits, and where
under "sage" all the sources are. Then, in "build/sage/
temp.macosx-10.3-i386-2.6" (or the like) all the generated *.o files
from C extensions land (and nothing else), in "build/sage/
lib.macosx-10.3-i386-2.6" (or the like) all the .so files (generated
from the *.o files) land, plus the *.py source files, and in "build/
sage" the generated *.pyc files (together with copies of the .so files
and again, copies of the *.py files). In the end, instead of doing
some real "install" step, a symlink is created under "python/
sitepackages" to this last directory (essentially, putting it into the
PYTHONPATH should do the same trick).

So far so good, up to now all this can be read up e.g. in the
documentation at www.python.org.

But for Sage, things are more complicated, there are (at least) two
more features involved: interactive lookup, and doctesting. The first
means that you can do tab-completion from the Sage command line, or
look up functions and code with "foo?" and "foo??". For this, not only
all the *.py files are necessary under "build/sage" (if you wondered
before, why they are copied there *again*), but to make this also work
with Cython sources, too, the Sage build system has to manually copy
also all the *.pyx files under "build/sage". (Since we know this, we
do it right at the beginning, even before the *.c files are compiled
by the Cython compiler out of the *.pyx files, and profit later from
it.)

Maybe it is this part confusing you --- in this manual step, new
directories are created, if necessary. And currently, this manual step
is using SITE_PACKAGES as anchor (see in the Sage 4.6 setup.py the
function "compile_command0()", around lines 772 - 787). To adapt this
for "build/sage" as anchor instead might mean only a handful of
changes, but these should be thoroughly reviewed ...

On the one hand, the need to do this copying manually is a (well
known, sigh) shortcoming of the available general setuptools/distutils
framework (I think even the adapted version of it coming with Cython
does not do this, but I might be wrong here). On the other hand,
currently it is done incompletely --- all the Cython (header-like)
*.pyd files should also be copied, but aren't. I consider this a bug
(for which I should have made a trac ticket a log time ago). It would
be needed e.g. for being able to doctest correctly out of "build/
sage" (so you have to doctest out of "devel/sage/sage" instead).

I skipped a bit through #9967, one more point that comes to my mind is
that we currently have nothing like a CYTHONPATH or such. Remember,
that to compile the *.pyx files, the entire source dir tree is seen
from a different angle, namely from that of a Cython compiler instead
that of Python interpreter. (Often, in foo.pyx, it does not suffice to
do "cimport bar.pyx", one also has to do "import bar.pyx" in addition
to that --- neither forms are subsets of the other.) Currently, one
just adds manually to the Cython compile command the necessary magic
(both during really building, and also during doctesting), but all of
this seems to me ad-hoc, rather than a general framework.

After having compiled the Cython *.pyx files to C files (or C++
files), then a C (or C++) compiler is to be invoked to generate the
*.o files for the respective extensions. Again, one has to pay
attention with having the necessary include paths etc. to be set ---
but as said before, this one step is handled more or less gracefully
by the classical setuptools/distutils machinery.

I hope that helped!


Cheers,
Georg

-- 
To post to this group, send an email to sage-devel@googlegroups.com
To unsubscribe from this group, send an email to 
sage-devel+unsubscr...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/sage-devel
URL: http://www.sagemath.org

Reply via email to