Hi Chris, On Sun, 18 Sep 2011 20:29:19 -0700 (PDT) Chris Kees <[email protected]> wrote:
> Apologies in advance for the long post. It boils down to this: Is > there any interest from the sage community in participating in the > development of a python distribution for large-scale distributed > memory parallel machines? I'm posting this on behalf of (but not > representing) a group of government scientists who are trying to > work toward a common python distribution on the government systems we > use. The reasons we're doing this are 1) we can't trust the system > python on many HPC systems if it even exists; 2) Due to 1. almost all > of us spend too much time building and maintaining our own python > "stack" based on some mixture of make, cmake, autoconf, and/or the > sage spkg system; and 3) our community suffers from the fact that we > can't always share python modules and scripts on these systems > because we're not working from equivalent python environments. Francois Bissey and I have been working on using gentoo portage to replace the Sage distribution system for a while. This consists of putting sage-on-gentoo [1] on top of gentoo-prefix [2] and modifying the bootstrap scripts from gentoo-prefix so that they use the tool chain from the host instead of rebuilding it. [1] https://github.com/cschwan/sage-on-gentoo [2] http://www.gentoo.org/proj/en/gentoo-alt/prefix/ Here is what we have so far: https://bitbucket.org/burcin/sage-prefix It should build on most linux systems. I regularly try on gentoo and opensuse. I know that there are problems on recent Ubuntu & Debian (Python 2.7 needs a patch for the multiarch library paths). It is quite heavy weight at the moment since we didn't spend much time deciding what to use from the host and what to rebuild. This is definitely on the todo list, after getting it to build reliably on linux and OSX at least. I looked at other options before starting with gentoo-prefix, like Nix [3], or virtualenv. They either needed too much work to get off the ground and to maintain or didn't offer enough flexibility. [3] https://github.com/dagss/scidist/blob/b12bf0b99989a694dfc0eea2ffc30000725fb9c8/ideas.rst Most mathematics software has a handwritten fragile build system. Gentoo already has build scripts (ebuilds) for most packages I care about, for example Macaulay2 and polymake besides Sage. It also offers frameworks to customize how packages are built (so called use flags) and handle different blas libraries or compilers installed side by side on the same system (eselect blas or gcc-config). I especially like the idea of reusing an existing package manager and cooperating with a linux distribution to prepare the build scripts. Gentoo science project is already very active. They maintain a large repository of scientific software packages: http://git.overlays.gentoo.org/gitweb/?p=proj/sci.git;a=tree Would you be willing to move away from the spkg system and try this approach? It is possible to build different configurations into this distribution. For example, we could have Sage, psage (William Stein's number theory distribution which removes the numpy, scipy, maxima, etc. from Sage), Polybori + Singular, as well as one with your package list below. Cheers, Burcin <snip> > 1) Standard: > > Python 2.7.2 PLUS: > > - numpy * > - scipy > - matplotlib * > - vtk (python wrappers + C++ libs) * > - elementtree * > - ctypes * > - readline (i.e. a functional readline extension module) * > - swig > - mpi4py * > - petsc4py * > - pympi > - nose * > - pytables * > - basemap > - cython * > - sympy * > - pycuda > - pyopencl > - IPython * > - wxpython > - PyQt * > - pygtk > - PyTrilinos > - virtualenv * > - Pandas > - numexpr * > - pygrib -- To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/sage-support URL: http://www.sagemath.org
