#8306: Parallel inter/intra-spkg builds
----------------------------+-----------------------------------------------
Reporter: mpatel | Owner: GeorgSWeber
Type: enhancement | Status: needs_work
Priority: major | Milestone: sage-4.3.4
Component: build | Keywords:
Author: Mitesh Patel | Upstream: N/A
Reviewer: David Kirkby | Merged:
Work_issues: |
----------------------------+-----------------------------------------------
Changes (by drkirkby):
* status: needs_review => needs_work
* reviewer: => David Kirkby
Comment:
The idea seems excellent. Ir is such a waste on modern machines to
I think it would be a lot better if an environment variable
SAGE_PARALLEL_BUILD or SAGE_PARALLEL_SPKG_BUILD was set to "yes", rather
than require the user to edit the spkg/install. Since virtually all other
environement variables are prefixed with SAGE (e.g. SAGE_FORTRAN,
SAGE_FORTRAN_LIB, SAGE64 ...) I would do likewise for consistency.
It would be necessary to test this on different architectures and
operating systems. It is quite possible that the time different packages
take to build differs vastly by the processor and operating system. It is
likely that package X always builds in parallel before package Y on
sage.math, but Y would build before X on another system. That could well
mean there are dependencies existing that are not apparerent in serial
builds.
For example, ATLAS takes at least 10x as long to build on 't2' as it does
on my own SPARC, simply because there are no default tuning parameters for
ATLAS on t2, so it is tuned each time. That is an extreme example, but it
is well known some machine are faster at some tasks than others, but
slower on other tasks. Some packages have assembly language support for
certain processors, but not others. Several packages go through some sort
of tuning process to determine the optimal build parameters. So the
timings could be expected to be different on different operating systems.
At the very least, this should be tested on t2 and bsd, as they run
Solaris and OS X repectively. (When testing on t2, I would suggest using j
of 256 or 512. That machine has 128 threads). For t2, one would need to
use sage 4.3.0.1. (I think any changes to spkg/install and
spkg/standard/deps should minimal between 4.3.0.1 and 4.3.3. There are
probably no changes at all.
It would be safer to compare the md5 checksums of libraries & binaries
built in series and parallel to prove they are indeed the same. It is
quite possible that there are failures that just do not get exercised by
doctests. However, that may not be fully possible, as perhaps some would
have the build time information encoded in some way. But I would at least
investigate.
Overall, I think this is a really '''excellent''' idea, but it needs
further testing before I'd want to give it a positive review.
Dave
--
Ticket URL: <http://trac.sagemath.org/sage_trac/ticket/8306#comment:13>
Sage <http://www.sagemath.org>
Sage: Creating a Viable Open Source Alternative to Magma, Maple, Mathematica,
and MATLAB
--
You received this message because you are subscribed to the Google Groups
"sage-trac" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/sage-trac?hl=en.
