#9508: Fix all ATLAS build problems on Solaris/OpenSolaris
----------------------------+-----------------------------------------------
Reporter: drkirkby | Owner: drkirkby
Type: defect | Status: new
Priority: major | Milestone: sage-4.5.3
Component: solaris | Keywords:
Author: David Kirkby | Upstream: N/A
Reviewer: | Merged:
Work_issues: |
----------------------------+-----------------------------------------------
Changes (by newvalueoldvalue):
* milestone: sage-4.5.2 => sage-4.5.3
* author: => David Kirkby
Old description:
> The ATLAS package in Sage (atlas-3.8.3.p12 in sage 4.5.rc1) has a problem
> in the way shared libraries are built and installed.
> * libatlas and libcblas never get built as shared libraries on Solaris
> or !OpenSolaris, despite they do get built on Linux, OS X and FreeBSD.
> * liblapack gets built, but later gets removed by the following few
> lines of code in the script {{{make_correct_shared.sh}}}, which was
> probably added in #5024.
> {{{
> # on Solaris a dynamic liblapack.so leads to import errors in numpy, so
> delete them for now.
> if [ `uname` = "SunOS" ]; then
> echo "Deleting liblapack.so on Solaris due to bug in numpy/scipy"
> cd "$SAGE_LOCAL"/lib
> rm -rf liblapack.so*
> fi
> }}}
> * On 64-bit builds of Sage, Linbox reports ATLAS is not installed - see
> #9101. Closer inspection of Linbox's config.log shows that linbox
> determines the ATLAS libraries are 32-bit, not 64-bit. Since it finds no
> 64-bit libraries, it considers ATLAS is not installed.
>
> I suspect the reasons the shared libraries are not currently built in
> Sage is that they were originally built incorrectly, so rather than fix
> the problem, they were just deleted. I'm aware Michael Abshoff was using
> the GNU linker, despite this has never had a good reputation on Solaris,
> and even the GCC documentation advises using the Sun linker. Hence I
> think we should
>
> * Build the shared libraries properly, using the Sun linker
> {{{/usr/ccs/bin/ld}}}
> * Consider deleting the static libraries. I don't think they would
> perform any useful function if the shared libraries existed.
>
> This will have several advantages.
>
> * Shared libraries are much smaller than static libraries.
> * Shared libraries take up far less memory when th
> * Linbox will hopefully not report ATLAS is not installed.
>
> The following command will build a a 32-bit or 64-bit shared library
> liblapack.so, although it could easily be extended to other libraries
> too.
> {{{
> if [ "x$SAGE64" = xyes ] ; then
> # To create a 64-bit shared library, the linker flag
> # '-64' must be added. Note this is not the same as
> # the compiler flag '-m64'
> LINKER_BITS=-64
> else
> LINKER_BITS=-32
> fi
>
> # Build liblapack.so
> lapack_command="/usr/ccs/bin/ld $LINKER_BITS -L"$SAGE_LOCAL/lib" -G
> -h liblapack.so -o liblapack.so -zallextract liblapack.a
> -zdefaultextract -
> lc -lm -lgfortran"
> $lapack_command
> }}}
>
> This will need some testing, but at least this will all be specific to
> Solaris or !OpenSolaris, so will not cause any problems on other
> platforms.
>
> Dave
New description:
The ATLAS package in Sage (atlas-3.8.3.p12 in sage 4.5.1) has a number of
problems on Solaris or !OpenSolaris. It only works fully on 32-bit SPARC.
These problems range from serious to trivial.
Fortunately, all problems can be resolved by putting code inside sections
which only get executed on Solaris, so virtually eliminating any risk on
non Solaris/!OpenSolaris system.
These problems are:
* An erroneous, but harmless message printed from {{{spkg-install-
script}}}
* Shared libraries were not built properly, and as a result they were
deleted.
* Linker flags were not set correctly on Solaris 10 x86 or !OpenSolaris
in 32-bit mode.
These will now be considered in turn.
== An erroneous, but harmless message from {{{spkg-install-script}}} ==
The script {{{spkg-install-script}}} would always print:
{{{
Change ldflag -melf_x86_64 to -64 as needed for Sun ld
}}}
when the Sun linker was used on Solaris, which was not true on anything
except 64-bit builds on Solaris or !OpenSolaris.
== Shared libraries were deleted in {{{make_correct_shared.sh}}} ==
ATLAS's shared libraries are built on Linux, OS X and FreeBSD, but were
deleted on Solaris due to the fact they did not work. I suspect the
problem was they were never made correctly in the first place, as they
appear to work if built correctly.
The original code has this.
{{{
# on Solaris a dynamic liblapack.so leads to import errors in numpy, so
delete them for now.
if [ `uname` = "SunOS" ]; then
echo "Deleting liblapack.so on Solaris due to bug in numpy/scipy"
cd "$SAGE_LOCAL"/lib
rm -rf liblapack.so*
fi
}}}
If the libraries are built correctly, with the following code, which loops
though the libraries libatlas liblapack libf77blas and libcblas, the
problem is solved.
{{{
# Build dynamic libraries on Solaris, using the Sun equivalent of the GNU
# options above. Rather than call 'ld', the exact path to 'ld' is given
# since we know for 100% sure that 'ld' will reside in /usr/ccs/bin
# on Solaris - this is true of both Solaris 10 and OpenSolaris
# I'm not 100% sure if these options are optimal (there might be some
# unnecessary commands, but its the exact equivalent of the GNU
# commands above.
if [ "x`uname`" = xSunOS ]; then
cd "$SAGE_LOCAL/lib"
if [ "x$SAGE64" = xyes ] ; then
# To create a 64-bit shared library, the linker flag
# '-64' must be added. Note this is not the same as
# the compiler flag '-m64'
LINKER_FLAG=-64
fi
# Build libatlas.so liblapack.so libf77blas.so and libcblas.so
# Actually, libatlas.so and libcblas.so gets built from spkg-install,
# but this is not working correctly for 64-bit libraries. Hence I
simply
# remake those two libraries here. This is why 4 libraries are built
# in this Solaris section, but only two are shown above for FreeBSD,
# OS X and Linux.
for ATLAS_LIBRARY in libatlas liblapack libf77blas libcblas ; do
echo "Building shared library $ATLAS_LIBRARY.so from the static
library $A
TLAS_LIBRARY.a"
/usr/ccs/bin/ld $LINKER_FLAG -L"$SAGE_LOCAL/lib" -G -h
$ATLAS_LIBRARY.so -
o $ATLAS_LIBRARY.so -zallextract $ATLAS_LIBRARY.a -zdefaultextract -lc
-lm -lg
fortran
if [ $? -ne 0 ]; then
echo "Failed to build ATLAS library $ATLAS_LIBRARY.so"
exit 1
else
echo "$ATLAS_LIBRARY.so has been built on Solaris."
fi
done
fi
}}}
there is no need to delete them. This has the advantage that linbox can
use the libraries - otherwise linbox reports ATLAS is not installed - see
#9101.
== Link options not set correctly in spkg-install-script ==
Code in {{{spkg-install-script}}} tried to change the linker option
{{{-melf_x86_64}}} to {{{-64}}}, irrespective of type of Solaris or
!OpenSolaris system. In fact, this is only appropiate on 64-bit Solaris or
!OpenSolaris builds. On 32-bit builds, a similar option must just be
deleted. On SPARC builds, there is no need to change the options at all,
as ATLAS gets it right.
The following code correctly sets the options. The first parts works on
64-bit code ({{{SAGE64=yes}}}) and the second part on 32-bit code, where
{{{SAGE64}}} is not set to yes.
{{{
if [ "x$SAGE64" = xyes ] && [ "x`uname -m`" = xi86pc ] ; then
# Solaris or OpenSolaris on x86/x64 on a 64-bit build
echo "Change ldflag -melf_x86_64 to -64 as needed for Sun ld"
echo "on 64-bit builds of ATLAS on x64 hardware"
sed 's/-melf_x86_64/-64/g' Make.inc > makeinc
mv makeinc Make.inc
fi
if [ "x$SAGE64" != xyes ] && [ "x`uname -m`" = xi86pc ] ; then
# Solaris or OpenSolaris on x86/x64 on a 32-bit build
echo "Remove the linker flag -melf_i386 as needed for Sun ld"
echo "on 32-bit builds of ATLAS on x86/x64 hardware"
sed 's/-melf_i386//g' Make.inc > makeinc
mv makeinc Make.inc
fi
}}}
--
--
Ticket URL: <http://trac.sagemath.org/sage_trac/ticket/9508#comment:1>
Sage <http://www.sagemath.org>
Sage: Creating a Viable Open Source Alternative to Magma, Maple, Mathematica,
and MATLAB
--
You received this message because you are subscribed to the Google Groups
"sage-trac" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/sage-trac?hl=en.
