#9508: Fix all ATLAS build problems on Solaris/OpenSolaris
----------------------------+-----------------------------------------------
Reporter: drkirkby | Owner: drkirkby
Type: defect | Status: needs_info
Priority: major | Milestone: sage-4.5.3
Component: solaris | Keywords:
Author: David Kirkby | Upstream: N/A
Reviewer: | Merged:
Work_issues: |
----------------------------+-----------------------------------------------
Old description:
> The ATLAS package in Sage (atlas-3.8.3.p12 in sage 4.5.1) has a number of
> problems on Solaris or !OpenSolaris. It only works fully on 32-bit SPARC.
> These problems range from serious to trivial.
>
> The {{{spkg-check}}} file also failed to be very informative, and would
> have not exited properly if a test had failed.
>
> Fortunately, with the exception of more informative messages from
> {{{spkg-check}}}, all problems can be resolved by putting code inside
> sections which only get executed on Solaris, so virtually eliminating any
> risk on non Solaris/!OpenSolaris system.
>
> These Solaris/!OpenSolaris problems are:
> * An erroneous, but harmless message printed from {{{spkg-install-
> script}}}
> * Shared libraries were not built properly, and as a result they were
> deleted.
> * Linker flags were not set correctly on Solaris 10 x86 or !OpenSolaris
> in 32-bit mode.
>
> These will now be considered in turn.
> == An erroneous, but harmless message from {{{spkg-install-script}}} ==
> The script {{{spkg-install-script}}} would always print:
>
> {{{
> Change ldflag -melf_x86_64 to -64 as needed for Sun ld
> }}}
>
> when the Sun linker was used on Solaris, which was not true on anything
> except 64-bit builds on Solaris or !OpenSolaris.
>
> == Shared libraries were deleted in {{{make_correct_shared.sh}}} ==
> ATLAS's shared libraries are built on Linux, OS X and FreeBSD, but were
> deleted on Solaris due to the fact they did not work. I suspect the
> problem was they were never made correctly in the first place, as they
> appear to work if built correctly.
>
> The original code has this.
> {{{
> # on Solaris a dynamic liblapack.so leads to import errors in numpy, so
> delete them for now.
> if [ `uname` = "SunOS" ]; then
> echo "Deleting liblapack.so on Solaris due to bug in numpy/scipy"
> cd "$SAGE_LOCAL"/lib
> rm -rf liblapack.so*
> fi
> }}}
>
> If the libraries are built correctly, with the following code, which
> loops though the libraries libatlas liblapack libf77blas and libcblas,
> the problem is solved.
>
> {{{
> # Build dynamic libraries on Solaris, using the Sun equivalent of the GNU
> # options above. Rather than call 'ld', the exact path to 'ld' is given
> # since we know for 100% sure that 'ld' will reside in /usr/ccs/bin
> # on Solaris - this is true of both Solaris 10 and OpenSolaris
> # I'm not 100% sure if these options are optimal (there might be some
> # unnecessary commands, but its the exact equivalent of the GNU
> # commands above.
>
> if [ "x`uname`" = xSunOS ]; then
> cd "$SAGE_LOCAL/lib"
>
> if [ "x$SAGE64" = xyes ] ; then
> # To create a 64-bit shared library, the linker flag
> # '-64' must be added. Note this is not the same as
> # the compiler flag '-m64'
> LINKER_FLAG=-64
> fi
>
> # Build libatlas.so liblapack.so libf77blas.so and libcblas.so
> # Actually, libatlas.so and libcblas.so gets built from spkg-install,
> # but this is not working correctly for 64-bit libraries. Hence I
> simply
> # remake those two libraries here. This is why 4 libraries are built
> # in this Solaris section, but only two are shown above for FreeBSD,
> # OS X and Linux.
>
> for ATLAS_LIBRARY in libatlas liblapack libf77blas libcblas ; do
> echo "Building shared library $ATLAS_LIBRARY.so from the static
> library $A
> TLAS_LIBRARY.a"
> /usr/ccs/bin/ld $LINKER_FLAG -L"$SAGE_LOCAL/lib" -G -h
> $ATLAS_LIBRARY.so -
> o $ATLAS_LIBRARY.so -zallextract $ATLAS_LIBRARY.a -zdefaultextract -lc
> -lm -lg
> fortran
> if [ $? -ne 0 ]; then
> echo "Failed to build ATLAS library $ATLAS_LIBRARY.so"
> exit 1
> else
> echo "$ATLAS_LIBRARY.so has been built on Solaris."
> fi
> done
> fi
> }}}
>
> there is no need to delete them. This has the advantage that linbox can
> use the libraries - otherwise linbox reports ATLAS is not installed - see
> #9101.
>
> == Link options not set correctly in spkg-install-script ==
> Code in {{{spkg-install-script}}} tried to change the linker option
> {{{-melf_x86_64}}} to {{{-64}}}, irrespective of type of Solaris or
> !OpenSolaris system. In fact, this is only appropiate on 64-bit Solaris
> or !OpenSolaris builds. On 32-bit builds, a similar option must just be
> deleted. On SPARC builds, there is no need to change the options at all,
> as ATLAS gets it right.
>
> The following code correctly sets the options. The first parts works on
> 64-bit code ({{{SAGE64=yes}}}) and the second part on 32-bit code, where
> {{{SAGE64}}} is not set to yes.
>
> {{{
> if [ "x$SAGE64" = xyes ] && [ "x`uname -m`" = xi86pc ] ; then
> # Solaris or OpenSolaris on x86/x64 on a 64-bit build
> echo "Change ldflag -melf_x86_64 to -64 as needed for Sun ld"
> echo "on 64-bit builds of ATLAS on x64 hardware"
> sed 's/-melf_x86_64/-64/g' Make.inc > makeinc
> mv makeinc Make.inc
> fi
> if [ "x$SAGE64" != xyes ] && [ "x`uname -m`" = xi86pc ] ; then
> # Solaris or OpenSolaris on x86/x64 on a 32-bit build
> echo "Remove the linker flag -melf_i386 as needed for Sun ld"
> echo "on 32-bit builds of ATLAS on x86/x64 hardware"
> sed 's/-melf_i386//g' Make.inc > makeinc
> mv makeinc Make.inc
> fi
> }}}
New description:
The ATLAS package in Sage (atlas-3.8.3.p12 in sage 4.5.1) has a number of
problems on Solaris or !OpenSolaris. It only works fully on 32-bit SPARC.
These problems range from serious to trivial.
The {{{spkg-check}}} file also failed to be very informative, and would
have not exited properly if a test had failed.
Fortunately, with the exception of more informative messages from {{{spkg-
check}}}, all problems can be resolved by putting code inside sections
which only get executed on Solaris, so virtually eliminating any risk on
non Solaris/!OpenSolaris system.
These Solaris/!OpenSolaris problems are:
* An erroneous, but harmless message printed from {{{spkg-install-
script}}}
* Shared libraries were not built properly, and as a result they were
deleted.
* Linker flags were not set correctly on Solaris 10 x86 or !OpenSolaris
in 32-bit mode.
These will now be considered in turn.
== An erroneous, but harmless message from {{{spkg-install-script}}} ==
The script {{{spkg-install-script}}} would always print:
{{{
Change ldflag -melf_x86_64 to -64 as needed for Sun ld
}}}
when the Sun linker was used on Solaris, which was not true on anything
except 64-bit builds on Solaris or !OpenSolaris.
== Shared libraries were deleted in {{{make_correct_shared.sh}}} ==
ATLAS's shared libraries are built on Linux, OS X and FreeBSD, but were
deleted on Solaris due to the fact they did not work. I suspect the
problem was they were never made correctly in the first place, as with the
exception of liblapack.so, they appear to work if built correctly.
If the libraries are built correctly, with the following code, which loops
though the libraries libatlas libf77blas and libcblas, the problem is
solved.
{{{
if [ "x`uname`" = xSunOS ]; then
cd "$SAGE_LOCAL/lib"
if [ "x$SAGE64" = xyes ] ; then
# To create a 64-bit shared library, the linker flag
# '-64' must be added. Note this is not the same as
# the compiler flag '-m64'
LINKER_FLAG=-64
fi
# Build libatlas.so libf77blas.so and libcblas.so
# Actually, libatlas.so and libcblas.so gets built from spkg-install,
# but this is not working correctly for 64-bit libraries. Hence I
simply
# remake those two libraries here. This is why 4 libraries are built
# in this Solaris section, but only two are shown above for FreeBSD,
# OS X and Linux.
for ATLAS_LIBRARY in libatlas libf77blas libcblas ; do
echo "Building shared library $ATLAS_LIBRARY.so from the static
library $ATLAS_LIBRARY.a"
/usr/ccs/bin/ld $LINKER_FLAG -L"$SAGE_LOCAL/lib" -G -h
$ATLAS_LIBRARY.so -o $ATLAS_LIBRARY.so -zallextract $ATLAS_LIBRARY.a
-zdefaultextract -lc -lm -lgfortran
if [ $? -ne 0 ]; then
echo "Failed to build ATLAS library $ATLAS_LIBRARY.so"
exit 1
else
echo "$ATLAS_LIBRARY.so has been built on Solaris."
fi
done
rm liblapack.so # liblapack.so causes problems with R on Solaris.
fi
}}}
This has the advantage that linbox can use the libraries - otherwise
linbox reports ATLAS is not installed - see #9101. However, liblapack.so
still presents problems if built with R, so that is not built.
== Link options not set correctly in spkg-install-script ==
Code in {{{spkg-install-script}}} tried to change the linker option
{{{-melf_x86_64}}} to {{{-64}}}, irrespective of type of Solaris or
!OpenSolaris system. In fact, this is only appropiate on 64-bit Solaris or
!OpenSolaris builds. On 32-bit builds, a similar option must just be
deleted. On SPARC builds, there is no need to change the options at all,
as ATLAS gets it right.
The following code correctly sets the options. The first parts works on
64-bit code ({{{SAGE64=yes}}}) and the second part on 32-bit code, where
{{{SAGE64}}} is not set to yes.
{{{
if [ "x$SAGE64" = xyes ] && [ "x`uname -m`" = xi86pc ] ; then
# Solaris or OpenSolaris on x86/x64 on a 64-bit build
echo "Change ldflag -melf_x86_64 to -64 as needed for Sun ld"
echo "on 64-bit builds of ATLAS on x64 hardware"
sed 's/-melf_x86_64/-64/g' Make.inc > makeinc
mv makeinc Make.inc
fi
if [ "x$SAGE64" != xyes ] && [ "x`uname -m`" = xi86pc ] ; then
# Solaris or OpenSolaris on x86/x64 on a 32-bit build
echo "Remove the linker flag -melf_i386 as needed for Sun ld"
echo "on 32-bit builds of ATLAS on x86/x64 hardware"
sed 's/-melf_i386//g' Make.inc > makeinc
mv makeinc Make.inc
fi
}}}
--
Comment(by drkirkby):
Since #9536 system_alias.py fixed problems with SAGE_ATLAS_LIB and was
marged as atlas-3.8.3.p13.spkg, I needed to create a atlas-3.8.3.p14.spkg
to avoid a name clash.
The changes made by #9356 have not been reversed, as it is still
necessary to check for liblapack.a, and not liblapack.so, as that's the
one library that if built causes problems for R.
The new package can be found at
http://boxen.math.washington.edu/home/kirkby/patches/atlas-3.8.3.p14.spkg
I've leaving this as "needs_info" until such time as I have more fully
tested this.
--
Ticket URL: <http://trac.sagemath.org/sage_trac/ticket/9508#comment:18>
Sage <http://www.sagemath.org>
Sage: Creating a Viable Open Source Alternative to Magma, Maple, Mathematica,
and MATLAB
--
You received this message because you are subscribed to the Google Groups
"sage-trac" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/sage-trac?hl=en.
