Re: [R-SIG-Mac] Building R 4.0.2 from source via clang/xcode for MKL on macOS

2020-09-29 Thread Simon Urbanek 



> On Sep 30, 2020, at 03:32, Prof Brian Ripley  wrote:
> 
> On 29/09/2020 12:27, Kasper Daniel Hansen wrote:
>> To use veclib you need
>>--with-blas="-framework Accelerate"
> 
> Details are in the R-admin manual, including that R fails one of its checks.
> 
> Simon's calling this vecLib is a bit misleading.  'vecLib' was a vectorized 
> library for PPC Macs, and the name lives on for a small part of the 
> Accelerate framework.  The part that supplies an enhanced BLAS is different, 
> and is based on ATLAS for x86_64 CPUs.
> 

This is just the way Apple names it (now on all platforms), the name has a long 
history and was related to PPC, but Apple chose not to change it even though it 
doesn't actually contain any PowerPC code anymore. Accelerate is just a stub 
that combines vecLib and vImage, and since R only uses vecLib it is sufficient 
to link to vecLib. I'm just referring to the actual framework involved and the 
vecLib framework itself contains all the libraries including LAPACK and BLAS ( 
https://developer.apple.com/documentation/accelerate/veclib ).

Cheers,
Simon


> Incidentally, on the configuration issue -- had you followed the posting 
> guide and either read the current manual or tried R-patched you would not 
> have encountered the problem (which AFAWK is very recent, from the Command 
> Line Tools for Xcode 12, released only for rather recent versions of Catalina 
> and pre-Big Sur about 10 days ago).  Now documented at
> 
> https://developer.apple.com/documentation/xcode-release-notes/xcode-12-release-notes
> 
>> On Tue, Sep 29, 2020 at 9:53 AM roy  wrote:
>>> Hi Simon,
>>> Thanks for the info.  I was totally unaware of ABI, vecLib, etc and that
>>> Apple has blas, lapack, etc.  But after reading up on this and re-reading
>>> your email, I'm beginning to understand more about this.
>>> 
>>> So, I would like to first checkout vecLib.  From what you say, would I have
>>> to do something like the following?
>>> 
>>> ./configure --enable-BLAS-shlib --with-blas="-lBLAS"  ...
>>> 
>>> Is this also possible with LAPACK?
>>> 
>>> tx again.
>>> cheers, roy
>>> 
>>> 
>>> On Mon, Sep 28, 2020 at 3:01 PM Simon Urbanek >>> 
>>> wrote:
>>> 
 Rollin,
 
 it has been several years since I last tested MKL, so take it with a
>>> grain
 of salt, but in general you don't necessarily have to build R with MKL in
 order to use it - you only need to use --enable-BLAS-shlib and link to
>>> any
 ABI-compatible BLAS which can be vecLib as well. Then you can change the
 link from vecLib to MKL in the BLAS stub. Note that we only need the C
>>> ABI,
 there are wrappers vecLibg95f.* which re-map the F entry points to C
>>> entry
 points as to avoid Fortran ABI issues thus you don't care about the
 Fortran. However, historically, MKL has not been much more performant
>>> than
 vecLib so it's unclear if it is worth the hassle. As with any accelerated
 BLAS, note that this may have effects on results in R.
 
 Cheers,
 Simon
 
 
 
> On 28/09/2020, at 7:07 PM, rollin  wrote:
> 
> I wanted to build R from source on macos (10.15.5) so I could include
> Intel's MKL.  So I first looked at building R from source without MKL.
> 
> From the installation doc, I modified config.site to have the
>>> following:
> 
> CC=clang
> OBJC=$CC
> FC=/usr/local/bin/gfortran
> CXX=clang++
> 
> 
> I then ran configuration via the command:
> 
> ./configure -C --enable-R-shlib --enable-memory-profiling
> --x-includes=/opt/X11/include --x-libraries=/opt/X11/lib
> 
 
>>> PKG_CONFIG_PATH=/opt/X11/lib/pkgconfig:/usr/local/lib/pkgconfig:/usr/lib/pkgconfig
> 
> 
> And received the following information and error:
> 
> checking if bzip2 version >= 1.0.6... no
> checking whether bzip2 support suffices... configure: error: bzip2
 library
> and headers are required
> 
> 
> By looking at the log, I saw a compiler error due to an implicit
 function.
> I then made the following change in config.site:
> 
> CFLAGS='-Wno-implicit-function-declaration -g -O2''
> 
> 
> And configure now ran without errors.
> 
> However, when I looked at configuring to use MKL, I discovered that MKL
 on
> macos does not support gnu fortran so, unless I purchase Intel's
>>> Fortran
> compiler, it looks like I'm sol.
> 
> Has anyone built R with MKL on macos (10.15)?  In any event, I wanted
>>> to
 at
> least note the issue and work around I encountered when building R on
> macos with clang/xcode.
> 
>   [[alternative HTML version deleted]]
> 
> ___
> R-SIG-Mac mailing list
> R-SIG-Mac@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-mac
 
 
>>> 
>>> [[alternative HTML version deleted]]
>>> 
>>>

Re: [R-SIG-Mac] Building R 4.0.2 from source via clang/xcode for MKL on macOS

2020-09-29 Thread Prof Brian Ripley 

On 29/09/2020 12:27, Kasper Daniel Hansen wrote:

To use veclib you need
--with-blas="-framework Accelerate"


Details are in the R-admin manual, including that R fails one of its checks.

Simon's calling this vecLib is a bit misleading.  'vecLib' was a 
vectorized library for PPC Macs, and the name lives on for a small part 
of the Accelerate framework.  The part that supplies an enhanced BLAS is 
different, and is based on ATLAS for x86_64 CPUs.


Incidentally, on the configuration issue -- had you followed the posting 
guide and either read the current manual or tried R-patched you would 
not have encountered the problem (which AFAWK is very recent, from the 
Command Line Tools for Xcode 12, released only for rather recent 
versions of Catalina and pre-Big Sur about 10 days ago).  Now documented at


https://developer.apple.com/documentation/xcode-release-notes/xcode-12-release-notes


On Tue, Sep 29, 2020 at 9:53 AM roy  wrote:


Hi Simon,
Thanks for the info.  I was totally unaware of ABI, vecLib, etc and that
Apple has blas, lapack, etc.  But after reading up on this and re-reading
your email, I'm beginning to understand more about this.

So, I would like to first checkout vecLib.  From what you say, would I have
to do something like the following?

./configure --enable-BLAS-shlib --with-blas="-lBLAS"  ...

Is this also possible with LAPACK?

tx again.
cheers, roy


On Mon, Sep 28, 2020 at 3:01 PM Simon Urbanek 


wrote:


Rollin,

it has been several years since I last tested MKL, so take it with a

grain

of salt, but in general you don't necessarily have to build R with MKL in
order to use it - you only need to use --enable-BLAS-shlib and link to

any

ABI-compatible BLAS which can be vecLib as well. Then you can change the
link from vecLib to MKL in the BLAS stub. Note that we only need the C

ABI,

there are wrappers vecLibg95f.* which re-map the F entry points to C

entry

points as to avoid Fortran ABI issues thus you don't care about the
Fortran. However, historically, MKL has not been much more performant

than

vecLib so it's unclear if it is worth the hassle. As with any accelerated
BLAS, note that this may have effects on results in R.

Cheers,
Simon




On 28/09/2020, at 7:07 PM, rollin  wrote:

I wanted to build R from source on macos (10.15.5) so I could include
Intel's MKL.  So I first looked at building R from source without MKL.

 From the installation doc, I modified config.site to have the

following:


CC=clang
OBJC=$CC
FC=/usr/local/bin/gfortran
CXX=clang++


I then ran configuration via the command:

./configure -C --enable-R-shlib --enable-memory-profiling
--x-includes=/opt/X11/include --x-libraries=/opt/X11/lib




PKG_CONFIG_PATH=/opt/X11/lib/pkgconfig:/usr/local/lib/pkgconfig:/usr/lib/pkgconfig



And received the following information and error:

checking if bzip2 version >= 1.0.6... no
checking whether bzip2 support suffices... configure: error: bzip2

library

and headers are required


By looking at the log, I saw a compiler error due to an implicit

function.

I then made the following change in config.site:

CFLAGS='-Wno-implicit-function-declaration -g -O2''


And configure now ran without errors.

However, when I looked at configuring to use MKL, I discovered that MKL

on

macos does not support gnu fortran so, unless I purchase Intel's

Fortran

compiler, it looks like I'm sol.

Has anyone built R with MKL on macos (10.15)?  In any event, I wanted

to

at

least note the issue and work around I encountered when building R on
macos with clang/xcode.

   [[alternative HTML version deleted]]

___
R-SIG-Mac mailing list
R-SIG-Mac@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-mac





 [[alternative HTML version deleted]]

___
R-SIG-Mac mailing list
R-SIG-Mac@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-mac







--
Brian D. Ripley,  rip...@stats.ox.ac.uk
Emeritus Professor of Applied Statistics, University of Oxford

___
R-SIG-Mac mailing list
R-SIG-Mac@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-mac

Re: [R-SIG-Mac] Building R 4.0.2 from source via clang/xcode for MKL on macOS

2020-09-29 Thread Kasper Daniel Hansen 
To use veclib you need
   --with-blas="-framework Accelerate"

My full building script on OS X is currently

../${SRCDIR}/configure SHELL='/bin/bash' \
  --prefix=/usr/local/R/${R_VERSION} --disable-R-framework\
  CC="/usr/local/clang8/bin/clang" \
  CXX="/usr/local/clang8/bin/clang++" \
  F77="/usr/local/gfortran/bin/gfortran" \
  FC="/usr/local/gfortran/bin/gfortran" \
  OBJC="clang" \
  CFLAGS="-Wall -mtune=native -g -O2" \
  CXXFLAGS="-Wall -mtune=native -g -O2" \
  OBJCFLAGS="-Wall -mtune=native -g -O2 -fobjc-exceptions" \
  F77FLAGS="-Wall -g -O2 -mtune=generic" \
  FCFLAGS="$F77FLAGS" \
  FLIBS="-L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin15/6.1.0
-L/usr/local/gfortran/lib -lgfortran -lquadmath -lm" \
  LDFLAGS="-L/usr/local/clang8/lib -L/usr/local/lib" \
  DYLD_FALLBACK_LIBRARY_PATH="/usr/local/clang8/lib:/usr/local/lib" \

PKG_CONFIG_PATH="/opt/X11/lib/pkgconfig:/usr/local/lib/pkgconfig:/usr/lib/pkgconfig"
\
  JAVA_HOME="/System/Library/Frameworks/JavaVM.framework/Home" \
  JAVA_CPPFLAGS="-I/System/Library/Frameworks/JavaVM.framework/Headers" \
  JAVA_LD_LIBRARY_PATH="" \
  JAVA_LIBS="-framework JavaVM" \
  --enable-memory-profiling\
  --x-includes=/opt/X11/include --x-libraries=/opt/X11/lib\
  --with-blas="-framework Accelerate" | tee ../configure-${R_VERSION}

although I think we have moved back to Apple's clang for C etc compiler for
the CRAN version of R instead of Simon's clang8.

Best,
Kasper


On Tue, Sep 29, 2020 at 9:53 AM roy  wrote:

> Hi Simon,
> Thanks for the info.  I was totally unaware of ABI, vecLib, etc and that
> Apple has blas, lapack, etc.  But after reading up on this and re-reading
> your email, I'm beginning to understand more about this.
>
> So, I would like to first checkout vecLib.  From what you say, would I have
> to do something like the following?
>
> ./configure --enable-BLAS-shlib --with-blas="-lBLAS"  ...
>
> Is this also possible with LAPACK?
>
> tx again.
> cheers, roy
>
>
> On Mon, Sep 28, 2020 at 3:01 PM Simon Urbanek  >
> wrote:
>
> > Rollin,
> >
> > it has been several years since I last tested MKL, so take it with a
> grain
> > of salt, but in general you don't necessarily have to build R with MKL in
> > order to use it - you only need to use --enable-BLAS-shlib and link to
> any
> > ABI-compatible BLAS which can be vecLib as well. Then you can change the
> > link from vecLib to MKL in the BLAS stub. Note that we only need the C
> ABI,
> > there are wrappers vecLibg95f.* which re-map the F entry points to C
> entry
> > points as to avoid Fortran ABI issues thus you don't care about the
> > Fortran. However, historically, MKL has not been much more performant
> than
> > vecLib so it's unclear if it is worth the hassle. As with any accelerated
> > BLAS, note that this may have effects on results in R.
> >
> > Cheers,
> > Simon
> >
> >
> >
> > > On 28/09/2020, at 7:07 PM, rollin  wrote:
> > >
> > > I wanted to build R from source on macos (10.15.5) so I could include
> > > Intel's MKL.  So I first looked at building R from source without MKL.
> > >
> > > From the installation doc, I modified config.site to have the
> following:
> > >
> > > CC=clang
> > > OBJC=$CC
> > > FC=/usr/local/bin/gfortran
> > > CXX=clang++
> > >
> > >
> > > I then ran configuration via the command:
> > >
> > > ./configure -C --enable-R-shlib --enable-memory-profiling
> > > --x-includes=/opt/X11/include --x-libraries=/opt/X11/lib
> > >
> >
> PKG_CONFIG_PATH=/opt/X11/lib/pkgconfig:/usr/local/lib/pkgconfig:/usr/lib/pkgconfig
> > >
> > >
> > > And received the following information and error:
> > >
> > > checking if bzip2 version >= 1.0.6... no
> > > checking whether bzip2 support suffices... configure: error: bzip2
> > library
> > > and headers are required
> > >
> > >
> > > By looking at the log, I saw a compiler error due to an implicit
> > function.
> > > I then made the following change in config.site:
> > >
> > > CFLAGS='-Wno-implicit-function-declaration -g -O2''
> > >
> > >
> > > And configure now ran without errors.
> > >
> > > However, when I looked at configuring to use MKL, I discovered that MKL
> > on
> > > macos does not support gnu fortran so, unless I purchase Intel's
> Fortran
> > > compiler, it looks like I'm sol.
> > >
> > > Has anyone built R with MKL on macos (10.15)?  In any event, I wanted
> to
> > at
> > > least note the issue and work around I encountered when building R on
> > > macos with clang/xcode.
> > >
> > >   [[alternative HTML version deleted]]
> > >
> > > ___
> > > R-SIG-Mac mailing list
> > > R-SIG-Mac@r-project.org
> > > https://stat.ethz.ch/mailman/listinfo/r-sig-mac
> >
> >
>
> [[alternative HTML version deleted]]
>
> ___
> R-SIG-Mac mailing list
> R-SIG-Mac@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-mac
>


-- 
Best,
Kasper

[[alternative HTML version deleted]]

___
R-SIG-Mac

Re: [R-SIG-Mac] Building R 4.0.2 from source via clang/xcode for MKL on macOS

2020-09-29 Thread Kasper Daniel Hansen 
In my understanding, almost all of the speed up comes from using an
optimized BLAS. Optimizing LAPACK tend to have negligible impact on
performance. At least that is my impression, although I won't say I'm an
expert.

Best,
Kasper

On Tue, Sep 29, 2020 at 9:53 AM roy  wrote:

> Hi Simon,
> Thanks for the info.  I was totally unaware of ABI, vecLib, etc and that
> Apple has blas, lapack, etc.  But after reading up on this and re-reading
> your email, I'm beginning to understand more about this.
>
> So, I would like to first checkout vecLib.  From what you say, would I have
> to do something like the following?
>
> ./configure --enable-BLAS-shlib --with-blas="-lBLAS"  ...
>
> Is this also possible with LAPACK?
>
> tx again.
> cheers, roy
>
>
> On Mon, Sep 28, 2020 at 3:01 PM Simon Urbanek  >
> wrote:
>
> > Rollin,
> >
> > it has been several years since I last tested MKL, so take it with a
> grain
> > of salt, but in general you don't necessarily have to build R with MKL in
> > order to use it - you only need to use --enable-BLAS-shlib and link to
> any
> > ABI-compatible BLAS which can be vecLib as well. Then you can change the
> > link from vecLib to MKL in the BLAS stub. Note that we only need the C
> ABI,
> > there are wrappers vecLibg95f.* which re-map the F entry points to C
> entry
> > points as to avoid Fortran ABI issues thus you don't care about the
> > Fortran. However, historically, MKL has not been much more performant
> than
> > vecLib so it's unclear if it is worth the hassle. As with any accelerated
> > BLAS, note that this may have effects on results in R.
> >
> > Cheers,
> > Simon
> >
> >
> >
> > > On 28/09/2020, at 7:07 PM, rollin  wrote:
> > >
> > > I wanted to build R from source on macos (10.15.5) so I could include
> > > Intel's MKL.  So I first looked at building R from source without MKL.
> > >
> > > From the installation doc, I modified config.site to have the
> following:
> > >
> > > CC=clang
> > > OBJC=$CC
> > > FC=/usr/local/bin/gfortran
> > > CXX=clang++
> > >
> > >
> > > I then ran configuration via the command:
> > >
> > > ./configure -C --enable-R-shlib --enable-memory-profiling
> > > --x-includes=/opt/X11/include --x-libraries=/opt/X11/lib
> > >
> >
> PKG_CONFIG_PATH=/opt/X11/lib/pkgconfig:/usr/local/lib/pkgconfig:/usr/lib/pkgconfig
> > >
> > >
> > > And received the following information and error:
> > >
> > > checking if bzip2 version >= 1.0.6... no
> > > checking whether bzip2 support suffices... configure: error: bzip2
> > library
> > > and headers are required
> > >
> > >
> > > By looking at the log, I saw a compiler error due to an implicit
> > function.
> > > I then made the following change in config.site:
> > >
> > > CFLAGS='-Wno-implicit-function-declaration -g -O2''
> > >
> > >
> > > And configure now ran without errors.
> > >
> > > However, when I looked at configuring to use MKL, I discovered that MKL
> > on
> > > macos does not support gnu fortran so, unless I purchase Intel's
> Fortran
> > > compiler, it looks like I'm sol.
> > >
> > > Has anyone built R with MKL on macos (10.15)?  In any event, I wanted
> to
> > at
> > > least note the issue and work around I encountered when building R on
> > > macos with clang/xcode.
> > >
> > >   [[alternative HTML version deleted]]
> > >
> > > ___
> > > R-SIG-Mac mailing list
> > > R-SIG-Mac@r-project.org
> > > https://stat.ethz.ch/mailman/listinfo/r-sig-mac
> >
> >
>
> [[alternative HTML version deleted]]
>
> ___
> R-SIG-Mac mailing list
> R-SIG-Mac@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-mac
>


-- 
Best,
Kasper

[[alternative HTML version deleted]]

___
R-SIG-Mac mailing list
R-SIG-Mac@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-mac

Re: [R-SIG-Mac] Building R 4.0.2 from source via clang/xcode for MKL on macOS

2020-09-29 Thread roy 
Hi Simon,
Thanks for the info.  I was totally unaware of ABI, vecLib, etc and that
Apple has blas, lapack, etc.  But after reading up on this and re-reading
your email, I'm beginning to understand more about this.

So, I would like to first checkout vecLib.  From what you say, would I have
to do something like the following?

./configure --enable-BLAS-shlib --with-blas="-lBLAS"  ...

Is this also possible with LAPACK?

tx again.
cheers, roy


On Mon, Sep 28, 2020 at 3:01 PM Simon Urbanek 
wrote:

> Rollin,
>
> it has been several years since I last tested MKL, so take it with a grain
> of salt, but in general you don't necessarily have to build R with MKL in
> order to use it - you only need to use --enable-BLAS-shlib and link to any
> ABI-compatible BLAS which can be vecLib as well. Then you can change the
> link from vecLib to MKL in the BLAS stub. Note that we only need the C ABI,
> there are wrappers vecLibg95f.* which re-map the F entry points to C entry
> points as to avoid Fortran ABI issues thus you don't care about the
> Fortran. However, historically, MKL has not been much more performant than
> vecLib so it's unclear if it is worth the hassle. As with any accelerated
> BLAS, note that this may have effects on results in R.
>
> Cheers,
> Simon
>
>
>
> > On 28/09/2020, at 7:07 PM, rollin  wrote:
> >
> > I wanted to build R from source on macos (10.15.5) so I could include
> > Intel's MKL.  So I first looked at building R from source without MKL.
> >
> > From the installation doc, I modified config.site to have the following:
> >
> > CC=clang
> > OBJC=$CC
> > FC=/usr/local/bin/gfortran
> > CXX=clang++
> >
> >
> > I then ran configuration via the command:
> >
> > ./configure -C --enable-R-shlib --enable-memory-profiling
> > --x-includes=/opt/X11/include --x-libraries=/opt/X11/lib
> >
> PKG_CONFIG_PATH=/opt/X11/lib/pkgconfig:/usr/local/lib/pkgconfig:/usr/lib/pkgconfig
> >
> >
> > And received the following information and error:
> >
> > checking if bzip2 version >= 1.0.6... no
> > checking whether bzip2 support suffices... configure: error: bzip2
> library
> > and headers are required
> >
> >
> > By looking at the log, I saw a compiler error due to an implicit
> function.
> > I then made the following change in config.site:
> >
> > CFLAGS='-Wno-implicit-function-declaration -g -O2''
> >
> >
> > And configure now ran without errors.
> >
> > However, when I looked at configuring to use MKL, I discovered that MKL
> on
> > macos does not support gnu fortran so, unless I purchase Intel's Fortran
> > compiler, it looks like I'm sol.
> >
> > Has anyone built R with MKL on macos (10.15)?  In any event, I wanted to
> at
> > least note the issue and work around I encountered when building R on
> > macos with clang/xcode.
> >
> >   [[alternative HTML version deleted]]
> >
> > ___
> > R-SIG-Mac mailing list
> > R-SIG-Mac@r-project.org
> > https://stat.ethz.ch/mailman/listinfo/r-sig-mac
>
>

[[alternative HTML version deleted]]

___
R-SIG-Mac mailing list
R-SIG-Mac@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-mac