Re: [R-SIG-Mac] OpenMP on CRAN (Simon Urbanek)

[Simon Urbanek]

JJB,

good idea, I didn't think of it, I have now moved this one and the recipes repo 
to

https://github.com/R-macos

Cheers,
Simon


> On 2/05/2020, at 4:32 AM, Balamuta, James Joseph  
> wrote:
> 
> Simon,
> 
> Thank you for the official instructions on using OpenMP with R 4.0.0!
> 
> Also, thanks for making the source behind mac.r-project.org available on 
> GitHub! 
> https://github.com/s-u/R-mac-dev
> 
> Would you consider creating an organization to house all repositories related 
> to R for macOS in a manner similar to R for Windows?
> 
> e.g. https://github.com/r-windows 
> 
> Best,
> 
> JJB
> 
> On 4/30/20, 8:10 PM, "R-SIG-Mac on behalf of Simon Urbanek" 
>  
> wrote:
> 
>I have now created a page about this:
> 
>http://mac.r-project.org/openmp
> 
>which also provides libomp binaries for all recent versions of Xcode (and 
> more).
> 
>Important note: this is about Xcode - it is NOT about using Homebrew tools 
> nor the custom compilers we used before R 4.0.0. Both of the latter are not 
> suitable for use with R 4.0.0 binaries.
> 
>Please give it a shot. Thanks for those participating in the discussion.
> 
>Cheers,
>Simon
> 
> 
> 
>> On 1/05/2020, at 2:23 AM, Wright, Erik Scott  wrote:
>> 
>> Hi Kevin et al.,
>> 
>> The setup you suggested did not work for me, but I was able to get OpenMP to 
>> work on Mac (10.13) with R v4.0 packages.  Here's what I did:
>> 
>> (1) After installing Homebrew, ran
>> brew install libomp
>> # Note it is also possible to install from OpenMP Source code: 
>> https://releases.llvm.org/download.html#10.0.0
>> (2) Determined the install path with
>> brew --prefix libomp
>> # In my case /usr/local/opt/libomp
>> (3) Added lines to ~/.R/Makevars
>> CC=/usr/local/clang4/bin/clang -fopenmp -I/usr/local/opt/libomp/include
>> LDFLAGS=-L/usr/local/opt/libomp/lib
>> 
>> I imagine the process is similar for C++ code by setting CXX.  Now 
>> parallelization for C code with OpenMP is working again on Mac.
>> 
>> As always, I am thankful to the R community for helping problem solve.
>> 
>> I wish more direction along these lines could be added to mac.r-project.org
>> 
>> Erik
>> 
>> 
>>> On Apr 29, 2020, at 1:51 AM, Dmitriy Selivanov 
>>>  wrote:
>>> 
>>> Thanks, Kevin, I can confirm suggested setup works fine so far. Thank you!
>>> 
>>> I understand it, is that there's
>>> no guarantee that this will work properly (or continue to work
>>> properly) as the Apple toolchain continues to be updated -- e.g. a new
>>> version of macOS / Xcode could install a version of Apple Clang that
>>> is then incompatible with the version of libomp currently in use. In
>>> such a case, I suspect one would need to find and reinstall libomp.
>>> 
>>> Thats understandable. 
>>> 
>>> On Sat, Apr 25, 2020 at 8:09 PM Kevin Ushey  wrote:
>>> The following works for me on macOS 10.15.4 using Apple Clang + libomp
>>> from LLVM 10 (via Homebrew).
>>> 
>>> Install libomp from Homebrew with:
>>> 
>>>brew install libomp
>>> 
>>> Then, put the following in ~/.R/Makevars
>>> 
>>>CPPFLAGS += -I/usr/local/opt/libomp/include -Xclang -fopenmp
>>>LDFLAGS += -L/usr/local/opt/libomp/lib -lomp
>>> 
>>> Then R will automatically enable + use OpenMP as appropriate when
>>> installing packages from sources.
>>> 
>>> The main danger of this approach, as I understand it, is that there's
>>> no guarantee that this will work properly (or continue to work
>>> properly) as the Apple toolchain continues to be updated -- e.g. a new
>>> version of macOS / Xcode could install a version of Apple Clang that
>>> is then incompatible with the version of libomp currently in use. In
>>> such a case, I suspect one would need to find and reinstall libomp.
>>> 
>>> In theory, this could be alleviated by ensuring all users download and
>>> use the same version of Xcode as is being used by the macOS build
>>> machine (Xcode 10.1), but in practice users will likely just be using
>>> the "default" set of command line tools that comes with their version
>>> of macOS.
>>> 
>>> Cunningham's law will hopefully ensure someone else will chime in if
>>> I've got something wrong :-)
>>> 
>>> Best,
>>> Kevin
>>> 
>>> On Sat, Apr 25, 2020 at 8:29 AM Wright, Erik Scott  
>>> wrote:
 
 I second this request.  Dropping OpenMP support in R v4.0 on Mac is 
 unfortunate.  My R package is highly parallelized via OpenMP, and many of 
 my end-users take advantage of the excellent speedups.  It would be much 
 appreciated if user-level instructions could be provided for how to enable 
 OpenMP support on the Mac.
 
 Thanks,
 Erik
 
 
> On Apr 23, 2020, at 7:08 AM, Dmitriy Selivanov 
>  wrote:
> 
> Hi Simon,
> 
> Just wanted follow up on this topic.
> 
> It would be very helpful if you can provide some guide on
> https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmac.r-project.org%2F&data=02%7C01%7Ceswright%40pitt.edu%7C65097590dea445

Re: [R-SIG-Mac] OpenMP on CRAN (Simon Urbanek)

[Manuel Spínola]

That will be a great idea. I don’t know much of programming and I get
confused with so many technical details.

Manuel Spínola

On Fri, May 1, 2020 at 10:32 Balamuta, James Joseph 
wrote:

> Simon,
>
> Thank you for the official instructions on using OpenMP with R 4.0.0!
>
> Also, thanks for making the source behind mac.r-project.org available on
> GitHub!
> https://github.com/s-u/R-mac-dev
>
> Would you consider creating an organization to house all repositories
> related to R for macOS in a manner similar to R for Windows?
>
> e.g. https://github.com/r-windows
>
> Best,
>
> JJB
>
> On 4/30/20, 8:10 PM, "R-SIG-Mac on behalf of Simon Urbanek" <
> r-sig-mac-boun...@r-project.org on behalf of simon.urba...@r-project.org>
> wrote:
>
> I have now created a page about this:
>
> http://mac.r-project.org/openmp
>
> which also provides libomp binaries for all recent versions of Xcode
> (and more).
>
> Important note: this is about Xcode - it is NOT about using Homebrew
> tools nor the custom compilers we used before R 4.0.0. Both of the latter
> are not suitable for use with R 4.0.0 binaries.
>
> Please give it a shot. Thanks for those participating in the
> discussion.
>
> Cheers,
> Simon
>
>
>
> > On 1/05/2020, at 2:23 AM, Wright, Erik Scott 
> wrote:
> >
> > Hi Kevin et al.,
> >
> > The setup you suggested did not work for me, but I was able to get
> OpenMP to work on Mac (10.13) with R v4.0 packages.  Here's what I did:
> >
> > (1) After installing Homebrew, ran
> > brew install libomp
> > # Note it is also possible to install from OpenMP Source code:
> https://releases.llvm.org/download.html#10.0.0
> > (2) Determined the install path with
> > brew --prefix libomp
> > # In my case /usr/local/opt/libomp
> > (3) Added lines to ~/.R/Makevars
> > CC=/usr/local/clang4/bin/clang -fopenmp
> -I/usr/local/opt/libomp/include
> > LDFLAGS=-L/usr/local/opt/libomp/lib
> >
> > I imagine the process is similar for C++ code by setting CXX.  Now
> parallelization for C code with OpenMP is working again on Mac.
> >
> > As always, I am thankful to the R community for helping problem
> solve.
> >
> > I wish more direction along these lines could be added to
> mac.r-project.org
> >
> > Erik
> >
> >
> >> On Apr 29, 2020, at 1:51 AM, Dmitriy Selivanov <
> selivanov.dmit...@gmail.com> wrote:
> >>
> >> Thanks, Kevin, I can confirm suggested setup works fine so far.
> Thank you!
> >>
> >> I understand it, is that there's
> >> no guarantee that this will work properly (or continue to work
> >> properly) as the Apple toolchain continues to be updated -- e.g. a
> new
> >> version of macOS / Xcode could install a version of Apple Clang that
> >> is then incompatible with the version of libomp currently in use. In
> >> such a case, I suspect one would need to find and reinstall libomp.
> >>
> >> Thats understandable.
> >>
> >> On Sat, Apr 25, 2020 at 8:09 PM Kevin Ushey 
> wrote:
> >> The following works for me on macOS 10.15.4 using Apple Clang +
> libomp
> >> from LLVM 10 (via Homebrew).
> >>
> >> Install libomp from Homebrew with:
> >>
> >> brew install libomp
> >>
> >> Then, put the following in ~/.R/Makevars
> >>
> >> CPPFLAGS += -I/usr/local/opt/libomp/include -Xclang -fopenmp
> >> LDFLAGS += -L/usr/local/opt/libomp/lib -lomp
> >>
> >> Then R will automatically enable + use OpenMP as appropriate when
> >> installing packages from sources.
> >>
> >> The main danger of this approach, as I understand it, is that
> there's
> >> no guarantee that this will work properly (or continue to work
> >> properly) as the Apple toolchain continues to be updated -- e.g. a
> new
> >> version of macOS / Xcode could install a version of Apple Clang that
> >> is then incompatible with the version of libomp currently in use. In
> >> such a case, I suspect one would need to find and reinstall libomp.
> >>
> >> In theory, this could be alleviated by ensuring all users download
> and
> >> use the same version of Xcode as is being used by the macOS build
> >> machine (Xcode 10.1), but in practice users will likely just be
> using
> >> the "default" set of command line tools that comes with their
> version
> >> of macOS.
> >>
> >> Cunningham's law will hopefully ensure someone else will chime in if
> >> I've got something wrong :-)
> >>
> >> Best,
> >> Kevin
> >>
> >> On Sat, Apr 25, 2020 at 8:29 AM Wright, Erik Scott <
> eswri...@pitt.edu> wrote:
> >> >
> >> > I second this request.  Dropping OpenMP support in R v4.0 on Mac
> is unfortunate.  My R package is highly parallelized via OpenMP, and many
> of my end-users take advantage of the excellent speedups.  It would be much
> appreciated if user-level instructions could 

Re: [R-SIG-Mac] OpenMP on CRAN (Simon Urbanek)

[Balamuta, James Joseph]

Simon,

Thank you for the official instructions on using OpenMP with R 4.0.0!

Also, thanks for making the source behind mac.r-project.org available on 
GitHub! 
https://github.com/s-u/R-mac-dev

Would you consider creating an organization to house all repositories related 
to R for macOS in a manner similar to R for Windows?

e.g. https://github.com/r-windows 

Best,

JJB

On 4/30/20, 8:10 PM, "R-SIG-Mac on behalf of Simon Urbanek" 
 
wrote:

I have now created a page about this:

http://mac.r-project.org/openmp

which also provides libomp binaries for all recent versions of Xcode (and 
more).

Important note: this is about Xcode - it is NOT about using Homebrew tools 
nor the custom compilers we used before R 4.0.0. Both of the latter are not 
suitable for use with R 4.0.0 binaries.

Please give it a shot. Thanks for those participating in the discussion.

Cheers,
Simon



> On 1/05/2020, at 2:23 AM, Wright, Erik Scott  wrote:
> 
> Hi Kevin et al.,
> 
> The setup you suggested did not work for me, but I was able to get OpenMP 
to work on Mac (10.13) with R v4.0 packages.  Here's what I did:
> 
> (1) After installing Homebrew, ran
> brew install libomp
> # Note it is also possible to install from OpenMP Source code: 
https://releases.llvm.org/download.html#10.0.0
> (2) Determined the install path with
> brew --prefix libomp
> # In my case /usr/local/opt/libomp
> (3) Added lines to ~/.R/Makevars
> CC=/usr/local/clang4/bin/clang -fopenmp -I/usr/local/opt/libomp/include
> LDFLAGS=-L/usr/local/opt/libomp/lib
> 
> I imagine the process is similar for C++ code by setting CXX.  Now 
parallelization for C code with OpenMP is working again on Mac.
> 
> As always, I am thankful to the R community for helping problem solve.
> 
> I wish more direction along these lines could be added to 
mac.r-project.org
> 
> Erik
> 
> 
>> On Apr 29, 2020, at 1:51 AM, Dmitriy Selivanov 
 wrote:
>> 
>> Thanks, Kevin, I can confirm suggested setup works fine so far. Thank 
you!
>> 
>> I understand it, is that there's
>> no guarantee that this will work properly (or continue to work
>> properly) as the Apple toolchain continues to be updated -- e.g. a new
>> version of macOS / Xcode could install a version of Apple Clang that
>> is then incompatible with the version of libomp currently in use. In
>> such a case, I suspect one would need to find and reinstall libomp.
>> 
>> Thats understandable. 
>> 
>> On Sat, Apr 25, 2020 at 8:09 PM Kevin Ushey  wrote:
>> The following works for me on macOS 10.15.4 using Apple Clang + libomp
>> from LLVM 10 (via Homebrew).
>> 
>> Install libomp from Homebrew with:
>> 
>> brew install libomp
>> 
>> Then, put the following in ~/.R/Makevars
>> 
>> CPPFLAGS += -I/usr/local/opt/libomp/include -Xclang -fopenmp
>> LDFLAGS += -L/usr/local/opt/libomp/lib -lomp
>> 
>> Then R will automatically enable + use OpenMP as appropriate when
>> installing packages from sources.
>> 
>> The main danger of this approach, as I understand it, is that there's
>> no guarantee that this will work properly (or continue to work
>> properly) as the Apple toolchain continues to be updated -- e.g. a new
>> version of macOS / Xcode could install a version of Apple Clang that
>> is then incompatible with the version of libomp currently in use. In
>> such a case, I suspect one would need to find and reinstall libomp.
>> 
>> In theory, this could be alleviated by ensuring all users download and
>> use the same version of Xcode as is being used by the macOS build
>> machine (Xcode 10.1), but in practice users will likely just be using
>> the "default" set of command line tools that comes with their version
>> of macOS.
>> 
>> Cunningham's law will hopefully ensure someone else will chime in if
>> I've got something wrong :-)
>> 
>> Best,
>> Kevin
>> 
>> On Sat, Apr 25, 2020 at 8:29 AM Wright, Erik Scott  
wrote:
>> >
>> > I second this request.  Dropping OpenMP support in R v4.0 on Mac is 
unfortunate.  My R package is highly parallelized via OpenMP, and many of my 
end-users take advantage of the excellent speedups.  It would be much 
appreciated if user-level instructions could be provided for how to enable 
OpenMP support on the Mac.
>> >
>> > Thanks,
>> > Erik
>> >
>> >
>> > > On Apr 23, 2020, at 7:08 AM, Dmitriy Selivanov 
 wrote:
>> > >
>> > > Hi Simon,
>> > >
>> > > Just wanted follow up on this topic.
>> > >
>> > > It would be very helpful if you can provide some guide on
>> > > 
https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmac.r-project.org%2F&data=02%7C01%7Ceswright%40pitt.edu%7C65097590dea4453ab8cf08d7e776b868%7C9ef9

Re: [R-SIG-Mac] OpenMP on CRAN (Simon Urbanek)

[Dmitriy Selivanov]

Thank you Simon for the official instruction and explanation about caveats!
I've tried to use libomp (9.0.1) you've provided and everything works
smoothly.
With new R4.0 toolchain and optional openmp support we got really good
tradeoff between simplicity for the majority of the users and feature-rich
set-up for more tech-savvy users.


On Fri, May 1, 2020 at 5:29 AM Simon Urbanek 
wrote:

> I have now created a page about this:
>
> http://mac.r-project.org/openmp
>
> which also provides libomp binaries for all recent versions of Xcode (and
> more).
>
> Important note: this is about Xcode - it is NOT about using Homebrew tools
> nor the custom compilers we used before R 4.0.0. Both of the latter are not
> suitable for use with R 4.0.0 binaries.
>
> Please give it a shot. Thanks for those participating in the discussion.
>
> Cheers,
> Simon
>
>
>
> > On 1/05/2020, at 2:23 AM, Wright, Erik Scott  wrote:
> >
> > Hi Kevin et al.,
> >
> > The setup you suggested did not work for me, but I was able to get
> OpenMP to work on Mac (10.13) with R v4.0 packages.  Here's what I did:
> >
> > (1) After installing Homebrew, ran
> > brew install libomp
> > # Note it is also possible to install from OpenMP Source code:
> https://releases.llvm.org/download.html#10.0.0
> > (2) Determined the install path with
> > brew --prefix libomp
> > # In my case /usr/local/opt/libomp
> > (3) Added lines to ~/.R/Makevars
> > CC=/usr/local/clang4/bin/clang -fopenmp -I/usr/local/opt/libomp/include
> > LDFLAGS=-L/usr/local/opt/libomp/lib
> >
> > I imagine the process is similar for C++ code by setting CXX.  Now
> parallelization for C code with OpenMP is working again on Mac.
> >
> > As always, I am thankful to the R community for helping problem solve.
> >
> > I wish more direction along these lines could be added to
> mac.r-project.org
> >
> > Erik
> >
> >
> >> On Apr 29, 2020, at 1:51 AM, Dmitriy Selivanov <
> selivanov.dmit...@gmail.com> wrote:
> >>
> >> Thanks, Kevin, I can confirm suggested setup works fine so far. Thank
> you!
> >>
> >> I understand it, is that there's
> >> no guarantee that this will work properly (or continue to work
> >> properly) as the Apple toolchain continues to be updated -- e.g. a new
> >> version of macOS / Xcode could install a version of Apple Clang that
> >> is then incompatible with the version of libomp currently in use. In
> >> such a case, I suspect one would need to find and reinstall libomp.
> >>
> >> Thats understandable.
> >>
> >> On Sat, Apr 25, 2020 at 8:09 PM Kevin Ushey 
> wrote:
> >> The following works for me on macOS 10.15.4 using Apple Clang + libomp
> >> from LLVM 10 (via Homebrew).
> >>
> >> Install libomp from Homebrew with:
> >>
> >> brew install libomp
> >>
> >> Then, put the following in ~/.R/Makevars
> >>
> >> CPPFLAGS += -I/usr/local/opt/libomp/include -Xclang -fopenmp
> >> LDFLAGS += -L/usr/local/opt/libomp/lib -lomp
> >>
> >> Then R will automatically enable + use OpenMP as appropriate when
> >> installing packages from sources.
> >>
> >> The main danger of this approach, as I understand it, is that there's
> >> no guarantee that this will work properly (or continue to work
> >> properly) as the Apple toolchain continues to be updated -- e.g. a new
> >> version of macOS / Xcode could install a version of Apple Clang that
> >> is then incompatible with the version of libomp currently in use. In
> >> such a case, I suspect one would need to find and reinstall libomp.
> >>
> >> In theory, this could be alleviated by ensuring all users download and
> >> use the same version of Xcode as is being used by the macOS build
> >> machine (Xcode 10.1), but in practice users will likely just be using
> >> the "default" set of command line tools that comes with their version
> >> of macOS.
> >>
> >> Cunningham's law will hopefully ensure someone else will chime in if
> >> I've got something wrong :-)
> >>
> >> Best,
> >> Kevin
> >>
> >> On Sat, Apr 25, 2020 at 8:29 AM Wright, Erik Scott 
> wrote:
> >> >
> >> > I second this request.  Dropping OpenMP support in R v4.0 on Mac is
> unfortunate.  My R package is highly parallelized via OpenMP, and many of
> my end-users take advantage of the excellent speedups.  It would be much
> appreciated if user-level instructions could be provided for how to enable
> OpenMP support on the Mac.
> >> >
> >> > Thanks,
> >> > Erik
> >> >
> >> >
> >> > > On Apr 23, 2020, at 7:08 AM, Dmitriy Selivanov <
> selivanov.dmit...@gmail.com> wrote:
> >> > >
> >> > > Hi Simon,
> >> > >
> >> > > Just wanted follow up on this topic.
> >> > >
> >> > > It would be very helpful if you can provide some guide on
> >> > >
> https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmac.r-project.org%2F&data=02%7C01%7Ceswright%40pitt.edu%7C65097590dea4453ab8cf08d7e776b868%7C9ef9f489e0a04eeb87cc3a526112fd0d%7C1%7C0%7C637232369415901734&sdata=qlhXuEubgD6PNmkbD%2BWQy5OI9X%2BGjxSYHMNjA%2BQHZzI%3D&reserved=0
> <
> https://nam05.safelinks.protection.outlook.com/?url=h

Re: [R-SIG-Mac] OpenMP on CRAN (Simon Urbanek)

[Simon Urbanek]

I have now created a page about this:

http://mac.r-project.org/openmp

which also provides libomp binaries for all recent versions of Xcode (and more).

Important note: this is about Xcode - it is NOT about using Homebrew tools nor 
the custom compilers we used before R 4.0.0. Both of the latter are not 
suitable for use with R 4.0.0 binaries.

Please give it a shot. Thanks for those participating in the discussion.

Cheers,
Simon



> On 1/05/2020, at 2:23 AM, Wright, Erik Scott  wrote:
> 
> Hi Kevin et al.,
> 
> The setup you suggested did not work for me, but I was able to get OpenMP to 
> work on Mac (10.13) with R v4.0 packages.  Here's what I did:
> 
> (1) After installing Homebrew, ran
> brew install libomp
> # Note it is also possible to install from OpenMP Source code: 
> https://releases.llvm.org/download.html#10.0.0
> (2) Determined the install path with
> brew --prefix libomp
> # In my case /usr/local/opt/libomp
> (3) Added lines to ~/.R/Makevars
> CC=/usr/local/clang4/bin/clang -fopenmp -I/usr/local/opt/libomp/include
> LDFLAGS=-L/usr/local/opt/libomp/lib
> 
> I imagine the process is similar for C++ code by setting CXX.  Now 
> parallelization for C code with OpenMP is working again on Mac.
> 
> As always, I am thankful to the R community for helping problem solve.
> 
> I wish more direction along these lines could be added to mac.r-project.org
> 
> Erik
> 
> 
>> On Apr 29, 2020, at 1:51 AM, Dmitriy Selivanov  
>> wrote:
>> 
>> Thanks, Kevin, I can confirm suggested setup works fine so far. Thank you!
>> 
>> I understand it, is that there's
>> no guarantee that this will work properly (or continue to work
>> properly) as the Apple toolchain continues to be updated -- e.g. a new
>> version of macOS / Xcode could install a version of Apple Clang that
>> is then incompatible with the version of libomp currently in use. In
>> such a case, I suspect one would need to find and reinstall libomp.
>> 
>> Thats understandable. 
>> 
>> On Sat, Apr 25, 2020 at 8:09 PM Kevin Ushey  wrote:
>> The following works for me on macOS 10.15.4 using Apple Clang + libomp
>> from LLVM 10 (via Homebrew).
>> 
>> Install libomp from Homebrew with:
>> 
>> brew install libomp
>> 
>> Then, put the following in ~/.R/Makevars
>> 
>> CPPFLAGS += -I/usr/local/opt/libomp/include -Xclang -fopenmp
>> LDFLAGS += -L/usr/local/opt/libomp/lib -lomp
>> 
>> Then R will automatically enable + use OpenMP as appropriate when
>> installing packages from sources.
>> 
>> The main danger of this approach, as I understand it, is that there's
>> no guarantee that this will work properly (or continue to work
>> properly) as the Apple toolchain continues to be updated -- e.g. a new
>> version of macOS / Xcode could install a version of Apple Clang that
>> is then incompatible with the version of libomp currently in use. In
>> such a case, I suspect one would need to find and reinstall libomp.
>> 
>> In theory, this could be alleviated by ensuring all users download and
>> use the same version of Xcode as is being used by the macOS build
>> machine (Xcode 10.1), but in practice users will likely just be using
>> the "default" set of command line tools that comes with their version
>> of macOS.
>> 
>> Cunningham's law will hopefully ensure someone else will chime in if
>> I've got something wrong :-)
>> 
>> Best,
>> Kevin
>> 
>> On Sat, Apr 25, 2020 at 8:29 AM Wright, Erik Scott  wrote:
>> >
>> > I second this request.  Dropping OpenMP support in R v4.0 on Mac is 
>> > unfortunate.  My R package is highly parallelized via OpenMP, and many of 
>> > my end-users take advantage of the excellent speedups.  It would be much 
>> > appreciated if user-level instructions could be provided for how to enable 
>> > OpenMP support on the Mac.
>> >
>> > Thanks,
>> > Erik
>> >
>> >
>> > > On Apr 23, 2020, at 7:08 AM, Dmitriy Selivanov 
>> > >  wrote:
>> > >
>> > > Hi Simon,
>> > >
>> > > Just wanted follow up on this topic.
>> > >
>> > > It would be very helpful if you can provide some guide on
>> > > https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmac.r-project.org%2F&data=02%7C01%7Ceswright%40pitt.edu%7C65097590dea4453ab8cf08d7e776b868%7C9ef9f489e0a04eeb87cc3a526112fd0d%7C1%7C0%7C637232369415901734&sdata=qlhXuEubgD6PNmkbD%2BWQy5OI9X%2BGjxSYHMNjA%2BQHZzI%3D&reserved=0
>> > >  for those users
>> > > (advanced?) and developers who wants to be able to use OpenMP on mac. 
>> > > From
>> > > what I've understood from this mail thread the easiest way is to install 
>> > > R
>> > > from homebrew as it is built with non-standard Apple toolchain.
>> > > Apart from that you've mentioned you may consider to bundle binary  iomp
>> > > with R installatio

Re: [R-SIG-Mac] OpenMP on CRAN (Simon Urbanek)

[Wright, Erik Scott]

Hi Kevin et al.,

The setup you suggested did not work for me, but I was able to get OpenMP to 
work on Mac (10.13) with R v4.0 packages.  Here's what I did:

(1) After installing Homebrew, ran
brew install libomp
# Note it is also possible to install from OpenMP Source code: 
https://releases.llvm.org/download.html#10.0.0
(2) Determined the install path with
brew --prefix libomp
# In my case /usr/local/opt/libomp
(3) Added lines to ~/.R/Makevars
CC=/usr/local/clang4/bin/clang -fopenmp -I/usr/local/opt/libomp/include
LDFLAGS=-L/usr/local/opt/libomp/lib

I imagine the process is similar for C++ code by setting CXX.  Now 
parallelization for C code with OpenMP is working again on Mac.

As always, I am thankful to the R community for helping problem solve.

I wish more direction along these lines could be added to 
mac.r-project.org

Erik


On Apr 29, 2020, at 1:51 AM, Dmitriy Selivanov 
mailto:selivanov.dmit...@gmail.com>> wrote:

Thanks, Kevin, I can confirm suggested setup works fine so far. Thank you!

I understand it, is that there's
no guarantee that this will work properly (or continue to work
properly) as the Apple toolchain continues to be updated -- e.g. a new
version of macOS / Xcode could install a version of Apple Clang that
is then incompatible with the version of libomp currently in use. In
such a case, I suspect one would need to find and reinstall libomp.

Thats understandable.

On Sat, Apr 25, 2020 at 8:09 PM Kevin Ushey 
mailto:kevinus...@gmail.com>> wrote:
The following works for me on macOS 10.15.4 using Apple Clang + libomp
from LLVM 10 (via Homebrew).

Install libomp from Homebrew with:

brew install libomp

Then, put the following in ~/.R/Makevars

CPPFLAGS += -I/usr/local/opt/libomp/include -Xclang -fopenmp
LDFLAGS += -L/usr/local/opt/libomp/lib -lomp

Then R will automatically enable + use OpenMP as appropriate when
installing packages from sources.

The main danger of this approach, as I understand it, is that there's
no guarantee that this will work properly (or continue to work
properly) as the Apple toolchain continues to be updated -- e.g. a new
version of macOS / Xcode could install a version of Apple Clang that
is then incompatible with the version of libomp currently in use. In
such a case, I suspect one would need to find and reinstall libomp.

In theory, this could be alleviated by ensuring all users download and
use the same version of Xcode as is being used by the macOS build
machine (Xcode 10.1), but in practice users will likely just be using
the "default" set of command line tools that comes with their version
of macOS.

Cunningham's law will hopefully ensure someone else will chime in if
I've got something wrong :-)

Best,
Kevin

On Sat, Apr 25, 2020 at 8:29 AM Wright, Erik Scott 
mailto:eswri...@pitt.edu>> wrote:
>
> I second this request.  Dropping OpenMP support in R v4.0 on Mac is 
> unfortunate.  My R package is highly parallelized via OpenMP, and many of my 
> end-users take advantage of the excellent speedups.  It would be much 
> appreciated if user-level instructions could be provided for how to enable 
> OpenMP support on the Mac.
>
> Thanks,
> Erik
>
>
> > On Apr 23, 2020, at 7:08 AM, Dmitriy Selivanov 
> > mailto:selivanov.dmit...@gmail.com>> wrote:
> >
> > Hi Simon,
> >
> > Just wanted follow up on this topic.
> >
> > It would be very helpful if you can provide some guide on
> > https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmac.r-project.org%2F&data=02%7C01%7Ceswright%40pitt.edu%7C65097590dea4453ab8cf08d7e776b868%7C9ef9f489e0a04eeb87cc3a526112fd0d%7C1%7C0%7C637232369415901734&sdata=qlhXuEubgD6PNmkbD%2BWQy5OI9X%2BGjxSYHMNjA%2BQHZzI%3D&reserved=0>
 for those users
> > (advanced?) and developers who wants to be able to use OpenMP on mac. From
> > what I've understood from this mail thread the easiest way is to install R
> > from homebrew as it is built with non-standard Apple toolchain.
> > Apart from that you've mentioned you may consider to bundle binary  iomp
> > with R installation, but "it would be on the package author to make sure
> > that the

Re: [R-SIG-Mac] OpenMP on CRAN (Simon Urbanek)

[Dmitriy Selivanov]

Thanks, Kevin, I can confirm suggested setup works fine so far. Thank you!

I understand it, is that there's
> no guarantee that this will work properly (or continue to work
> properly) as the Apple toolchain continues to be updated -- e.g. a new
> version of macOS / Xcode could install a version of Apple Clang that
> is then incompatible with the version of libomp currently in use. In
> such a case, I suspect one would need to find and reinstall libomp.


Thats understandable.

On Sat, Apr 25, 2020 at 8:09 PM Kevin Ushey  wrote:

> The following works for me on macOS 10.15.4 using Apple Clang + libomp
> from LLVM 10 (via Homebrew).
>
> Install libomp from Homebrew with:
>
> brew install libomp
>
> Then, put the following in ~/.R/Makevars
>
> CPPFLAGS += -I/usr/local/opt/libomp/include -Xclang -fopenmp
> LDFLAGS += -L/usr/local/opt/libomp/lib -lomp
>
> Then R will automatically enable + use OpenMP as appropriate when
> installing packages from sources.
>
> The main danger of this approach, as I understand it, is that there's
> no guarantee that this will work properly (or continue to work
> properly) as the Apple toolchain continues to be updated -- e.g. a new
> version of macOS / Xcode could install a version of Apple Clang that
> is then incompatible with the version of libomp currently in use. In
> such a case, I suspect one would need to find and reinstall libomp.
>
> In theory, this could be alleviated by ensuring all users download and
> use the same version of Xcode as is being used by the macOS build
> machine (Xcode 10.1), but in practice users will likely just be using
> the "default" set of command line tools that comes with their version
> of macOS.
>
> Cunningham's law will hopefully ensure someone else will chime in if
> I've got something wrong :-)
>
> Best,
> Kevin
>
> On Sat, Apr 25, 2020 at 8:29 AM Wright, Erik Scott 
> wrote:
> >
> > I second this request.  Dropping OpenMP support in R v4.0 on Mac is
> unfortunate.  My R package is highly parallelized via OpenMP, and many of
> my end-users take advantage of the excellent speedups.  It would be much
> appreciated if user-level instructions could be provided for how to enable
> OpenMP support on the Mac.
> >
> > Thanks,
> > Erik
> >
> >
> > > On Apr 23, 2020, at 7:08 AM, Dmitriy Selivanov <
> selivanov.dmit...@gmail.com> wrote:
> > >
> > > Hi Simon,
> > >
> > > Just wanted follow up on this topic.
> > >
> > > It would be very helpful if you can provide some guide on
> > >
> https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmac.r-project.org%2F&data=02%7C01%7Ceswright%40pitt.edu%7C65097590dea4453ab8cf08d7e776b868%7C9ef9f489e0a04eeb87cc3a526112fd0d%7C1%7C0%7C637232369415901734&sdata=qlhXuEubgD6PNmkbD%2BWQy5OI9X%2BGjxSYHMNjA%2BQHZzI%3D&reserved=0
> <
> https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmac.r-project.org%2F&data=02%7C01%7Ceswright%40pitt.edu%7C65097590dea4453ab8cf08d7e776b868%7C9ef9f489e0a04eeb87cc3a526112fd0d%7C1%7C0%7C637232369415911728&sdata=5XR5LJuAKu4%2B3%2FUHsWVPMlyXTLY87P6OF%2FP%2FTZt1W8Y%3D&reserved=0>
> for those users
> > > (advanced?) and developers who wants to be able to use OpenMP on mac.
> From
> > > what I've understood from this mail thread the easiest way is to
> install R
> > > from homebrew as it is built with non-standard Apple toolchain.
> > > Apart from that you've mentioned you may consider to bundle binary
> iomp
> > > with R installation, but "it would be on the package author to make
> sure
> > > that the way the package operates is compatible with that binary".
> Could
> > > you please elaborate on that?
> > >
> > > I believe I'm not alone who would like to be able to use OpenMP on mac
> and
> > > "official" guidance would be very helpful.
> > >
> > > --
> > > Regards
> > > Dmitriy Selivanov
> > >
> > >   [[alternative HTML version deleted]]
> > >
> > > ___
> > > R-SIG-Mac mailing list
> > > R-SIG-Mac@r-project.org
> > >
> https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-sig-mac&data=02%7C01%7Ceswright%40pitt.edu%7C65097590dea4453ab8cf08d7e776b868%7C9ef9f489e0a04eeb87cc3a526112fd0d%7C1%7C0%7C637232369415911728&sdata=yAu4MKX2Ka5yEaoq51byWJSkbL%2FpO1HvtUsI%2BQXmtJQ%3D&reserved=0
> >
> > ___
> > R-SIG-Mac mailing list
> > R-SIG-Mac@r-project.org
> > https://stat.ethz.ch/mailman/listinfo/r-sig-mac
>


-- 
Regards
Dmitriy Selivanov

[[alternative HTML version deleted]]

___
R-SIG-Mac mailing list
R-SIG-Mac@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-mac

Re: [R-SIG-Mac] OpenMP on CRAN (Simon Urbanek)

[Kevin Ushey]

The following works for me on macOS 10.15.4 using Apple Clang + libomp
from LLVM 10 (via Homebrew).

Install libomp from Homebrew with:

brew install libomp

Then, put the following in ~/.R/Makevars

CPPFLAGS += -I/usr/local/opt/libomp/include -Xclang -fopenmp
LDFLAGS += -L/usr/local/opt/libomp/lib -lomp

Then R will automatically enable + use OpenMP as appropriate when
installing packages from sources.

The main danger of this approach, as I understand it, is that there's
no guarantee that this will work properly (or continue to work
properly) as the Apple toolchain continues to be updated -- e.g. a new
version of macOS / Xcode could install a version of Apple Clang that
is then incompatible with the version of libomp currently in use. In
such a case, I suspect one would need to find and reinstall libomp.

In theory, this could be alleviated by ensuring all users download and
use the same version of Xcode as is being used by the macOS build
machine (Xcode 10.1), but in practice users will likely just be using
the "default" set of command line tools that comes with their version
of macOS.

Cunningham's law will hopefully ensure someone else will chime in if
I've got something wrong :-)

Best,
Kevin

On Sat, Apr 25, 2020 at 8:29 AM Wright, Erik Scott  wrote:
>
> I second this request.  Dropping OpenMP support in R v4.0 on Mac is 
> unfortunate.  My R package is highly parallelized via OpenMP, and many of my 
> end-users take advantage of the excellent speedups.  It would be much 
> appreciated if user-level instructions could be provided for how to enable 
> OpenMP support on the Mac.
>
> Thanks,
> Erik
>
>
> > On Apr 23, 2020, at 7:08 AM, Dmitriy Selivanov 
> >  wrote:
> >
> > Hi Simon,
> >
> > Just wanted follow up on this topic.
> >
> > It would be very helpful if you can provide some guide on
> > https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmac.r-project.org%2F&data=02%7C01%7Ceswright%40pitt.edu%7C65097590dea4453ab8cf08d7e776b868%7C9ef9f489e0a04eeb87cc3a526112fd0d%7C1%7C0%7C637232369415901734&sdata=qlhXuEubgD6PNmkbD%2BWQy5OI9X%2BGjxSYHMNjA%2BQHZzI%3D&reserved=0
> >  
> > 
> >  for those users
> > (advanced?) and developers who wants to be able to use OpenMP on mac. From
> > what I've understood from this mail thread the easiest way is to install R
> > from homebrew as it is built with non-standard Apple toolchain.
> > Apart from that you've mentioned you may consider to bundle binary  iomp
> > with R installation, but "it would be on the package author to make sure
> > that the way the package operates is compatible with that binary". Could
> > you please elaborate on that?
> >
> > I believe I'm not alone who would like to be able to use OpenMP on mac and
> > "official" guidance would be very helpful.
> >
> > --
> > Regards
> > Dmitriy Selivanov
> >
> >   [[alternative HTML version deleted]]
> >
> > ___
> > R-SIG-Mac mailing list
> > R-SIG-Mac@r-project.org
> > https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-sig-mac&data=02%7C01%7Ceswright%40pitt.edu%7C65097590dea4453ab8cf08d7e776b868%7C9ef9f489e0a04eeb87cc3a526112fd0d%7C1%7C0%7C637232369415911728&sdata=yAu4MKX2Ka5yEaoq51byWJSkbL%2FpO1HvtUsI%2BQXmtJQ%3D&reserved=0
>
> ___
> R-SIG-Mac mailing list
> R-SIG-Mac@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-mac

___
R-SIG-Mac mailing list
R-SIG-Mac@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-mac

Re: [R-SIG-Mac] OpenMP on CRAN (Simon Urbanek)

[Wright, Erik Scott]

I second this request.  Dropping OpenMP support in R v4.0 on Mac is 
unfortunate.  My R package is highly parallelized via OpenMP, and many of my 
end-users take advantage of the excellent speedups.  It would be much 
appreciated if user-level instructions could be provided for how to enable 
OpenMP support on the Mac.

Thanks,
Erik


> On Apr 23, 2020, at 7:08 AM, Dmitriy Selivanov  
> wrote:
> 
> Hi Simon,
> 
> Just wanted follow up on this topic.
> 
> It would be very helpful if you can provide some guide on
> https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmac.r-project.org%2F&data=02%7C01%7Ceswright%40pitt.edu%7C65097590dea4453ab8cf08d7e776b868%7C9ef9f489e0a04eeb87cc3a526112fd0d%7C1%7C0%7C637232369415901734&sdata=qlhXuEubgD6PNmkbD%2BWQy5OI9X%2BGjxSYHMNjA%2BQHZzI%3D&reserved=0
>  
> 
>  for those users
> (advanced?) and developers who wants to be able to use OpenMP on mac. From
> what I've understood from this mail thread the easiest way is to install R
> from homebrew as it is built with non-standard Apple toolchain.
> Apart from that you've mentioned you may consider to bundle binary  iomp
> with R installation, but "it would be on the package author to make sure
> that the way the package operates is compatible with that binary". Could
> you please elaborate on that?
> 
> I believe I'm not alone who would like to be able to use OpenMP on mac and
> "official" guidance would be very helpful.
> 
> -- 
> Regards
> Dmitriy Selivanov
> 
>   [[alternative HTML version deleted]]
> 
> ___
> R-SIG-Mac mailing list
> R-SIG-Mac@r-project.org
> https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-sig-mac&data=02%7C01%7Ceswright%40pitt.edu%7C65097590dea4453ab8cf08d7e776b868%7C9ef9f489e0a04eeb87cc3a526112fd0d%7C1%7C0%7C637232369415911728&sdata=yAu4MKX2Ka5yEaoq51byWJSkbL%2FpO1HvtUsI%2BQXmtJQ%3D&reserved=0

___
R-SIG-Mac mailing list
R-SIG-Mac@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-mac

Re: [R-SIG-Mac] OpenMP on CRAN (Simon Urbanek)

[Dmitriy Selivanov]

Hi Simon,

Just wanted follow up on this topic.

It would be very helpful if you can provide some guide on
https://mac.R-project.org  for those users
(advanced?) and developers who wants to be able to use OpenMP on mac. From
what I've understood from this mail thread the easiest way is to install R
from homebrew as it is built with non-standard Apple toolchain.
Apart from that you've mentioned you may consider to bundle binary  iomp
with R installation, but "it would be on the package author to make sure
that the way the package operates is compatible with that binary". Could
you please elaborate on that?

I believe I'm not alone who would like to be able to use OpenMP on mac and
"official" guidance would be very helpful.

-- 
Regards
Dmitriy Selivanov

[[alternative HTML version deleted]]

___
R-SIG-Mac mailing list
R-SIG-Mac@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-mac

Re: [R-SIG-Mac] OpenMP on CRAN

[Patrick Schratz]

Simon,

thanks. While this feels like a somewhat 1/1 discussion now, I also think that 
others might profit from it.

1. I am aware that LLVM is a wrapper bundling more than just the compiler and 
that it comes with its own copy of clang
2. I don’t use homebrew because it does not easily allow multiple R 
installations next to each other. This is possible via the official installers. 
Combining this with RSwitch from Bob I can switch between R interpreters with a 
keystroke. I am using homebrew for everything else and even contribute to some 
formulas from time to time.
3. In the end everyone if responsible for their own setup and deviating from 
standards make its more complicated to get help (if at all). However, I am 
still considering blogging about my setup in the future once I have found a 
robust one that worked for some time across all instances I faced issues with 
in the past (focusing on source installs). And don’t worry, I will make sure to 
point out that this is custom, what the CRAN setup is and the differences 
between homebrew and the CRAN thing. There are quite some myths floating around 
that this should be clarified in more depth on a static page (maybe even from 
you as the expert? ;) )

Regarding CI and testing SDK 10.15: A pointer to this discussion where the 
toolchain for GitHub Actions is discussed. I’ll move my thoughts about this to 
a new thread.
On 3. Apr 2020, 22:11 +0200, Simon Urbanek , wrote:
> Patrick,
>
>
> > On 4/04/2020, at 1:33 AM, Patrick Schratz  wrote:
> >
> > Simon,
> >
> > thanks. I assume that you mean “clang from homebrew” = llvm (from homebrew)?
>
> LLVM is the compiler infrastructure project, clang is the C/C++ compiler from 
> that project. We don't use any of the other compilers form LLVM.
>
>
> > Indeed I am currently trying that out and it looks really robust for source 
> > installations (more than the systems clang (+ eventual openMP flags like 
> > suggested by Kevin) and other variants (gcc from homebrew or openMP enabled 
> > clang7 or 8).
> >
> > Note that I do not use R via homebrew.
> >
> > However, all in all I am essentially mixing the custom llvm from homebrew 
> > with the official R installer and the old 10.13 SDK.
> > It looks quite complex and I’d wish everything would be easier. However, in 
> > the end I just want to have a stable “install from source” environment that 
> > works for all packages out there (I do not use the binaries).
> >
>
> Why don't use use Homebrew? That's exactly what Homebrew provides - its own 
> ecosystem, everything form source (with cached binaries if you want them).
>
>
> > All in all I am a bit confused now about all the mixing and options 
> > available. Let’s see what the foreseeable future brings and where things 
> > are going.
> >
>
> Mixing is not available. You pick one system and go with it, but can't mix 
> between them, because of the different run-time libraries.
>
>
> > Regarding the SDK issue: I think most users just use the binaries on macOS 
> > (across all OS versions) and rarely face such issues. And there are 
> > presumably not many people out there that do actual R-devel testing on 
> > Catalina (?), otherwise I’d expect way more people to jump into the 
> > discussion. However, I am not alone with these issues as you can see in the 
> > Rcpp issue we were talking about.
> > Maybe you can even reproduce the issues by linking a custom install of the 
> > 10.15 SDK on a non-Catalina machine? I don’t know how much trouble that 
> > would bring up when trying to jump into the future - but this ist just an 
> > idea :)
> >
>
> That's an entirely different, unrelated topic. Testing cutting edge (or even 
> pre-release) systems makes sense, but that's not our current worry (there was 
> a separate thread about CI).
>
> Cheers,
> Simon
>
>
> > Thanks for your work!
> > On 3. Apr 2020, 14:13 +0200, Simon Urbanek , 
> > wrote:
> > > Patrick,
> > >
> > > you were commenting on the thread where we talked about CRAN R - that one 
> > > is now compiled using Apple clang. You were talking about using clang 
> > > from Homebrew - those are incompatible as they use different run-time. 
> > > Unfortunately, the Intel OpenMP run-time varies by clang compiler version 
> > > and is known to fail when used with the wrong compiler. Analogously, 
> > > mixing gomp (from gcc) and iomp is problematic (and GNU Fortran uses 
> > > gomp). As discussed in the Homebrew thread, if you are compiling 
> > > everything via Homebrew including R then it's all good, you just can't 
> > > mix Apple tools and Homebrew in general.
> > >
> > > Also at the bottom I was only talking about 10.14 SDK - that's what we 
> > > use on CRAN and I have not seen any issues with it - you were claiming 
> > > that it doesn't work with Rcpp and igraph.
> > >
> > > Cheers,
> > > Simon
> > >
> > >
> > >
> > > > On 4/04/2020, at 1:00 AM, Patrick Schratz  
> > > > wrote:
> > > >
> > > > Simon,
> > > >
> > > > I don’t understand fully. I am using llv

Re: [R-SIG-Mac] OpenMP on CRAN

[Simon Urbanek]

Patrick,


> On 4/04/2020, at 1:33 AM, Patrick Schratz  wrote:
> 
> Simon,
> 
> thanks. I assume that you mean “clang from homebrew” = llvm (from homebrew)?

LLVM is the compiler infrastructure project, clang is the C/C++ compiler from 
that project. We don't use any of the other compilers form LLVM.


> Indeed I am currently trying that out and it looks really robust for source 
> installations (more than the systems clang (+ eventual openMP flags like 
> suggested by Kevin) and other variants (gcc from homebrew or openMP enabled 
> clang7 or 8).
> 
> Note that I do not use R via homebrew.
> 
> However, all in all I am essentially mixing the custom llvm from homebrew 
> with the official R installer and the old 10.13 SDK.
> It looks quite complex and I’d wish everything would be easier. However, in 
> the end I just want to have a stable “install from source” environment that 
> works for all packages out there (I do not use the binaries).
> 

Why don't use use Homebrew? That's exactly what Homebrew provides - its own 
ecosystem, everything form source (with cached binaries if you want them).


> All in all I am a bit confused now about all the mixing and options 
> available. Let’s see what the foreseeable future brings and where things are 
> going.
> 

Mixing is not available. You pick one system and go with it, but can't mix 
between them, because of the different run-time libraries.


> Regarding the SDK issue: I think most users just use the binaries on macOS 
> (across all OS versions) and rarely face such issues. And there are 
> presumably not many people out there that do actual R-devel testing on 
> Catalina (?), otherwise I’d expect way more people to jump into the 
> discussion. However, I am not alone with these issues as you can see in the 
> Rcpp issue we were talking about. 
> Maybe you can even reproduce the issues by linking a custom install of the 
> 10.15 SDK on a non-Catalina machine? I don’t know how much trouble that would 
> bring up when trying to jump into the future - but this ist just an idea :)
> 

That's an entirely different, unrelated topic. Testing cutting edge (or even 
pre-release) systems makes sense, but that's not our current worry (there was a 
separate thread about CI).

Cheers,
Simon


> Thanks for your work!
> On 3. Apr 2020, 14:13 +0200, Simon Urbanek , 
> wrote:
>> Patrick,
>> 
>> you were commenting on the thread where we talked about CRAN R - that one is 
>> now compiled using Apple clang. You were talking about using clang from 
>> Homebrew - those are incompatible as they use different run-time. 
>> Unfortunately, the Intel OpenMP run-time varies by clang compiler version 
>> and is known to fail when used with the wrong compiler. Analogously, mixing 
>> gomp (from gcc) and iomp is problematic (and GNU Fortran uses gomp). As 
>> discussed in the Homebrew thread, if you are compiling everything via 
>> Homebrew including R then it's all good, you just can't mix Apple tools and 
>> Homebrew in general.
>> 
>> Also at the bottom I was only talking about 10.14 SDK - that's what we use 
>> on CRAN and I have not seen any issues with it - you were claiming that it 
>> doesn't work with Rcpp and igraph.
>> 
>> Cheers,
>> Simon
>> 
>> 
>> 
>>> On 4/04/2020, at 1:00 AM, Patrick Schratz  wrote:
>>> 
>>> Simon,
>>> 
>>> I don’t understand fully. I am using llvm for all C variants (just now 
>>> shown) in combination with the 10.13 SDK.
>>> So far this combination works flawlessly for all “problematic” packages 
>>> like data.table, igraph or Rcpp.
>>> 
>>> I don’t have deeper knowledge about the “iomp” setup but as of right now I 
>>> don’t know what I am mixing up here. Can you shine more light on this?
>>> If it is about llvm in general: data.table recommends to use llvm openly in 
>>> their wiki due to the lack of openMP for the standard clang.
>>> And while this could be handled as “any other package” in R, we all know 
>>> that it has quite some impact and probably devs who know what they do in 
>>> terms of C configs.
>>> 
>>> Regarding the SDK issue: Both Rcpp v1.0.4 and igraph 1.2.5 can’t be 
>>> installed from source with SDK 10.15 and standard apple clang (or any C 
>>> compiler). Switching to SDK 10.13 solves the issue. You were even 
>>> commenting on the Rcpp issue.
>>> Since SDK 10.15 is the current release and many users have no choice on 
>>> which OS version they are (for different reasons), this issue should imo be 
>>> inspected in more detail. What do you think?
>>> 
>>> Best, Patrick
>>> On 2. Apr 2020, 23:38 +0200, Simon Urbanek , 
>>> wrote:
 Patrick,
 
 you can't mix compilers - it really matters which iomp your'e using. llvm 
 has its own modified version which may not be the same as the official 
 Intel release. From your tests it looks like you're using the llvm one 
 which will likely work only with that compiler. Since Apple doesn't have 
 official omp support it's hard to say which versions work and which don't

Re: [R-SIG-Mac] OpenMP on CRAN

[Patrick Schratz]

Simon,

thanks. I assume that you mean “clang from homebrew” = llvm (from homebrew)?
Indeed I am currently trying that out and it looks really robust for source 
installations (more than the systems clang (+ eventual openMP flags like 
suggested by Kevin) and other variants (gcc from homebrew or openMP enabled 
clang7 or 8).

Note that I do not use R via homebrew.

However, all in all I am essentially mixing the custom llvm from homebrew with 
the official R installer and the old 10.13 SDK.
It looks quite complex and I’d wish everything would be easier. However, in the 
end I just want to have a stable “install from source” environment that works 
for all packages out there (I do not use the binaries).

All in all I am a bit confused now about all the mixing and options available. 
Let’s see what the foreseeable future brings and where things are going.

Regarding the SDK issue: I think most users just use the binaries on macOS 
(across all OS versions) and rarely face such issues. And there are presumably 
not many people out there that do actual R-devel testing on Catalina (?), 
otherwise I’d expect way more people to jump into the discussion. However, I am 
not alone with these issues as you can see in the Rcpp issue we were talking 
about.
Maybe you can even reproduce the issues by linking a custom install of the 
10.15 SDK on a non-Catalina machine? I don’t know how much trouble that would 
bring up when trying to jump into the future - but this ist just an idea :)

Thanks for your work!
On 3. Apr 2020, 14:13 +0200, Simon Urbanek , wrote:
> Patrick,
>
> you were commenting on the thread where we talked about CRAN R - that one is 
> now compiled using Apple clang. You were talking about using clang from 
> Homebrew - those are incompatible as they use different run-time. 
> Unfortunately, the Intel OpenMP run-time varies by clang compiler version and 
> is known to fail when used with the wrong compiler. Analogously, mixing gomp 
> (from gcc) and iomp is problematic (and GNU Fortran uses gomp). As discussed 
> in the Homebrew thread, if you are compiling everything via Homebrew 
> including R then it's all good, you just can't mix Apple tools and Homebrew 
> in general.
>
> Also at the bottom I was only talking about 10.14 SDK - that's what we use on 
> CRAN and I have not seen any issues with it - you were claiming that it 
> doesn't work with Rcpp and igraph.
>
> Cheers,
> Simon
>
>
>
> > On 4/04/2020, at 1:00 AM, Patrick Schratz  wrote:
> >
> > Simon,
> >
> > I don’t understand fully. I am using llvm for all C variants (just now 
> > shown) in combination with the 10.13 SDK.
> > So far this combination works flawlessly for all “problematic” packages 
> > like data.table, igraph or Rcpp.
> >
> > I don’t have deeper knowledge about the “iomp” setup but as of right now I 
> > don’t know what I am mixing up here. Can you shine more light on this?
> > If it is about llvm in general: data.table recommends to use llvm openly in 
> > their wiki due to the lack of openMP for the standard clang.
> > And while this could be handled as “any other package” in R, we all know 
> > that it has quite some impact and probably devs who know what they do in 
> > terms of C configs.
> >
> > Regarding the SDK issue: Both Rcpp v1.0.4 and igraph 1.2.5 can’t be 
> > installed from source with SDK 10.15 and standard apple clang (or any C 
> > compiler). Switching to SDK 10.13 solves the issue. You were even 
> > commenting on the Rcpp issue.
> > Since SDK 10.15 is the current release and many users have no choice on 
> > which OS version they are (for different reasons), this issue should imo be 
> > inspected in more detail. What do you think?
> >
> > Best, Patrick
> > On 2. Apr 2020, 23:38 +0200, Simon Urbanek , 
> > wrote:
> > > Patrick,
> > >
> > > you can't mix compilers - it really matters which iomp your'e using. llvm 
> > > has its own modified version which may not be the same as the official 
> > > Intel release. From your tests it looks like you're using the llvm one 
> > > which will likely work only with that compiler. Since Apple doesn't have 
> > > official omp support it's hard to say which versions work and which don't.
> > >
> > >
> > > > On 3/04/2020, at 10:20 AM, Patrick Schratz  
> > > > wrote:
> > > >
> > > > Thanks Kevin, interesting approach.
> > > >
> > > > However, when testing it against a few packages I get symbol pointer 
> > > > issues (e.g. for data.table and xgboost).
> > > > Using llvm everything works.
> > > > Could llvm be the best middle way? Easy installation via brew and still 
> > > > clang compliant.
> > > >
> > > > Currently my Makevars look as follows
> > > >
> > > > CFLAGS=-isysroot 
> > > > /Library/Developer/CommandLineTools/SDKs/MacOSX10.13.sdk
> > > > […]
> > > >
> > > > CC = ccache /usr/local/opt/llvm/bin/clang
> > > > […]
> > > >
> > > > Where llvm was installed via `brew install llvm`.
> > > > SDK 10.13 because of {igraph} and {Rcpp} issues with SDK 10.14 and SDK 
> > > > 10.

Re: [R-SIG-Mac] OpenMP on CRAN

[Simon Urbanek]

Patrick,

you were commenting on the thread where we talked about CRAN R - that one is 
now compiled using Apple clang. You were talking about using clang from 
Homebrew - those are incompatible as they use different run-time. 
Unfortunately, the Intel OpenMP run-time varies by clang compiler version and 
is known to fail when used with the wrong compiler. Analogously, mixing gomp 
(from gcc) and iomp is problematic (and GNU Fortran uses gomp). As discussed in 
the Homebrew thread, if you are compiling everything via Homebrew including R 
then it's all good, you just can't mix Apple tools and Homebrew in general.

Also at the bottom I was only talking about 10.14 SDK - that's what we use on 
CRAN and I have not seen any issues with it - you were claiming that it doesn't 
work with Rcpp and igraph.

Cheers,
Simon



> On 4/04/2020, at 1:00 AM, Patrick Schratz  wrote:
> 
> Simon,
> 
> I don’t understand fully. I am using llvm for all C variants (just now shown) 
> in combination with the 10.13 SDK.
> So far this combination works flawlessly for all “problematic” packages like 
> data.table, igraph or Rcpp.
> 
> I don’t have deeper knowledge about the “iomp” setup but as of right now I 
> don’t know what I am mixing up here. Can you shine more light on this?
> If it is about llvm in general: data.table recommends to use llvm openly in 
> their wiki due to the lack of openMP for the standard clang.
> And while this could be handled as “any other package” in R, we all know that 
> it has quite some impact and probably devs who know what they do in terms of 
> C configs.
> 
> Regarding the SDK issue: Both Rcpp v1.0.4 and igraph 1.2.5 can’t be installed 
> from source with SDK 10.15 and standard apple clang (or any C compiler). 
> Switching to SDK 10.13 solves the issue. You were even commenting on the Rcpp 
> issue.
> Since SDK 10.15 is the current release and many users have no choice on which 
> OS version they are (for different reasons), this issue should imo be 
> inspected in more detail. What do you think?
> 
> Best, Patrick
> On 2. Apr 2020, 23:38 +0200, Simon Urbanek , 
> wrote:
>> Patrick,
>> 
>> you can't mix compilers - it really matters which iomp your'e using. llvm 
>> has its own modified version which may not be the same as the official Intel 
>> release. From your tests it looks like you're using the llvm one which will 
>> likely work only with that compiler. Since Apple doesn't have official omp 
>> support it's hard to say which versions work and which don't.
>> 
>> 
>>> On 3/04/2020, at 10:20 AM, Patrick Schratz  
>>> wrote:
>>> 
>>> Thanks Kevin, interesting approach.
>>> 
>>> However, when testing it against a few packages I get symbol pointer issues 
>>> (e.g. for data.table and xgboost).
>>> Using llvm everything works.
>>> Could llvm be the best middle way? Easy installation via brew and still 
>>> clang compliant.
>>> 
>>> Currently my Makevars look as follows
>>> 
>>> CFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.13.sdk
>>> […]
>>> 
>>> CC = ccache /usr/local/opt/llvm/bin/clang
>>> […]
>>> 
>>> Where llvm was installed via `brew install llvm`.
>>> SDK 10.13 because of {igraph} and {Rcpp} issues with SDK 10.14 and SDK 10.15
>> 
>> 
>> I mentioned that before, but I do not see issues with 10.14 SDK.
>> 
>> Cheers,
>> Simon
>> 
>> 
>>> On 2. Apr 2020, 23:01 +0200, Simon Urbanek , 
>>> wrote:
 Tim,
 
 
> On 3/04/2020, at 2:01 AM, BATES Timothy  wrote:
> 
> Moving to a compiler that drops support for OpenMP seems a sad choice, 
> especially now we’ve all climbed the learning curve of the non-Apple 
> compiler (the real barrier was lack of a pkg installer and that’s done 
> now).
> 
 
 It has caused a lot of issues, it trips people on a daily basis which is 
 just not worth it. Also with Apple's increasingly strict rules about what 
 can be distributed we don't want to be in the business in maintaining a 
 separate toolchain. There have been numerous issues with C++ exceptions so 
 the fall out was much bigger than originally expected and it would only 
 get worse.
 
 
> Losing OpenMP for the CRAN version of OpenMx/umx (our SEM packages) would 
> be a big loss for users (for whom the CRAN version now supports OpenMP 
> giving them a 2-12x speed up). In general, R on Mac is made more viable 
> by having OpenMP
> 
> Re Brian’s points, I’d say that the distribution problem is crucial: 
> Packages not on CRAN have dramatically diminished accessibility/useage.
> 
 
 The idea is that if a package deems this critical, it can enable this for 
 the users. As it is now, packages can still supply iomp and use it.
 
 That said, I would open for discussion the ability to distribute iomp with 
 the R binary, but it would not be supported by R directly, i.e., it would 
 be on the package author to make sure that the way the package operates is 
 c

Re: [R-SIG-Mac] OpenMP on CRAN

[Patrick Schratz]

Simon,

I don’t understand fully. I am using llvm for all C variants (just now shown) 
in combination with the 10.13 SDK.
So far this combination works flawlessly for all “problematic” packages like 
data.table, igraph or Rcpp.

I don’t have deeper knowledge about the “iomp” setup but as of right now I 
don’t know what I am mixing up here. Can you shine more light on this?
If it is about llvm in general: data.table recommends to use llvm openly in 
their wiki due to the lack of openMP for the standard clang.
And while this could be handled as “any other package” in R, we all know that 
it has quite some impact and probably devs who know what they do in terms of C 
configs.

Regarding the SDK issue: Both Rcpp v1.0.4 and igraph 1.2.5 can’t be installed 
from source with SDK 10.15 and standard apple clang (or any C compiler). 
Switching to SDK 10.13 solves the issue. You were even commenting on the Rcpp 
issue.
Since SDK 10.15 is the current release and many users have no choice on which 
OS version they are (for different reasons), this issue should imo be inspected 
in more detail. What do you think?

Best, Patrick
On 2. Apr 2020, 23:38 +0200, Simon Urbanek , wrote:
> Patrick,
>
> you can't mix compilers - it really matters which iomp your'e using. llvm has 
> its own modified version which may not be the same as the official Intel 
> release. From your tests it looks like you're using the llvm one which will 
> likely work only with that compiler. Since Apple doesn't have official omp 
> support it's hard to say which versions work and which don't.
>
>
> > On 3/04/2020, at 10:20 AM, Patrick Schratz  
> > wrote:
> >
> > Thanks Kevin, interesting approach.
> >
> > However, when testing it against a few packages I get symbol pointer issues 
> > (e.g. for data.table and xgboost).
> > Using llvm everything works.
> > Could llvm be the best middle way? Easy installation via brew and still 
> > clang compliant.
> >
> > Currently my Makevars look as follows
> >
> > CFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.13.sdk
> > […]
> >
> > CC = ccache /usr/local/opt/llvm/bin/clang
> > […]
> >
> > Where llvm was installed via `brew install llvm`.
> > SDK 10.13 because of {igraph} and {Rcpp} issues with SDK 10.14 and SDK 10.15
>
>
> I mentioned that before, but I do not see issues with 10.14 SDK.
>
> Cheers,
> Simon
>
>
> > On 2. Apr 2020, 23:01 +0200, Simon Urbanek , 
> > wrote:
> > > Tim,
> > >
> > >
> > > > On 3/04/2020, at 2:01 AM, BATES Timothy  wrote:
> > > >
> > > > Moving to a compiler that drops support for OpenMP seems a sad choice, 
> > > > especially now we’ve all climbed the learning curve of the non-Apple 
> > > > compiler (the real barrier was lack of a pkg installer and that’s done 
> > > > now).
> > > >
> > >
> > > It has caused a lot of issues, it trips people on a daily basis which is 
> > > just not worth it. Also with Apple's increasingly strict rules about what 
> > > can be distributed we don't want to be in the business in maintaining a 
> > > separate toolchain. There have been numerous issues with C++ exceptions 
> > > so the fall out was much bigger than originally expected and it would 
> > > only get worse.
> > >
> > >
> > > > Losing OpenMP for the CRAN version of OpenMx/umx (our SEM packages) 
> > > > would be a big loss for users (for whom the CRAN version now supports 
> > > > OpenMP giving them a 2-12x speed up). In general, R on Mac is made more 
> > > > viable by having OpenMP
> > > >
> > > > Re Brian’s points, I’d say that the distribution problem is crucial: 
> > > > Packages not on CRAN have dramatically diminished accessibility/useage.
> > > >
> > >
> > > The idea is that if a package deems this critical, it can enable this for 
> > > the users. As it is now, packages can still supply iomp and use it.
> > >
> > > That said, I would open for discussion the ability to distribute iomp 
> > > with the R binary, but it would not be supported by R directly, i.e., it 
> > > would be on the package author to make sure that the way the package 
> > > operates is compatible with that binary. Let me know what you think.
> > >
> > >
> > > > Second, a great range of compute-intensive problems are amenable to 
> > > > division amongst cores, including nearly all models that take more than 
> > > > a nominal amount of time to run: So simulations, CIs, bootstrapping, 
> > > > nearly everything in genetics all speeds up.
> > > >
> > >
> > > Yes, but OpenMP is rarely used for those tasks. In R OpenMP can be only 
> > > used for very small subset of such tasks - which is why alternative 
> > > approaches are much more common.
> > >
> > > Cheers,
> > > Simon
> > >
> > >
> > > > I’d say especially on desktop/laptop. The big advantage of multi blade 
> > > > systems requires snowfall-type solutions, but desktops profit 
> > > > automatically from their multi-core structure and don;’t have multiple 
> > > > processors (except graphics, which no-one seems to be exploiting on 
> > > > CRA

Re: [R-SIG-Mac] OpenMP on CRAN

[Simon Urbanek]

Patrick,

you can't mix compilers - it really matters which iomp your'e using. llvm has 
its own modified version which may not be the same as the official Intel 
release. From your tests it looks like you're using the llvm one which will 
likely work only with that compiler. Since Apple doesn't have official omp 
support it's hard to say which versions work and which don't.


> On 3/04/2020, at 10:20 AM, Patrick Schratz  wrote:
> 
> Thanks Kevin, interesting approach.
> 
> However, when testing it against a few packages I get symbol pointer issues 
> (e.g. for data.table and xgboost). 
> Using llvm everything works. 
> Could llvm be the best middle way? Easy installation via brew and still clang 
> compliant.
> 
> Currently my Makevars look as follows
> 
> CFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.13.sdk
> […]
> 
> CC = ccache /usr/local/opt/llvm/bin/clang
> […]
> 
> Where llvm was installed via `brew install llvm`.
> SDK 10.13 because of {igraph} and {Rcpp} issues with SDK 10.14 and SDK 10.15


I mentioned that before, but I do not see issues with 10.14 SDK.

Cheers,
Simon


> On 2. Apr 2020, 23:01 +0200, Simon Urbanek , 
> wrote:
>> Tim,
>> 
>> 
>>> On 3/04/2020, at 2:01 AM, BATES Timothy  wrote:
>>> 
>>> Moving to a compiler that drops support for OpenMP seems a sad choice, 
>>> especially now we’ve all climbed the learning curve of the non-Apple 
>>> compiler (the real barrier was lack of a pkg installer and that’s done now).
>>> 
>> 
>> It has caused a lot of issues, it trips people on a daily basis which is 
>> just not worth it. Also with Apple's increasingly strict rules about what 
>> can be distributed we don't want to be in the business in maintaining a 
>> separate toolchain. There have been numerous issues with C++ exceptions so 
>> the fall out was much bigger than originally expected and it would only get 
>> worse.
>> 
>> 
>>> Losing OpenMP for the CRAN version of OpenMx/umx (our SEM packages) would 
>>> be a big loss for users (for whom the CRAN version now supports OpenMP 
>>> giving them a 2-12x speed up). In general, R on Mac is made more viable by 
>>> having OpenMP
>>> 
>>> Re Brian’s points, I’d say that the distribution problem is crucial: 
>>> Packages not on CRAN have dramatically diminished accessibility/useage.
>>> 
>> 
>> The idea is that if a package deems this critical, it can enable this for 
>> the users. As it is now, packages can still supply iomp and use it.
>> 
>> That said, I would open for discussion the ability to distribute iomp with 
>> the R binary, but it would not be supported by R directly, i.e., it would be 
>> on the package author to make sure that the way the package operates is 
>> compatible with that binary. Let me know what you think.
>> 
>> 
>>> Second, a great range of compute-intensive problems are amenable to 
>>> division amongst cores, including nearly all models that take more than a 
>>> nominal amount of time to run: So simulations, CIs, bootstrapping, nearly 
>>> everything in genetics all speeds up.
>>> 
>> 
>> Yes, but OpenMP is rarely used for those tasks. In R OpenMP can be only used 
>> for very small subset of such tasks - which is why alternative approaches 
>> are much more common.
>> 
>> Cheers,
>> Simon
>> 
>> 
>>> I’d say especially on desktop/laptop. The big advantage of multi blade 
>>> systems requires snowfall-type solutions, but desktops profit automatically 
>>> from their multi-core structure and don;’t have multiple processors (except 
>>> graphics, which no-one seems to be exploiting on CRAN-style R), so OpenMP 
>>> is their one trick. I’d hope not to lose it.
>>> 
>>> Best, t
>>> 
>>> 
 On 2 Apr 2020, at 05:18, Prof Brian Ripley  wrote:
 
 On 01/04/2020 22:02, Simon Urbanek wrote:
> JJB,
> 1. correct, there was too much trouble in this. But please feel free to 
> start a new thread about this here if you have strong opinions.
 
 Also note that it is possible (and not hard) to install packages from 
 source with an OpenMP-supporting compiler, and how to do so is in the 
 R-admin manual. The problems come in distributing them.
 
 The benefits of OpenMP are often overestimated, especially on 
 desktop/laptop level hardware. But it is available for the small (tiny?) 
 proportion of users who need it.
>>> The University of Edinburgh is a charitable body, registered in Scotland, 
>>> with registration number SC005336.
>>> ___
>>> R-SIG-Mac mailing list
>>> R-SIG-Mac@r-project.org
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mac
>> 
>> ___
>> R-SIG-Mac mailing list
>> R-SIG-Mac@r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-mac

___
R-SIG-Mac mailing list
R-SIG-Mac@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-mac

Re: [R-SIG-Mac] OpenMP on CRAN

[Patrick Schratz]

Thanks Kevin, interesting approach.

However, when testing it against a few packages I get symbol pointer issues 
(e.g. for data.table and xgboost).
Using llvm everything works.
Could llvm be the best middle way? Easy installation via brew and still clang 
compliant.

Currently my Makevars look as follows

CFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.13.sdk
[…]

CC = ccache /usr/local/opt/llvm/bin/clang
[…]

Where llvm was installed via `brew install llvm`.
SDK 10.13 because of {igraph} and {Rcpp} issues with SDK 10.14 and SDK 10.15
On 2. Apr 2020, 23:01 +0200, Simon Urbanek , wrote:
> Tim,
>
>
> > On 3/04/2020, at 2:01 AM, BATES Timothy  wrote:
> >
> > Moving to a compiler that drops support for OpenMP seems a sad choice, 
> > especially now we’ve all climbed the learning curve of the non-Apple 
> > compiler (the real barrier was lack of a pkg installer and that’s done now).
> >
>
> It has caused a lot of issues, it trips people on a daily basis which is just 
> not worth it. Also with Apple's increasingly strict rules about what can be 
> distributed we don't want to be in the business in maintaining a separate 
> toolchain. There have been numerous issues with C++ exceptions so the fall 
> out was much bigger than originally expected and it would only get worse.
>
>
> > Losing OpenMP for the CRAN version of OpenMx/umx (our SEM packages) would 
> > be a big loss for users (for whom the CRAN version now supports OpenMP 
> > giving them a 2-12x speed up). In general, R on Mac is made more viable by 
> > having OpenMP
> >
> > Re Brian’s points, I’d say that the distribution problem is crucial: 
> > Packages not on CRAN have dramatically diminished accessibility/useage.
> >
>
> The idea is that if a package deems this critical, it can enable this for the 
> users. As it is now, packages can still supply iomp and use it.
>
> That said, I would open for discussion the ability to distribute iomp with 
> the R binary, but it would not be supported by R directly, i.e., it would be 
> on the package author to make sure that the way the package operates is 
> compatible with that binary. Let me know what you think.
>
>
> > Second, a great range of compute-intensive problems are amenable to 
> > division amongst cores, including nearly all models that take more than a 
> > nominal amount of time to run: So simulations, CIs, bootstrapping, nearly 
> > everything in genetics all speeds up.
> >
>
> Yes, but OpenMP is rarely used for those tasks. In R OpenMP can be only used 
> for very small subset of such tasks - which is why alternative approaches are 
> much more common.
>
> Cheers,
> Simon
>
>
> > I’d say especially on desktop/laptop. The big advantage of multi blade 
> > systems requires snowfall-type solutions, but desktops profit automatically 
> > from their multi-core structure and don;’t have multiple processors (except 
> > graphics, which no-one seems to be exploiting on CRAN-style R), so OpenMP 
> > is their one trick. I’d hope not to lose it.
> >
> > Best, t
> >
> >
> > > On 2 Apr 2020, at 05:18, Prof Brian Ripley  wrote:
> > >
> > > On 01/04/2020 22:02, Simon Urbanek wrote:
> > > > JJB,
> > > > 1. correct, there was too much trouble in this. But please feel free to 
> > > > start a new thread about this here if you have strong opinions.
> > >
> > > Also note that it is possible (and not hard) to install packages from 
> > > source with an OpenMP-supporting compiler, and how to do so is in the 
> > > R-admin manual. The problems come in distributing them.
> > >
> > > The benefits of OpenMP are often overestimated, especially on 
> > > desktop/laptop level hardware. But it is available for the small (tiny?) 
> > > proportion of users who need it.
> > The University of Edinburgh is a charitable body, registered in Scotland, 
> > with registration number SC005336.
> > ___
> > R-SIG-Mac mailing list
> > R-SIG-Mac@r-project.org
> > https://stat.ethz.ch/mailman/listinfo/r-sig-mac
>
> ___
> R-SIG-Mac mailing list
> R-SIG-Mac@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-mac

[[alternative HTML version deleted]]

___
R-SIG-Mac mailing list
R-SIG-Mac@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-mac

Re: [R-SIG-Mac] OpenMP on CRAN

[Simon Urbanek]

Tim,


> On 3/04/2020, at 2:01 AM, BATES Timothy  wrote:
> 
> Moving to a compiler that drops support for OpenMP seems a sad choice, 
> especially now we’ve all climbed the learning curve of the non-Apple compiler 
> (the real barrier was lack of a pkg installer and that’s done now).
> 

It has caused a lot of issues, it trips people on a daily basis which is just 
not worth it. Also with Apple's increasingly strict rules about what can be 
distributed we don't want to be in the business in maintaining a separate 
toolchain. There have been numerous issues with C++ exceptions so the fall out 
was much bigger than originally expected and it would only get worse.


> Losing OpenMP for the CRAN version of OpenMx/umx (our SEM packages) would be 
> a big loss for users (for whom the CRAN version now supports OpenMP giving 
> them a 2-12x speed up). In general, R on Mac is made more viable by having 
> OpenMP
> 
> Re Brian’s points, I’d say that the distribution problem is crucial: Packages 
> not on CRAN have dramatically diminished accessibility/useage.
> 

The idea is that if a package deems this critical, it can enable this for the 
users. As it is now, packages can still supply iomp and use it.

That said, I would open for discussion the ability to distribute iomp with the 
R binary, but it would not be supported by R directly, i.e., it would be on the 
package author to make sure that the way the package operates is compatible 
with that binary. Let me know what you think.


> Second, a great range of compute-intensive problems are amenable to division 
> amongst cores, including nearly all models that take more than a nominal 
> amount of time to run: So simulations, CIs, bootstrapping, nearly everything 
> in genetics all speeds up.
> 

Yes, but OpenMP is rarely used for those tasks. In R OpenMP can be only used 
for very small subset of such tasks - which is why alternative approaches are 
much more common.

Cheers,
Simon


> I’d say especially on desktop/laptop. The big advantage of multi blade 
> systems requires snowfall-type solutions, but desktops profit automatically 
> from their multi-core structure and don;’t have multiple processors (except 
> graphics, which no-one seems to be exploiting on CRAN-style R), so OpenMP is 
> their one trick. I’d hope not to lose it.
> 
> Best, t
> 
> 
>> On 2 Apr 2020, at 05:18, Prof Brian Ripley  wrote:
>> 
>> On 01/04/2020 22:02, Simon Urbanek wrote:
>>> JJB,
>>> 1. correct, there was too much trouble in this. But please feel free to 
>>> start a new thread about this here if you have strong opinions.
>> 
>> Also note that it is possible (and not hard) to install packages from source 
>> with an OpenMP-supporting compiler, and how to do so is in the R-admin 
>> manual.  The problems come in distributing them.
>> 
>> The benefits of OpenMP are often overestimated, especially on desktop/laptop 
>> level hardware.  But it is available for the small (tiny?) proportion of 
>> users who need it.
> The University of Edinburgh is a charitable body, registered in Scotland, 
> with registration number SC005336.
> ___
> R-SIG-Mac mailing list
> R-SIG-Mac@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-mac

___
R-SIG-Mac mailing list
R-SIG-Mac@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-mac

Re: [R-SIG-Mac] OpenMP on CRAN

[Balamuta, James Joseph]

Kevin,

Simon discussed why they opted to avoid this back in June '19 when Jon Clayden 
brought up a similar success.

https://stat.ethz.ch/pipermail/r-sig-mac/2019-June/012998.html

The sentiment was using the system compiler is in manner is unsupported and 
works only on some systems. I'm not sure with the OS bump whether there still 
is disparity across 10.13/14/15.

That said, I do agree with Tim that it would be nice to have OpenMP enabled-by 
default. But, I'm also equally okay with it being disabled to simplify workflow 
and encourage more parallelization through snow/future as getting into 
parallelized compiled code with R has far more dragons afoot.

Best,

JJB

On 4/2/20, 12:05 PM, "R-SIG-Mac on behalf of Kevin Ushey" 
 wrote:

Hi,

For what it's worth, it looks like it is still possible to use OpenMP
on macOS with the system toolchain. Using the example file here:

https://computing.llnl.gov/tutorials/openMP/samples/C/omp_hello.c

I can compile + link + run this on macOS 10.15.4 and with:

$ /usr/bin/clang -Xpreprocessor -fopenmp
-I/usr/local/opt/libomp/include -L/usr/local/opt/libomp/lib -lomp
omp_hello.c

As for whether this is 'safe', or whether R could conceivably bundle
and use its own copy of libomp is a separate question I cannot answer.
But at least this is a mechanism for enterprising users to enable
OpenMP in packages built from source if they so desire.

Best,
Kevin

On Thu, Apr 2, 2020 at 6:01 AM BATES Timothy  wrote:
>
> Moving to a compiler that drops support for OpenMP seems a sad choice, 
especially now we’ve all climbed the learning curve of the non-Apple compiler 
(the real barrier was lack of a pkg installer and that’s done now).
>
> Losing OpenMP for the CRAN version of OpenMx/umx (our SEM packages) would 
be a big loss for users (for whom the CRAN version now supports OpenMP giving 
them a 2-12x speed up). In general, R on Mac is made more viable by having 
OpenMP
>
> Re Brian’s points, I’d say that the distribution problem is crucial: 
Packages not on CRAN have dramatically diminished accessibility/useage.
>
> Second, a great range of compute-intensive problems are amenable to 
division amongst cores, including nearly all models that take more than a 
nominal amount of time to run: So simulations, CIs, bootstrapping, nearly 
everything in genetics all speeds up.
>
> I’d say especially on desktop/laptop. The big advantage of multi blade 
systems requires snowfall-type solutions, but desktops profit automatically 
from their multi-core structure and don;’t have multiple processors (except 
graphics, which no-one seems to be exploiting on CRAN-style R), so OpenMP is 
their one trick. I’d hope not to lose it.
>
> Best, t
>
>
> > On 2 Apr 2020, at 05:18, Prof Brian Ripley  
wrote:
> >
> > On 01/04/2020 22:02, Simon Urbanek wrote:
> >> JJB,
> >> 1. correct, there was too much trouble in this. But please feel free 
to start a new thread about this here if you have strong opinions.
> >
> > Also note that it is possible (and not hard) to install packages from 
source with an OpenMP-supporting compiler, and how to do so is in the R-admin 
manual.  The problems come in distributing them.
> >
> > The benefits of OpenMP are often overestimated, especially on 
desktop/laptop level hardware.  But it is available for the small (tiny?) 
proportion of users who need it.
> The University of Edinburgh is a charitable body, registered in Scotland, 
with registration number SC005336.
> ___
> R-SIG-Mac mailing list
> R-SIG-Mac@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-mac

___
R-SIG-Mac mailing list
R-SIG-Mac@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-mac


___
R-SIG-Mac mailing list
R-SIG-Mac@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-mac

Re: [R-SIG-Mac] OpenMP on CRAN

[Kevin Ushey]

Hi,

For what it's worth, it looks like it is still possible to use OpenMP
on macOS with the system toolchain. Using the example file here:

https://computing.llnl.gov/tutorials/openMP/samples/C/omp_hello.c

I can compile + link + run this on macOS 10.15.4 and with:

$ /usr/bin/clang -Xpreprocessor -fopenmp
-I/usr/local/opt/libomp/include -L/usr/local/opt/libomp/lib -lomp
omp_hello.c

As for whether this is 'safe', or whether R could conceivably bundle
and use its own copy of libomp is a separate question I cannot answer.
But at least this is a mechanism for enterprising users to enable
OpenMP in packages built from source if they so desire.

Best,
Kevin

On Thu, Apr 2, 2020 at 6:01 AM BATES Timothy  wrote:
>
> Moving to a compiler that drops support for OpenMP seems a sad choice, 
> especially now we’ve all climbed the learning curve of the non-Apple compiler 
> (the real barrier was lack of a pkg installer and that’s done now).
>
> Losing OpenMP for the CRAN version of OpenMx/umx (our SEM packages) would be 
> a big loss for users (for whom the CRAN version now supports OpenMP giving 
> them a 2-12x speed up). In general, R on Mac is made more viable by having 
> OpenMP
>
> Re Brian’s points, I’d say that the distribution problem is crucial: Packages 
> not on CRAN have dramatically diminished accessibility/useage.
>
> Second, a great range of compute-intensive problems are amenable to division 
> amongst cores, including nearly all models that take more than a nominal 
> amount of time to run: So simulations, CIs, bootstrapping, nearly everything 
> in genetics all speeds up.
>
> I’d say especially on desktop/laptop. The big advantage of multi blade 
> systems requires snowfall-type solutions, but desktops profit automatically 
> from their multi-core structure and don;’t have multiple processors (except 
> graphics, which no-one seems to be exploiting on CRAN-style R), so OpenMP is 
> their one trick. I’d hope not to lose it.
>
> Best, t
>
>
> > On 2 Apr 2020, at 05:18, Prof Brian Ripley  wrote:
> >
> > On 01/04/2020 22:02, Simon Urbanek wrote:
> >> JJB,
> >> 1. correct, there was too much trouble in this. But please feel free to 
> >> start a new thread about this here if you have strong opinions.
> >
> > Also note that it is possible (and not hard) to install packages from 
> > source with an OpenMP-supporting compiler, and how to do so is in the 
> > R-admin manual.  The problems come in distributing them.
> >
> > The benefits of OpenMP are often overestimated, especially on 
> > desktop/laptop level hardware.  But it is available for the small (tiny?) 
> > proportion of users who need it.
> The University of Edinburgh is a charitable body, registered in Scotland, 
> with registration number SC005336.
> ___
> R-SIG-Mac mailing list
> R-SIG-Mac@r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-mac

___
R-SIG-Mac mailing list
R-SIG-Mac@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-mac

[R-SIG-Mac] OpenMP on CRAN

[BATES Timothy]

Moving to a compiler that drops support for OpenMP seems a sad choice, 
especially now we’ve all climbed the learning curve of the non-Apple compiler 
(the real barrier was lack of a pkg installer and that’s done now).

Losing OpenMP for the CRAN version of OpenMx/umx (our SEM packages) would be a 
big loss for users (for whom the CRAN version now supports OpenMP giving them a 
2-12x speed up). In general, R on Mac is made more viable by having OpenMP

Re Brian’s points, I’d say that the distribution problem is crucial: Packages 
not on CRAN have dramatically diminished accessibility/useage.

Second, a great range of compute-intensive problems are amenable to division 
amongst cores, including nearly all models that take more than a nominal amount 
of time to run: So simulations, CIs, bootstrapping, nearly everything in 
genetics all speeds up.

I’d say especially on desktop/laptop. The big advantage of multi blade systems 
requires snowfall-type solutions, but desktops profit automatically from their 
multi-core structure and don;’t have multiple processors (except graphics, 
which no-one seems to be exploiting on CRAN-style R), so OpenMP is their one 
trick. I’d hope not to lose it.

Best, t


> On 2 Apr 2020, at 05:18, Prof Brian Ripley  wrote:
>
> On 01/04/2020 22:02, Simon Urbanek wrote:
>> JJB,
>> 1. correct, there was too much trouble in this. But please feel free to 
>> start a new thread about this here if you have strong opinions.
>
> Also note that it is possible (and not hard) to install packages from source 
> with an OpenMP-supporting compiler, and how to do so is in the R-admin 
> manual.  The problems come in distributing them.
>
> The benefits of OpenMP are often overestimated, especially on desktop/laptop 
> level hardware.  But it is available for the small (tiny?) proportion of 
> users who need it.
The University of Edinburgh is a charitable body, registered in Scotland, with 
registration number SC005336.
___
R-SIG-Mac mailing list
R-SIG-Mac@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-mac