Typo below ... not *_CPPFLAGS, but rather *_CFLAGS or *_CXXFLAGS: -fopenmp
is not a preprocessor option ...

Mikael

On 2024-07-05 4:53 pm, Mikael Jagan wrote:
Some thoughts in line below, not "certified" but maybe helpful.  Corrections
welcome ...

Mikael

From: "Balamuta, James" <balam...@illinois.edu>
To: Simon Urbanek <simon.urba...@r-project.org>
Cc: Timothy Bates <tim.ba...@ed.ac.uk>, Kasper Daniel Hansen
        <kasperdanielhan...@gmail.com>, "Sparapani, Rodney"
        <rspar...@mcw.edu>, R list <R-SIG-Mac@r-project.org>
Subject: Re: [R-SIG-Mac]  procedure to ship libomp.dylib run-time with
        package
Message-ID:
        
<ia0pr11mb781323fb004cd73f96e34e86eb...@ia0pr11mb7813.namprd11.prod.outlook.com>
        
Content-Type: text/plain; charset="utf-8"

Simon,

Thanks for the update. I’m aware of the linking instructions [1,2,3,4] and the 
purpose of configure{.ac} [5]. What I was not aware of was the CRAN macOS build 
machine were outfitted with the appropriate OpenMP setup [6, 7] in a post R 
4.0.0 world. Thus, more than just local configurations that adopted [8] could 
benefit from OpenMP especially since the top of [8] contains:

Warning! Everything described on this page is strictly experimental and not 
officially supported by CRAN, R-core or R Foundation. In may break at any time. 
The information is provided in the hope of being useful to some tech-savvy 
people. It is not intended for the regular R user.

Changing focus, I’m interested in re-enabling OpenMP with respect to 
RcppArmadillo, my questions here are:


    1.  When did libomp.dylib start shipping with the CRAN build? R 4.1, 4.2, 
4.3, or 4.4?
       *   This will allow me to petition to have an R version dependency 
change.

The binaries are archived, so one can just look "there" ...

      https://cran.r-project.org/bin/macosx/base/
      https://cran.r-project.org/bin/macosx/big-sur-x86_64/base/
      https://cran.r-project.org/bin/macosx/big-sur-arm64/base/

I see libomp.dylib in R 4.3.0, but not in R 4.2.3, just inspecting the arm64
ones ...

      $ lsbom -p f `pkgutil --bom R-4.4.1-arm64.pkg` | grep libomp[.]dylib
      ./R.framework/Versions/4.4-arm64/Resources/lib/libomp.dylib

But I don't see how that would motivate a dependency on R >= 4.3.  Not everyone
running R under macOS obtains R from CRAN, and not everyone running CRAN R has
omp.h in the search path ...

    2.  If a package only needs to compile code only once, then it’s sufficient to 
use the usual ` $(SHLIB_OPENMP_CXXFLAGS)` in src/Makevars with the appropriate ifdef 
pre-processor on including #include <omp.h>?
       *   This provides improved guidance related to best package development 
practices.

CRAN configures R with empty SHLIB_OPENMP_*FLAGS, which you can verify by
looking at any CRAN R installation's etc/Makeconf.

      $ grep SHLIB_OPENMP `R RHOME`/etc/Makeconf
      SHLIB_OPENMP_CFLAGS =
      SHLIB_OPENMP_CXXFLAGS =
      SHLIB_OPENMP_FFLAGS =

So, AFAICT, _automatic_ OpenMP support [1] still requires _regular_ users [2]
to install OpenMP-using packages from sources and still requires OpenMP-using
packages to have a src/Makevars containing

      PKG_CPPFLAGS = @THISPACKAGE_CPPFLAGS@
      PKG_LIBS = @THISPACKAGE_LIBS@

and a configure.ac arranging for those macros to be substituted with

      THISPACKAGE_CPPFLAGS=[-Xclang ]-fopenmp
      THISPACKAGE_LIBS=-lomp

conditional on a proper configure test (one that attempts to compile a program
including omp.h and calling (say) omp_get_num_threads()).  On macOS, you'd try
compiling twice, first with -fopenmp and again with -Xclang -fopenmp if -fopenmp
alone was not enough.  Simon has said this already ...

Even if CRAN did configure R with nonempty SHLIB_OPENMP_*FLAGS, it does not
(yet) bundle omp.h with R.

      $ lsbom -p f `pkgutil --bom R-4.4.1-arm64.pkg` | grep omp[.]h

The user would in many cases still need to arrange for omp.h to be found on the
search path, notably as R-exts says:

      For portability, any C/C++ code using the omp_* functions should
      include the omp.h header: some compilers (but not all) include it
      when OpenMP mode is switched on (e.g. via flag -fopenmp).

But maybe CRAN would consider shipping omp.h in addition to libomp.dylib
in the future ... ?

----

      [1] "automatic" in the sense of _not_ requiring the user to pass flags to
          R CMD INSTALL on the command line or through their personal Makevars
      [2] "regular" meaning that they obtained R from CRAN rather than, say,
          configuring R themselves with SHLIB_OPENMP_*FLAGS set

    3.  For packages that compile code first on CRAN and, then, dynamically on 
a user’s computer, what is the best approach to ensuring that instructions in 
[8] are set?
       *   This would ensure armadillo-oriented code could take full use of the 
cores on macOS.


As Simon has emphasized already, the only robust test for OpenMP support is
to attempt to compile a program with suitable flags.  Testing for a specific
user configuration (such as that described in mac.r-project.org/openmp/) is
fragile and restrictive for no reason.

If your package has an R function that invokes R CMD SHLIB, then that function
could perform a configure step, adjusting flags depending on the test result.

Best regards,

James

[1]: https://stat.ethz.ch/pipermail/r-sig-mac/2020-April/013346.html
[2]: https://stat.ethz.ch/pipermail/r-sig-mac/2019-May/012987.html
[3]: https://stat.ethz.ch/pipermail/r-sig-mac/2019-June/012996.html
[4]: https://stat.ethz.ch/pipermail/r-sig-mac/2020-May/013451.html
[5]: https://stackoverflow.com/a/42281495/1345455
[6]: https://github.com/RcppCore/RcppArmadillo/pull/219#issuecomment-383100722
[7]: https://stat.ethz.ch/pipermail/r-sig-mac/2020-April/013289.html
[8]: https://mac.r-project.org/openmp/



From: Simon Urbanek <simon.urba...@r-project.org>
Date: Friday, June 28, 2024 at 12:31 AM
To: Balamuta, James <balam...@illinois.edu>
Cc: Timothy Bates <tim.ba...@ed.ac.uk>, Kasper Daniel Hansen 
<kasperdanielhan...@gmail.com>, Sparapani, Rodney <rspar...@mcw.edu>, R list 
<R-SIG-Mac@r-project.org>
Subject: Re: [R-SIG-Mac] procedure to ship libomp.dylib run-time with package


On 27 Jun 2024, at 23:57, Balamuta, James <balam...@illinois.edu> wrote:

Simon,
   I think there is a communication issue regarding OpenMP for the macOS R 
community. That said, I’m both thrilled and delighted that I’m wrong in this 
case! To my knowledge, there was never a public facing note regarding `libomp`  
shipping with CRAN’s official R binary for mac, [1]. In fact, the last tracking 
I have with respect to OpenMP and R on macOS was its abrupt and painful removal 
that led to the instructions for experimental inclusion of OpenMP on mac within 
the developer website [2] (gently acknowledge by Prof. Ripley as being not 
current). This has led to a lot of lobotomization in compiled code regarding 
macOS that can now hopefully be adjusted for the multi-core world, [3].
   With this in mind, I suppose this means that we would need to revisit the 
`configure.ac` [3]. Potentially, as Prof. Ripley mentioned, we’re looking at 
including on path/detecting:
   `R 
RHOME`/lib/libomp.dylib/Library/Frameworks/R.framework/Resources/lib/libomp.dylib


That's entirely unnecessary - you should not be requiring specific file paths - 
that's not how linking tests work. All the instructions are on the OpenMP page 
for years - it is as simple as -lomp (which is standard almost everywhere). The 
only non-standard requirement compared to other platforms is that instead of 
-fopemp in C flags the package should also check for -Xclang -fopenmp.

If a package explicitly disables OpenMP for Apple compilers, it is its 
deliberate choice - it was never necessary. The whole point of configure 
scripts is to detect functionality by testing it and use it if available.

Cheers,
Simon


   Thus, the package binary downloaded from CRAN would correctly set and detect 
cores per:
   # install.packages("RcppArmadillo")
   # Determine the number of active threads
RcppArmadillo::armadillo_get_number_of_omp_threads()
[1] 1
   # Set number of threads
RcppArmadillo::armadillo_set_number_of_omp_threads(3)
   # Retrieve OpenMP threads after setting
RcppArmadillo::armadillo_get_number_of_omp_threads()
[1] 1
# This should be 3.
   I’m happy to talk more on the list or off the list.  Best,
   James
   [1]: 
https://urldefense.com/v3/__https://cran.r-project.org/doc/manuals/r-release/NEWS.html__;!!DZ3fjg!57krJgj2EE5sq_qaDsKls4jb5f99c2iyQaXHoJ_7OBQAbpapE-x_fRkzBh-Fy_cnsu-sUJpdhelgLqI006PhaLiBVYB5MDA$<https://urldefense.com/v3/__https:/cran.r-project.org/doc/manuals/r-release/NEWS.html__;!!DZ3fjg!57krJgj2EE5sq_qaDsKls4jb5f99c2iyQaXHoJ_7OBQAbpapE-x_fRkzBh-Fy_cnsu-sUJpdhelgLqI006PhaLiBVYB5MDA$>
[2]: 
https://urldefense.com/v3/__https://mac.r-project.org/openmp/__;!!DZ3fjg!57krJgj2EE5sq_qaDsKls4jb5f99c2iyQaXHoJ_7OBQAbpapE-x_fRkzBh-Fy_cnsu-sUJpdhelgLqI006PhaLiBmw9UV3o$<https://urldefense.com/v3/__https:/mac.r-project.org/openmp/__;!!DZ3fjg!57krJgj2EE5sq_qaDsKls4jb5f99c2iyQaXHoJ_7OBQAbpapE-x_fRkzBh-Fy_cnsu-sUJpdhelgLqI006PhaLiBmw9UV3o$>
[3]: 
https://urldefense.com/v3/__https://github.com/RcppCore/RcppArmadillo/blob/37461ba36472305c699263afc229919d37daa5e3/configure.ac*L109__;Iw!!DZ3fjg!57krJgj2EE5sq_qaDsKls4jb5f99c2iyQaXHoJ_7OBQAbpapE-x_fRkzBh-Fy_cnsu-sUJpdhelgLqI006PhaLiB8rnPPRw$<https://urldefense.com/v3/__https:/github.com/RcppCore/RcppArmadillo/blob/37461ba36472305c699263afc229919d37daa5e3/configure.ac*L109__;Iw!!DZ3fjg!57krJgj2EE5sq_qaDsKls4jb5f99c2iyQaXHoJ_7OBQAbpapE-x_fRkzBh-Fy_cnsu-sUJpdhelgLqI006PhaLiB8rnPPRw$>
    From: Simon Urbanek <simon.urba...@r-project.org>
Date: Thursday, June 27, 2024 at 10:25 AM
To: Balamuta, James <balam...@illinois.edu>
Cc: Timothy Bates <tim.ba...@ed.ac.uk>, Kasper Daniel Hansen 
<kasperdanielhan...@gmail.com>, Sparapani, Rodney <rspar...@mcw.edu>, R list 
<R-SIG-Mac@r-project.org>
Subject: Re: [R-SIG-Mac] procedure to ship libomp.dylib run-time with package
Most of the claims below are false - libomp is shipped with CRAN R for quite 
some time specifically so that packages which benefit from it significantly can 
enable OpenMP reasonably easily - they only need to add the flags as noted on 
the cited page and CRAN takes care of the rest. If there are any problems, 
please feel free to contact me.

Cheers,
Simon


On 26 Jun 2024, at 07:41, Balamuta, James <balam...@illinois.edu> wrote:

Tim,

Unfortunately, there isn't a nice way of including OpenMP for macOS within a 
package.  One potential avenue would be to use a `configure.ac` file to check 
for whether OpenMP headers are present on the macOS computer and, then, allow 
the correct OpenMP compilation flags to be set. However, this requires the user 
to compile from source instead of using a CRAN binary.

Unfortunately, there isn't going to be an easier way as OpenMP on macOS with R 
is experimental by nature.

Best regards,

James
________________________________
From: R-SIG-Mac <r-sig-mac-boun...@r-project.org> on behalf of Timothy Bates 
<tim.ba...@ed.ac.uk>
Sent: Tuesday, June 25, 2024 10:00 PM
To: Kasper Daniel Hansen <kasperdanielhan...@gmail.com>; Sparapani, Rodney 
<rspar...@mcw.edu>
Cc: R list <R-SIG-Mac@r-project.org>
Subject: Re: [R-SIG-Mac] procedure to ship libomp.dylib run-time with package

Thanks Kasper,
I read those lines and had installed the openmp software as instructed. It all 
works fine for me. But users don’t have my usr/ folder, not can a package write 
to their usr/ folder.

so what is confusing is how the user will get this lib. Where should make place 
libraries like libomp within the package so that if a user merely 
install.packages() it, the library will be found and loaded?
Best, tim


Sent from Outlook for 
iOS<https://urldefense.com/v3/__https://aka.ms/o0ukef__;!!DZ3fjg!_OUXHENJ_6aGlU-humThmTaUR5okhDKW2EGEY49AjJ4AN8eXBWtZCWE3O01pnyxbRpGQDhCl8MMLThVBbUXDuipI$
 >
________________________________
From: Kasper Daniel Hansen <kasperdanielhan...@gmail.com>
Sent: Tuesday, June 25, 2024 11:34:42 AM
To: Sparapani, Rodney <rspar...@mcw.edu>
Cc: Timothy Bates <tim.ba...@ed.ac.uk>; R list <R-SIG-Mac@r-project.org>
Subject: Re: [R-SIG-Mac] procedure to ship libomp.dylib run-time with package

This email was sent to you by someone outside the University.
You should only click on links or attachments if you are certain that the email 
is genuine and the content is safe.
If you read the next couple of sentences on that page, it says

The following are links to libomp OpenMP run-time built from official LLVM 
release sources using Xcode compilers. They are signed and support macOS 10.13 
(High Sierra) and higher. All tar-balls contain the system tree usr/local/lib 
and usr/local/include so the recommended installation is to type in Terminal:

     curl -O 
https://urldefense.com/v3/__https://mac.r-project.org/openmp/openmp-14.0.6-darwin20-Release.tar.gz__;!!DZ3fjg!_OUXHENJ_6aGlU-humThmTaUR5okhDKW2EGEY49AjJ4AN8eXBWtZCWE3O01pnyxbRpGQDhCl8MMLThVBbWfUKf7B$<https://urldefense.com/v3/__https:/mac.r-project.org/openmp/openmp-14.0.6-darwin20-Release.tar.gz__;!!DZ3fjg!_OUXHENJ_6aGlU-humThmTaUR5okhDKW2EGEY49AjJ4AN8eXBWtZCWE3O01pnyxbRpGQDhCl8MMLThVBbWfUKf7B$>
     sudo tar fvxz openmp-14.0.6-darwin20-Release.tar.gz -C /


I think that is pretty clear? Note the -C / as part of the last line, that 
tells tar where to put it on the system.

On Tue, Jun 25, 2024 at 8:20 AM Sparapani, Rodney via R-SIG-Mac 
<r-sig-mac@r-project.org<mailto:r-sig-mac@r-project.org>> wrote:
Hi Tim:

/usr/local/lib it seems.  See the following page
https://urldefense.com/v3/__https://mac.r-project.org/openmp/__;!!DZ3fjg!_OUXHENJ_6aGlU-humThmTaUR5okhDKW2EGEY49AjJ4AN8eXBWtZCWE3O01pnyxbRpGQDhCl8MMLThVBbaQGheD_$<https://urldefense.com/v3/__https:/mac.r-project.org/openmp/__;!!DZ3fjg!_OUXHENJ_6aGlU-humThmTaUR5okhDKW2EGEY49AjJ4AN8eXBWtZCWE3O01pnyxbRpGQDhCl8MMLThVBbaQGheD_$>

--
Rodney Sparapani, Associate Professor of Biostatistics, He/Him/His
Vice President, Wisconsin Chapter of the American Statistical Association
Institute for Health and Equity, Division of Biostatistics
Medical College of Wisconsin, Milwaukee Campus

If this is outside of working hours, then please respond when convenient.

From: R-SIG-Mac <r-sig-mac-boun...@r-project.org<mailto:r-sig-mac-boun...@r-project.org>> 
on behalf of Timothy Bates <tim.ba...@ed.ac.uk<mailto:tim.ba...@ed.ac.uk>>
Date: Monday, June 24, 2024 at 4:00 PM
To: R list <R-SIG-Mac@r-project.org<mailto:R-SIG-Mac@r-project.org>>
Subject: [R-SIG-Mac] procedure to ship libomp.dylib run-time with package
ATTENTION: This email originated from a sender outside of MCW. Use caution when 
clicking on links or opening attachments.
________________________________

At 
https://urldefense.com/v3/__https://mac.r-project.org/openmp/__;!!H8mHWRdzp34!4HZ0uFxJQ121gF_TnX99OuMlgIXM600pq8wCbWruoLsdLTtaFgRB5_gMopCtxD0N9M4tUOO-7hBGADHx1sk$<https://urldefense.com/v3/__https:/mac.r-project.org/openmp/__;!!H8mHWRdzp34!4HZ0uFxJQ121gF_TnX99OuMlgIXM600pq8wCbWruoLsdLTtaFgRB5_gMopCtxD0N9M4tUOO-7hBGADHx1sk$><https://urldefense.com/v3/__https:/mac.r-project.org/openmp/__;!!H8mHWRdzp34!4HZ0uFxJQ121gF_TnX99OuMlgIXM600pq8wCbWruoLsdLTtaFgRB5_gMopCtxD0N9M4tUOO-7hBGADHx1sk$%3chttps:/urldefense.com/v3/__https:/mac.r-project.org/openmp/__;!!H8mHWRdzp34!4HZ0uFxJQ121gF_TnX99OuMlgIXM600pq8wCbWruoLsdLTtaFgRB5_gMopCtxD0N9M4tUOO-7hBGADHx1sk$%3e>
  it  says
"any package you compile against libomp.dylib will need that run-time so you 
have to ship it with your package or have users install it”

Are there  any  instructions on what to do/where to put libomp.dylib so it will 
be found?

I’ve compiled a package (OpenMx) in which OpenMP is working locally but other 
users get the error:

require(OpenMx)
Error: package or namespace load failed for ‘OpenMx’ in dyn.load(file, DLLpath 
= DLLpath, ...):
unable to load shared object 
'/Users/***/Library/R/x86_64/4.4/library/OpenMx/libs/OpenMx.so':

dlopen(/Users/***/Library/R/x86_64/4.4/library/OpenMx/libs/OpenMx.so, 0x0006): 
Library not loaded: 
/Library/Frameworks/R.framework/Versions/4.3-x86_64/Resources/lib/libomp.dylib


The University of Edinburgh is a charitable body, registered in Scotland, with 
registration number SC005336. Is e buidheann carthannais a th’ ann an Oilthigh 
Dhùn Èideann, clàraichte an Alba, àireamh clàraidh SC005336.
_______________________________________________
R-SIG-Mac mailing list
R-SIG-Mac@r-project.org<mailto:R-SIG-Mac@r-project.org>
https://urldefense.com/v3/__https://stat.ethz.ch/mailman/listinfo/r-sig-mac__;!!H8mHWRdzp34!4HZ0uFxJQ121gF_TnX99OuMlgIXM600pq8wCbWruoLsdLTtaFgRB5_gMopCtxD0N9M4tUOO-7hBGdICDldE$<https://urldefense.com/v3/__https:/stat.ethz.ch/mailman/listinfo/r-sig-mac__;!!H8mHWRdzp34!4HZ0uFxJQ121gF_TnX99OuMlgIXM600pq8wCbWruoLsdLTtaFgRB5_gMopCtxD0N9M4tUOO-7hBGdICDldE$><https://urldefense.com/v3/__https:/stat.ethz.ch/mailman/listinfo/r-sig-mac__;!!H8mHWRdzp34!4HZ0uFxJQ121gF_TnX99OuMlgIXM600pq8wCbWruoLsdLTtaFgRB5_gMopCtxD0N9M4tUOO-7hBGdICDldE$%3chttps:/urldefense.com/v3/__https:/stat.ethz.ch/mailman/listinfo/r-sig-mac__;!!H8mHWRdzp34!4HZ0uFxJQ121gF_TnX99OuMlgIXM600pq8wCbWruoLsdLTtaFgRB5_gMopCtxD0N9M4tUOO-7hBGdICDldE$%3e>

         [[alternative HTML version deleted]]

_______________________________________________
R-SIG-Mac mailing list
R-SIG-Mac@r-project.org<mailto:R-SIG-Mac@r-project.org>
https://urldefense.com/v3/__https://stat.ethz.ch/mailman/listinfo/r-sig-mac__;!!DZ3fjg!_OUXHENJ_6aGlU-humThmTaUR5okhDKW2EGEY49AjJ4AN8eXBWtZCWE3O01pnyxbRpGQDhCl8MMLThVBbYOwLI1d$<https://urldefense.com/v3/__https:/stat.ethz.ch/mailman/listinfo/r-sig-mac__;!!DZ3fjg!_OUXHENJ_6aGlU-humThmTaUR5okhDKW2EGEY49AjJ4AN8eXBWtZCWE3O01pnyxbRpGQDhCl8MMLThVBbYOwLI1d$>


--
Best,
Kasper

         [[alternative HTML version deleted]]

_______________________________________________
R-SIG-Mac mailing list
R-SIG-Mac@r-project.org
https://urldefense.com/v3/__https://stat.ethz.ch/mailman/listinfo/r-sig-mac__;!!DZ3fjg!_OUXHENJ_6aGlU-humThmTaUR5okhDKW2EGEY49AjJ4AN8eXBWtZCWE3O01pnyxbRpGQDhCl8MMLThVBbYOwLI1d$<https://urldefense.com/v3/__https:/stat.ethz.ch/mailman/listinfo/r-sig-mac__;!!DZ3fjg!_OUXHENJ_6aGlU-humThmTaUR5okhDKW2EGEY49AjJ4AN8eXBWtZCWE3O01pnyxbRpGQDhCl8MMLThVBbYOwLI1d$>

[[alternative HTML version deleted]]

_______________________________________________
R-SIG-Mac mailing list
R-SIG-Mac@r-project.org
https://urldefense.com/v3/__https://stat.ethz.ch/mailman/listinfo/r-sig-mac__;!!DZ3fjg!9XPrFiayt1-V0BmNqWv4wn0V_G6ZIC6DgkviDYpRV4s-dmAnj9Qp_QFEHPG5dbO-Sk0IhdfC3lB0qeeJBDE2ann7kz0eS7A$<https://urldefense.com/v3/__https:/stat.ethz.ch/mailman/listinfo/r-sig-mac__;!!DZ3fjg!9XPrFiayt1-V0BmNqWv4wn0V_G6ZIC6DgkviDYpRV4s-dmAnj9Qp_QFEHPG5dbO-Sk0IhdfC3lB0qeeJBDE2ann7kz0eS7A$>


        [[alternative HTML version deleted]]



------------------------------

Subject: Digest Footer

_______________________________________________
R-SIG-Mac mailing list
R-SIG-Mac@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-mac


------------------------------


_______________________________________________
R-SIG-Mac mailing list
R-SIG-Mac@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-mac

Reply via email to