Re: [R-SIG-Mac] Problems with R-4.4.2 in MacOS Sequoia with Intel chip
Simon,
Excellent! Many thanks! I used your command to fix the problem for the time
being and also opened an issue for “torch” package maintainers:
https://github.com/mlverse/torch/issues/1213
Your support is much appreciated.
Best
Gilberto
Prof Dr Gilberto Camara
Senior Researcher
Getulio Vargas Foundation (FGV)
National Institute for Space Research (INPE), Brazil
https://gilbertocamara.org/
=
> On 23 Nov 2024, at 04:44, Simon Urbanek wrote:
>
> Gilberto,
>
> thanks! TLDR; its a bug in the torch package - fix below.
>
> It took a while to sort out your example as it was very incomplete, here are
> the full required steps:
>
> ---
> ## sits doesn't install tests, so have the get them from sources:
> download.packages("sits", type='source')
> untar(Sys.glob("sits_*.tar.gz"))
> ## there are many other packages required for the tests
> install.packages(c("sits","torch","lwgeom","randomForest","gdalcubes","xgboost","devtools"))
>
> ## also torch installation is incomplete so it requires additional downloads
> on first load
> # (NB: this looks like a serious problem - does it mean torch bypasses CRAN
> checks?)
> #> library(torch)
> # - Additional software needs to be downloaded and installed for torch to
> work correctly.
> #Do you want to continue? (Yes/no/cancel) yes
>
> devtools::test(“sits”)
> ---
>
> But all of it is not necessary since the key is that the tests load both
> data.table and torch so this is a more minimal example (based on one of the
> sits tests):
>
> library(data.table)
> library(sits)
> sits_tuning(samples_modis_ndvi,
>ml_method = sits_tempcnn(),
>params = sits_tuning_hparams(epochs = 10, optimizer =
> torch::optim_adam,
>opt_hparams = list(lr = choice(0.01, 0.05, 0.001, 0.0005))),
>trials = 2,multicores = 1, progress = FALSE
> )
>
> The use of data.table is just an example, any OpenMP-enabled package will
> trigger the bug. As the error you sent indicated the problem is the
> incompatibility of the run-time used by torch. If you use any R package with
> OpenMP support (which loads the official OpenMP run-time - in above example
> it is data.table) and then run torch afterwards, torch will fail, because it
> loads its private OpenMP run-time which is incompatible. This is precisely
> why we provide a common run-time and torch fails to use it so I would
> recommend filing a bug report with the torch package. It is easily fixed with
> something like
>
> system(
> paste("ln","-sfn",shQuote(file.path(R.home(),"lib","libomp.dylib")),
> shQuote(system.file("lib","libiomp5.dylib",package="torch"
>
> to point torch to the common run-time which guarantees compatibility between
> packages. After that fix torch and the above example works again.
>
> The current torch setup is really bad by design, because it essentially means
> that no one may use OpenMP as it will always clash with its private library
> (since no one else can use it). Even though it may not bite many people now
> given the sequence of packages involved, if we decided to enable OpenMP in
> CRAN build of R then torch won't work at all until this is fixed.
>
> Cheers,
> Simon
>
>
>> On Nov 23, 2024, at 5:36 PM, Gilberto Camara wrote:
>>
>> Dear Simon
>>
>> I understand your responses and your point. I hope you'll be able to
>> understand my problem. It is hard to provide a simple working example, since
>> we need to find a case where a package loads an OpenMP library that
>> conflicts with the OpenMP library loaded by R.
>>
>> Replicating the problems I'm facing can be done by following the same
>> actions I did. Please run the following commands on an R terminal to get the
>> same error as we have using version R-4.4.2
>>
>> == r code
>> % install.packages(“sits”)
>> % devtools::test(“sits”)
>> ===
>>
>> You should get the same error as I do.
>>
>> As I stated earlier, there is a conflict between “libomp.dylib" loaded in
>> the R core libraries and the "libiomp5.dylib" loaded by the “torch” package.
>> Since our package (“sits”) uses the torch package, I did the following
>> experiment: (a) removed “libomp.dylib” from the R core libraries; and (b)
>> ran the same two lines above. Everything works.
>>
>> Meanwhile, I will try to produce an MWE that does not need to use the “sits”
>> package.
>>
>> Best regards
>> Gilberto
>>
>>
>> Prof Dr Gilberto Camara
>> Senior Researcher
>> Getulio Vargas Foundation (FGV)
>> National Institute for Space Research (INPE), Brazil
>> https://gilbertocamara.org/
>> =
>>
>>
>
> ___
> R-SIG-Mac mailing list
> [email protected]
> https://stat.ethz.ch/mailman/listinfo/r-sig-mac
___
R-SIG-Mac mailing list
[email protected]
https://stat.ethz.ch/mailman/listinfo/r-sig-mac
Re: [R-SIG-Mac] Problems with R-4.4.2 in MacOS Sequoia with Intel chip
Gilberto,
thanks! TLDR; its a bug in the torch package - fix below.
It took a while to sort out your example as it was very incomplete, here are
the full required steps:
---
## sits doesn't install tests, so have the get them from sources:
download.packages("sits", type='source')
untar(Sys.glob("sits_*.tar.gz"))
## there are many other packages required for the tests
install.packages(c("sits","torch","lwgeom","randomForest","gdalcubes","xgboost","devtools"))
## also torch installation is incomplete so it requires additional downloads on
first load
# (NB: this looks like a serious problem - does it mean torch bypasses CRAN
checks?)
#> library(torch)
# - Additional software needs to be downloaded and installed for torch to work
correctly.
#Do you want to continue? (Yes/no/cancel) yes
devtools::test(“sits”)
---
But all of it is not necessary since the key is that the tests load both
data.table and torch so this is a more minimal example (based on one of the
sits tests):
library(data.table)
library(sits)
sits_tuning(samples_modis_ndvi,
ml_method = sits_tempcnn(),
params = sits_tuning_hparams(epochs = 10, optimizer = torch::optim_adam,
opt_hparams = list(lr = choice(0.01, 0.05, 0.001, 0.0005))),
trials = 2,multicores = 1, progress = FALSE
)
The use of data.table is just an example, any OpenMP-enabled package will
trigger the bug. As the error you sent indicated the problem is the
incompatibility of the run-time used by torch. If you use any R package with
OpenMP support (which loads the official OpenMP run-time - in above example it
is data.table) and then run torch afterwards, torch will fail, because it loads
its private OpenMP run-time which is incompatible. This is precisely why we
provide a common run-time and torch fails to use it so I would recommend filing
a bug report with the torch package. It is easily fixed with something like
system(
paste("ln","-sfn",shQuote(file.path(R.home(),"lib","libomp.dylib")),
shQuote(system.file("lib","libiomp5.dylib",package="torch"
to point torch to the common run-time which guarantees compatibility between
packages. After that fix torch and the above example works again.
The current torch setup is really bad by design, because it essentially means
that no one may use OpenMP as it will always clash with its private library
(since no one else can use it). Even though it may not bite many people now
given the sequence of packages involved, if we decided to enable OpenMP in CRAN
build of R then torch won't work at all until this is fixed.
Cheers,
Simon
> On Nov 23, 2024, at 5:36 PM, Gilberto Camara wrote:
>
> Dear Simon
>
> I understand your responses and your point. I hope you'll be able to
> understand my problem. It is hard to provide a simple working example, since
> we need to find a case where a package loads an OpenMP library that conflicts
> with the OpenMP library loaded by R.
>
> Replicating the problems I'm facing can be done by following the same actions
> I did. Please run the following commands on an R terminal to get the same
> error as we have using version R-4.4.2
>
> == r code
> % install.packages(“sits”)
> % devtools::test(“sits”)
> ===
>
> You should get the same error as I do.
>
> As I stated earlier, there is a conflict between “libomp.dylib" loaded in the
> R core libraries and the "libiomp5.dylib" loaded by the “torch” package.
> Since our package (“sits”) uses the torch package, I did the following
> experiment: (a) removed “libomp.dylib” from the R core libraries; and (b)
> ran the same two lines above. Everything works.
>
> Meanwhile, I will try to produce an MWE that does not need to use the “sits”
> package.
>
> Best regards
> Gilberto
>
>
> Prof Dr Gilberto Camara
> Senior Researcher
> Getulio Vargas Foundation (FGV)
> National Institute for Space Research (INPE), Brazil
> https://gilbertocamara.org/
> =
>
>
___
R-SIG-Mac mailing list
[email protected]
https://stat.ethz.ch/mailman/listinfo/r-sig-mac
Re: [R-SIG-Mac] Problems with R-4.4.2 in MacOS Sequoia with Intel chip
Dear Simon I understand your responses and your point. I hope you'll be able to understand my problem. It is hard to provide a simple working example, since we need to find a case where a package loads an OpenMP library that conflicts with the OpenMP library loaded by R. Replicating the problems I'm facing can be done by following the same actions I did. Please run the following commands on an R terminal to get the same error as we have using version R-4.4.2 == r code % install.packages(“sits”) % devtools::test(“sits”) === You should get the same error as I do. As I stated earlier, there is a conflict between “libomp.dylib" loaded in the R core libraries and the "libiomp5.dylib" loaded by the “torch” package. Since our package (“sits”) uses the torch package, I did the following experiment: (a) removed “libomp.dylib” from the R core libraries; and (b) ran the same two lines above. Everything works. Meanwhile, I will try to produce an MWE that does not need to use the “sits” package. Best regards Gilberto Prof Dr Gilberto Camara Senior Researcher Getulio Vargas Foundation (FGV) National Institute for Space Research (INPE), Brazil https://gilbertocamara.org/ = > On 22 Nov 2024, at 20:54, Simon Urbanek wrote: > > Gilberto, > > >> On Nov 23, 2024, at 12:40 PM, Gilberto Camara >> wrote: >> >> Dear Simon >> >> You state that “R itself doesn't load OpenMP”. So why is libomp.dylib >> loaded in core libraries of R-4.4.2 for Mac? >> > > It is not loaded - you can check yourself: > > echo '' | R CMD sh -c 'DYLD_PRINT_LIBRARIES=1 $R_HOME/bin/exec/R' 2>&1 | grep > libomp > > >> Please see the contents of the core R libaries for R-4.2 and R-4.4.2 in my >> MacOS. In both cases, these are downloads straight from CRAN. >> >> Why has libomp.dylib been included in the R core libraries? >> > > > We simply distribute it as a convenience, because some packages (like > data.table) benefit hugely from OpenMP so by providing a fixed location it > allows them to avoid shipping it separately and they all can use one shared > version. See also https://mac.r-project.org/openmp/ for details > > But as noted, I don't think this has anything to do with your problem > precisely because it is not loaded by R itself. > > As I said before I would appreciate is you avoided random speculations (there > are many changes between R 4.2 and 4.4 especially in the packages between the > two versions) as that is not productive, but instead share a reproducible > example that we can look at. Any discussion is entirely pointless until you > do that. > > Cheers, > Simon > > > >> Best regards >> Gilberto >> >> >> >> R 4.2 >> >> >> ❯ ls -lha /Library/Frameworks/R.framework/Versions/4.2/Resources/lib >> total 19976 >> drwxrwxr-x 15 root admin 480B 22 Nov 15:28 . >> drwxrwxr-x 18 root admin 576B 22 Nov 15:28 .. >> -rwxrwxr-x 1 root admin 4,0M 17 Mar 2023 libR.dylib >> drwxrwxr-x 3 root admin96B 17 Mar 2023 libR.dylib.dSYM >> -rwxrwxr-x 1 root admin 221K 17 Mar 2023 libRblas.0.dylib >> drwxrwxr-x 3 root admin96B 17 Mar 2023 libRblas.0.dylib.dSYM >> lrwxrwxr-x 1 root admin16B 22 Nov 15:28 libRblas.dylib -> >> libRblas.0.dylib >> drwxrwxr-x 3 root admin96B 17 Mar 2023 libRblas.dylib.dSYM >> -rwxrwxr-x 1 root admin 151K 17 Mar 2023 libRblas.vecLib.dylib >> drwxrwxr-x 3 root admin96B 17 Mar 2023 libRblas.vecLib.dylib.dSYM >> -rwxrwxr-x 1 root admin 2,2M 17 Mar 2023 libRlapack.dylib >> drwxrwxr-x 3 root admin96B 17 Mar 2023 libRlapack.dylib.dSYM >> -rw-rw-r-- 1 root admin 157K 17 Mar 2023 libgcc_s.1.dylib >> -rwxrwxr-x 1 root admin 2,7M 17 Mar 2023 libgfortran.5.dylib >> -rwxrwxr-x 1 root admin 302K 17 Mar 2023 libquadmath.0.dylib >> >> >> R 4.4.2 >> >> >> ❯ ls -lha /Library/Frameworks/R.framework/Versions/4.4-x86_64/Resources/lib >> total 27320 >> drwxrwxr-x 17 root admin 544B 22 Nov 15:03 . >> drwxrwxr-x 17 root admin 544B 22 Nov 15:03 .. >> -rwxrwxr-x 1 root admin 5,3M 31 Out 22:15 libR.dylib >> drwxrwxr-x 3 root admin96B 31 Out 22:14 libR.dylib.dSYM >> -rwxrwxr-x 1 root admin 238K 31 Out 22:15 libRblas.0.dylib >> drwxrwxr-x 3 root admin96B 31 Out 22:14 libRblas.0.dylib.dSYM >> lrwxrwxr-x 1 root admin16B 22 Nov 15:03 libRblas.dylib -> >> libRblas.0.dylib >> drwxrwxr-x 3 root admin96B 31 Out 22:14 libRblas.dylib.dSYM >> -rwxrwxr-x 1 root admin 151K 31 Out 22:15 libRblas.vecLib.dylib >> drwxrwxr-x 3 root admin96B 31 Out 22:14 libRblas.vecLib.dylib.dSYM >> -rwxrwxr-x 1 root admin 2,3M 31 Out 22:15 libRlapack.dylib >> drwxrwxr-x 3 root admin96B 31 Out 22:14 libRlapack.dylib.dSYM >> -rw-rw-r-- 1 root admin 183K 31 Out 22:15 libgcc_s.1.1.dylib >> -rwxrwxr-x 1 root admin 3,3M 31 Out 22:15 libgfortran.5.dylib >> -rwxrwxr-x 1 root admin 1,5M 31 Out
Re: [R-SIG-Mac] Problems with R-4.4.2 in MacOS Sequoia with Intel chip
Gilberto, > On Nov 23, 2024, at 12:40 PM, Gilberto Camara wrote: > > Dear Simon > > You state that “R itself doesn't load OpenMP”. So why is libomp.dylib loaded > in core libraries of R-4.4.2 for Mac? > It is not loaded - you can check yourself: echo '' | R CMD sh -c 'DYLD_PRINT_LIBRARIES=1 $R_HOME/bin/exec/R' 2>&1 | grep libomp > Please see the contents of the core R libaries for R-4.2 and R-4.4.2 in my > MacOS. In both cases, these are downloads straight from CRAN. > > Why has libomp.dylib been included in the R core libraries? > We simply distribute it as a convenience, because some packages (like data.table) benefit hugely from OpenMP so by providing a fixed location it allows them to avoid shipping it separately and they all can use one shared version. See also https://mac.r-project.org/openmp/ for details But as noted, I don't think this has anything to do with your problem precisely because it is not loaded by R itself. As I said before I would appreciate is you avoided random speculations (there are many changes between R 4.2 and 4.4 especially in the packages between the two versions) as that is not productive, but instead share a reproducible example that we can look at. Any discussion is entirely pointless until you do that. Cheers, Simon > Best regards > Gilberto > > > > R 4.2 > > > ❯ ls -lha /Library/Frameworks/R.framework/Versions/4.2/Resources/lib > total 19976 > drwxrwxr-x 15 root admin 480B 22 Nov 15:28 . > drwxrwxr-x 18 root admin 576B 22 Nov 15:28 .. > -rwxrwxr-x 1 root admin 4,0M 17 Mar 2023 libR.dylib > drwxrwxr-x 3 root admin96B 17 Mar 2023 libR.dylib.dSYM > -rwxrwxr-x 1 root admin 221K 17 Mar 2023 libRblas.0.dylib > drwxrwxr-x 3 root admin96B 17 Mar 2023 libRblas.0.dylib.dSYM > lrwxrwxr-x 1 root admin16B 22 Nov 15:28 libRblas.dylib -> > libRblas.0.dylib > drwxrwxr-x 3 root admin96B 17 Mar 2023 libRblas.dylib.dSYM > -rwxrwxr-x 1 root admin 151K 17 Mar 2023 libRblas.vecLib.dylib > drwxrwxr-x 3 root admin96B 17 Mar 2023 libRblas.vecLib.dylib.dSYM > -rwxrwxr-x 1 root admin 2,2M 17 Mar 2023 libRlapack.dylib > drwxrwxr-x 3 root admin96B 17 Mar 2023 libRlapack.dylib.dSYM > -rw-rw-r-- 1 root admin 157K 17 Mar 2023 libgcc_s.1.dylib > -rwxrwxr-x 1 root admin 2,7M 17 Mar 2023 libgfortran.5.dylib > -rwxrwxr-x 1 root admin 302K 17 Mar 2023 libquadmath.0.dylib > > > R 4.4.2 > > > ❯ ls -lha /Library/Frameworks/R.framework/Versions/4.4-x86_64/Resources/lib > total 27320 > drwxrwxr-x 17 root admin 544B 22 Nov 15:03 . > drwxrwxr-x 17 root admin 544B 22 Nov 15:03 .. > -rwxrwxr-x 1 root admin 5,3M 31 Out 22:15 libR.dylib > drwxrwxr-x 3 root admin96B 31 Out 22:14 libR.dylib.dSYM > -rwxrwxr-x 1 root admin 238K 31 Out 22:15 libRblas.0.dylib > drwxrwxr-x 3 root admin96B 31 Out 22:14 libRblas.0.dylib.dSYM > lrwxrwxr-x 1 root admin16B 22 Nov 15:03 libRblas.dylib -> > libRblas.0.dylib > drwxrwxr-x 3 root admin96B 31 Out 22:14 libRblas.dylib.dSYM > -rwxrwxr-x 1 root admin 151K 31 Out 22:15 libRblas.vecLib.dylib > drwxrwxr-x 3 root admin96B 31 Out 22:14 libRblas.vecLib.dylib.dSYM > -rwxrwxr-x 1 root admin 2,3M 31 Out 22:15 libRlapack.dylib > drwxrwxr-x 3 root admin96B 31 Out 22:14 libRlapack.dylib.dSYM > -rw-rw-r-- 1 root admin 183K 31 Out 22:15 libgcc_s.1.1.dylib > -rwxrwxr-x 1 root admin 3,3M 31 Out 22:15 libgfortran.5.dylib > -rwxrwxr-x 1 root admin 1,5M 31 Out 22:15 libomp.dylib > -rwxrwxr-x 1 root admin 376K 31 Out 22:15 libquadmath.0.dylib > drwxrwxr-x 3 root admin96B 31 Out 22:11 pkgconfig > > > > Prof Dr Gilberto Camara > Senior Researcher > Getulio Vargas Foundation (FGV) > National Institute for Space Research (INPE), Brazil > https://gilbertocamara.org/ > = > > > > > >> On 22 Nov 2024, at 19:48, Simon Urbanek wrote: >> >> Gilberto, >> >> you still didn't provide any useful examples, and all you have are baseless >> speculations. R itself doesn't load OpenMP, so the problem could very well >> be in the other software or libraries you use - it may have nothing to do >> with R, but without an example we can neither tell nor help. Based on this >> there is nothing to solve "at the highest level of R core developers for >> Mac" - it has to start with you providing a working example as no one else >> has that problem. >> >> Cheers, >> Simon >> >> >>> On Nov 23, 2024, at 7:30 AM, Gilberto Camara >>> wrote: >>> >>> Dear Peter and Simon >>> >>> I may have found the cause of the crashes in R-4.4.2 in the Intel MacOs >>> environment. In R-4.4.2 for MacOS, the “libomp.dylib” is loaded from the >>> CRAN version. Comparing with earlier version, this is a new addition to the >>> core R libraries. The OpenMP library is not provided in R-4.2. >>> >>> It so happens that the “torch” package is
Re: [R-SIG-Mac] Problems with R-4.4.2 in MacOS Sequoia with Intel chip
Dear Simon You state that “R itself doesn't load OpenMP”. So why is libomp.dylib loaded in core libraries of R-4.4.2 for Mac? Please see the contents of the core R libaries for R-4.2 and R-4.4.2 in my MacOS. In both cases, these are downloads straight from CRAN. Why has libomp.dylib been included in the R core libraries? Best regards Gilberto R 4.2 ❯ ls -lha /Library/Frameworks/R.framework/Versions/4.2/Resources/lib total 19976 drwxrwxr-x 15 root admin 480B 22 Nov 15:28 . drwxrwxr-x 18 root admin 576B 22 Nov 15:28 .. -rwxrwxr-x 1 root admin 4,0M 17 Mar 2023 libR.dylib drwxrwxr-x 3 root admin96B 17 Mar 2023 libR.dylib.dSYM -rwxrwxr-x 1 root admin 221K 17 Mar 2023 libRblas.0.dylib drwxrwxr-x 3 root admin96B 17 Mar 2023 libRblas.0.dylib.dSYM lrwxrwxr-x 1 root admin16B 22 Nov 15:28 libRblas.dylib -> libRblas.0.dylib drwxrwxr-x 3 root admin96B 17 Mar 2023 libRblas.dylib.dSYM -rwxrwxr-x 1 root admin 151K 17 Mar 2023 libRblas.vecLib.dylib drwxrwxr-x 3 root admin96B 17 Mar 2023 libRblas.vecLib.dylib.dSYM -rwxrwxr-x 1 root admin 2,2M 17 Mar 2023 libRlapack.dylib drwxrwxr-x 3 root admin96B 17 Mar 2023 libRlapack.dylib.dSYM -rw-rw-r-- 1 root admin 157K 17 Mar 2023 libgcc_s.1.dylib -rwxrwxr-x 1 root admin 2,7M 17 Mar 2023 libgfortran.5.dylib -rwxrwxr-x 1 root admin 302K 17 Mar 2023 libquadmath.0.dylib R 4.4.2 ❯ ls -lha /Library/Frameworks/R.framework/Versions/4.4-x86_64/Resources/lib total 27320 drwxrwxr-x 17 root admin 544B 22 Nov 15:03 . drwxrwxr-x 17 root admin 544B 22 Nov 15:03 .. -rwxrwxr-x 1 root admin 5,3M 31 Out 22:15 libR.dylib drwxrwxr-x 3 root admin96B 31 Out 22:14 libR.dylib.dSYM -rwxrwxr-x 1 root admin 238K 31 Out 22:15 libRblas.0.dylib drwxrwxr-x 3 root admin96B 31 Out 22:14 libRblas.0.dylib.dSYM lrwxrwxr-x 1 root admin16B 22 Nov 15:03 libRblas.dylib -> libRblas.0.dylib drwxrwxr-x 3 root admin96B 31 Out 22:14 libRblas.dylib.dSYM -rwxrwxr-x 1 root admin 151K 31 Out 22:15 libRblas.vecLib.dylib drwxrwxr-x 3 root admin96B 31 Out 22:14 libRblas.vecLib.dylib.dSYM -rwxrwxr-x 1 root admin 2,3M 31 Out 22:15 libRlapack.dylib drwxrwxr-x 3 root admin96B 31 Out 22:14 libRlapack.dylib.dSYM -rw-rw-r-- 1 root admin 183K 31 Out 22:15 libgcc_s.1.1.dylib -rwxrwxr-x 1 root admin 3,3M 31 Out 22:15 libgfortran.5.dylib -rwxrwxr-x 1 root admin 1,5M 31 Out 22:15 libomp.dylib -rwxrwxr-x 1 root admin 376K 31 Out 22:15 libquadmath.0.dylib drwxrwxr-x 3 root admin96B 31 Out 22:11 pkgconfig Prof Dr Gilberto Camara Senior Researcher Getulio Vargas Foundation (FGV) National Institute for Space Research (INPE), Brazil https://gilbertocamara.org/ = > On 22 Nov 2024, at 19:48, Simon Urbanek wrote: > > Gilberto, > > you still didn't provide any useful examples, and all you have are baseless > speculations. R itself doesn't load OpenMP, so the problem could very well be > in the other software or libraries you use - it may have nothing to do with > R, but without an example we can neither tell nor help. Based on this there > is nothing to solve "at the highest level of R core developers for Mac" - it > has to start with you providing a working example as no one else has that > problem. > > Cheers, > Simon > > >> On Nov 23, 2024, at 7:30 AM, Gilberto Camara wrote: >> >> Dear Peter and Simon >> >> I may have found the cause of the crashes in R-4.4.2 in the Intel MacOs >> environment. In R-4.4.2 for MacOS, the “libomp.dylib” is loaded from the >> CRAN version. Comparing with earlier version, this is a new addition to the >> core R libraries. The OpenMP library is not provided in R-4.2. >> >> It so happens that the “torch” package is also including the an OpenMP >> library (“libiomp5.dylib”) which is causing the crash. In previous R >> versions, this behavoir did not cause problems, since R did not provide the >> OpenMP library by default. >> >> Thus, the problem needs to be solved at the highest level of R core >> developers for Mac. Please bear in mind that the “torch” package is >> essential for running deep learning algorithms in R. I hope you can find a >> solution that preserves the use of “torch” in R. I am copying the message to >> Daniel Falbel, who is the maintainer of the “torch” package. >> >> Your support in solving this problem is highly appreciated. >> >> Best regards >> Gilberto >> >> Prof Dr Gilberto Camara >> Senior Researcher >> Getulio Vargas Foundation (FGV) >> National Institute for Space Research (INPE), Brazil >> https://gilbertocamara.org/ >> = >> >> >> >> >> >>> On 22 Nov 2024, at 14:18, Gilberto Camara wrote: >>> >>> Dear Peter >>> >>> Many thanks for your help. I ran R in a terminal, something I admit I had >>> not done before. >>> >>> It
Re: [R-SIG-Mac] Problems with R-4.4.2 in MacOS Sequoia with Intel chip
Gilberto, you still didn't provide any useful examples, and all you have are baseless speculations. R itself doesn't load OpenMP, so the problem could very well be in the other software or libraries you use - it may have nothing to do with R, but without an example we can neither tell nor help. Based on this there is nothing to solve "at the highest level of R core developers for Mac" - it has to start with you providing a working example as no one else has that problem. Cheers, Simon > On Nov 23, 2024, at 7:30 AM, Gilberto Camara wrote: > > Dear Peter and Simon > > I may have found the cause of the crashes in R-4.4.2 in the Intel MacOs > environment. In R-4.4.2 for MacOS, the “libomp.dylib” is loaded from the CRAN > version. Comparing with earlier version, this is a new addition to the core R > libraries. The OpenMP library is not provided in R-4.2. > > It so happens that the “torch” package is also including the an OpenMP > library (“libiomp5.dylib”) which is causing the crash. In previous R > versions, this behavoir did not cause problems, since R did not provide the > OpenMP library by default. > > Thus, the problem needs to be solved at the highest level of R core > developers for Mac. Please bear in mind that the “torch” package is essential > for running deep learning algorithms in R. I hope you can find a solution > that preserves the use of “torch” in R. I am copying the message to Daniel > Falbel, who is the maintainer of the “torch” package. > > Your support in solving this problem is highly appreciated. > > Best regards > Gilberto > > Prof Dr Gilberto Camara > Senior Researcher > Getulio Vargas Foundation (FGV) > National Institute for Space Research (INPE), Brazil > https://gilbertocamara.org/ > = > > > > > >> On 22 Nov 2024, at 14:18, Gilberto Camara wrote: >> >> Dear Peter >> >> Many thanks for your help. I ran R in a terminal, something I admit I had >> not done before. >> >> It crashed and produced the following message: >> >> === >> OMP: Error #15: Initializing libiomp5.dylib, but found libomp.dylib already >> initialized. >> OMP: Hint This means that multiple copies of the OpenMP runtime have been >> linked into the program. That is dangerous, since it can degrade performance >> or cause incorrect results. The best thing to do is to ensure that only a >> single OpenMP runtime is linked into the process, e.g. by avoiding static >> linking of the OpenMP runtime in any library. As an unsafe, unsupported, >> undocumented workaround you can set the environment variable >> KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but >> that may cause crashes or silently produce incorrect results. For more >> information, please see http://www.intel.com/software/products/support/. >> === >> >> I am using Rcpp Armadillo, and I am following its guidelines for C++ >> compilation. The “Makevars” file is produced by running >> >>> usethis::use_rcpp_armadillo() >> >> which produces the following Makevars file: >> >> PKG_CXXFLAGS = $(SHLIB_OPENMP_CXXFLAGS) >> PKG_LIBS = $(SHLIB_OPENMP_CXXFLAGS) $(LAPACK_LIBS) $(BLAS_LIBS) $(FLIBS) >> >> I removed the Makevars file, compiled my package from scratch but the error >> continues. Any help on how to solve this problem would be most appreciated. >> >> Best regards >> Gilberto >> >> >> Prof Dr Gilberto Camara >> Senior Researcher >> Getulio Vargas Foundation (FGV) >> National Institute for Space Research (INPE), Brazil >> https://gilbertocamara.org/ >> = >> >> >> >> >> >>> On 22 Nov 2024, at 12:08, peter dalgaard wrote: >>> >>> A couple of questions: >>> >>> - Is it the GUI or R itself that crashes? (i.e, can you run R in Terminal >>> an still crash it? or even RStudio?) >>> - Is it a Sequoia issue as such? I'm not seeing issues on Monterey/4.4.1 >>> (I'm a little superstitious about upgrading production machines >>> mid-semester and some of my machines are too old for Sequoia.) The source >>> tarballs 4.4.2 were built and tested on Intel/Monterey. >>> >>> - pd >>> On 21 Nov 2024, at 16:07 , Gilberto Camara wrote: Dear Simon Since your message last week, I have been trying to reproduce the errors I am finding with R-4.4.2 in an Intel MacMini to build a minimum testable example. I am not succeeding in doing so. The error is a total collapse of R and occurs in a random fashion. Sometimes calling another package (e.g. “xgboost”) produces the error. Sometimes making a simple operation in a data table leads to error. The problem affects only Intel-based MacMinis with R-4.4.2. Running MacMinis with R-4.2.3 work will. All is also well with R-4.4.2 in Macs with ARM, in Windows and in Lunix/Ubuntu and Linux/Fedora. One issue I noticed is that the default C++ compiler in
Re: [R-SIG-Mac] Problems with R-4.4.2 in MacOS Sequoia with Intel chip
Dear Peter and Simon I may have found the cause of the crashes in R-4.4.2 in the Intel MacOs environment. In R-4.4.2 for MacOS, the “libomp.dylib” is loaded from the CRAN version. Comparing with earlier version, this is a new addition to the core R libraries. The OpenMP library is not provided in R-4.2. It so happens that the “torch” package is also including the an OpenMP library (“libiomp5.dylib”) which is causing the crash. In previous R versions, this behavoir did not cause problems, since R did not provide the OpenMP library by default. Thus, the problem needs to be solved at the highest level of R core developers for Mac. Please bear in mind that the “torch” package is essential for running deep learning algorithms in R. I hope you can find a solution that preserves the use of “torch” in R. I am copying the message to Daniel Falbel, who is the maintainer of the “torch” package. Your support in solving this problem is highly appreciated. Best regards Gilberto Prof Dr Gilberto Camara Senior Researcher Getulio Vargas Foundation (FGV) National Institute for Space Research (INPE), Brazil https://gilbertocamara.org/ = > On 22 Nov 2024, at 14:18, Gilberto Camara wrote: > > Dear Peter > > Many thanks for your help. I ran R in a terminal, something I admit I had not > done before. > > It crashed and produced the following message: > > === > OMP: Error #15: Initializing libiomp5.dylib, but found libomp.dylib already > initialized. > OMP: Hint This means that multiple copies of the OpenMP runtime have been > linked into the program. That is dangerous, since it can degrade performance > or cause incorrect results. The best thing to do is to ensure that only a > single OpenMP runtime is linked into the process, e.g. by avoiding static > linking of the OpenMP runtime in any library. As an unsafe, unsupported, > undocumented workaround you can set the environment variable > KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but > that may cause crashes or silently produce incorrect results. For more > information, please see http://www.intel.com/software/products/support/. > === > > I am using Rcpp Armadillo, and I am following its guidelines for C++ > compilation. The “Makevars” file is produced by running > >> usethis::use_rcpp_armadillo() > > which produces the following Makevars file: > > PKG_CXXFLAGS = $(SHLIB_OPENMP_CXXFLAGS) > PKG_LIBS = $(SHLIB_OPENMP_CXXFLAGS) $(LAPACK_LIBS) $(BLAS_LIBS) $(FLIBS) > > I removed the Makevars file, compiled my package from scratch but the error > continues. Any help on how to solve this problem would be most appreciated. > > Best regards > Gilberto > > > Prof Dr Gilberto Camara > Senior Researcher > Getulio Vargas Foundation (FGV) > National Institute for Space Research (INPE), Brazil > https://gilbertocamara.org/ > = > > > > > >> On 22 Nov 2024, at 12:08, peter dalgaard wrote: >> >> A couple of questions: >> >> - Is it the GUI or R itself that crashes? (i.e, can you run R in Terminal >> an still crash it? or even RStudio?) >> - Is it a Sequoia issue as such? I'm not seeing issues on Monterey/4.4.1 >> (I'm a little superstitious about upgrading production machines mid-semester >> and some of my machines are too old for Sequoia.) The source tarballs 4.4.2 >> were built and tested on Intel/Monterey. >> >> - pd >> >>> On 21 Nov 2024, at 16:07 , Gilberto Camara wrote: >>> >>> Dear Simon >>> >>> Since your message last week, I have been trying to reproduce the errors I >>> am finding with R-4.4.2 in an Intel MacMini to build a minimum testable >>> example. I am not succeeding in doing so. >>> >>> The error is a total collapse of R and occurs in a random fashion. >>> Sometimes calling another package (e.g. “xgboost”) produces the error. >>> Sometimes making a simple operation in a data table leads to error. >>> >>> The problem affects only Intel-based MacMinis with R-4.4.2. Running >>> MacMinis with R-4.2.3 work will. All is also well with R-4.4.2 in Macs with >>> ARM, in Windows and in Lunix/Ubuntu and Linux/Fedora. >>> >>> One issue I noticed is that the default C++ compiler in MacOS Sequoia is >>> compatible with c++-17, while the one used by CRAN is version c++-14.00. >>> >>> Do you have any ideas on how to proceed? Any test data set I could use? >>> >>> Many thanks >>> Gilberto >>> >>> Prof Dr Gilberto Camara >>> Senior Researcher >>> Getulio Vargas Foundation (FGV) >>> National Institute for Space Research (INPE), Brazil >>> https://gilbertocamara.org/ >>> = >>> >>> >>> >>> >>> On 12 Nov 2024, at 18:53, Simon Urbanek wrote: Gilberto, please read https://www.r-project.org/bugs.html first, in particular about how to report a problem. Just saying "cannot execute
Re: [R-SIG-Mac] Problems with R-4.4.2 in MacOS Sequoia with Intel chip
Dear Peter Many thanks for your help. I ran R in a terminal, something I admit I had not done before. It crashed and produced the following message: === OMP: Error #15: Initializing libiomp5.dylib, but found libomp.dylib already initialized. OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://www.intel.com/software/products/support/. === I am using Rcpp Armadillo, and I am following its guidelines for C++ compilation. The “Makevars” file is produced by running >usethis::use_rcpp_armadillo() which produces the following Makevars file: PKG_CXXFLAGS = $(SHLIB_OPENMP_CXXFLAGS) PKG_LIBS = $(SHLIB_OPENMP_CXXFLAGS) $(LAPACK_LIBS) $(BLAS_LIBS) $(FLIBS) I removed the Makevars file, compiled my package from scratch but the error continues. Any help on how to solve this problem would be most appreciated. Best regards Gilberto Prof Dr Gilberto Camara Senior Researcher Getulio Vargas Foundation (FGV) National Institute for Space Research (INPE), Brazil https://gilbertocamara.org/ = > On 22 Nov 2024, at 12:08, peter dalgaard wrote: > > A couple of questions: > > - Is it the GUI or R itself that crashes? (i.e, can you run R in Terminal an > still crash it? or even RStudio?) > - Is it a Sequoia issue as such? I'm not seeing issues on Monterey/4.4.1 (I'm > a little superstitious about upgrading production machines mid-semester and > some of my machines are too old for Sequoia.) The source tarballs 4.4.2 were > built and tested on Intel/Monterey. > > - pd > >> On 21 Nov 2024, at 16:07 , Gilberto Camara wrote: >> >> Dear Simon >> >> Since your message last week, I have been trying to reproduce the errors I >> am finding with R-4.4.2 in an Intel MacMini to build a minimum testable >> example. I am not succeeding in doing so. >> >> The error is a total collapse of R and occurs in a random fashion. Sometimes >> calling another package (e.g. “xgboost”) produces the error. Sometimes >> making a simple operation in a data table leads to error. >> >> The problem affects only Intel-based MacMinis with R-4.4.2. Running MacMinis >> with R-4.2.3 work will. All is also well with R-4.4.2 in Macs with ARM, in >> Windows and in Lunix/Ubuntu and Linux/Fedora. >> >> One issue I noticed is that the default C++ compiler in MacOS Sequoia is >> compatible with c++-17, while the one used by CRAN is version c++-14.00. >> >> Do you have any ideas on how to proceed? Any test data set I could use? >> >> Many thanks >> Gilberto >> >> Prof Dr Gilberto Camara >> Senior Researcher >> Getulio Vargas Foundation (FGV) >> National Institute for Space Research (INPE), Brazil >> https://gilbertocamara.org/ >> = >> >> >> >> >> >>> On 12 Nov 2024, at 18:53, Simon Urbanek wrote: >>> >>> Gilberto, >>> >>> please read https://www.r-project.org/bugs.html first, in particular about >>> how to report a problem. Just saying "cannot execute my scripts" is not >>> helpful at all - please provide exact output, how it differs between the >>> versions etc. For example, the problem may be in your code or packages used >>> - we have no way of knowing without the necessary additional information. >>> >>> Thanks, >>> Simon >>> >>> On Nov 13, 2024, at 9:44 AM, Gilberto Camara wrote: Dear R-SIG-MAC I have problems with R-4.4.2 in the latest Mac OS Sequoia (15.1) version in a Mac mini with an Intel chip. R-4.4.2 cannot execute my scripts. No such problems occur with R-4.4.2 in MacBook with ARM chip. I went back to R-4.2.3 and all is well in my environment (MacMini, Intel, MacOS X 15.1). I will try to provide an MWE to help those of you who know a lot about Mac OS X. Best Gilberto Prof Dr Gilberto Camara Senior Researcher Getulio Vargas Foundation (FGV) National Institute for Space Research (INPE), Brazil https://gilbertocamara.org/ = ___ R-SIG-Mac mailing list [email protected] https://stat.ethz.ch/mailman/listinfo/r-sig-mac >>> >>> ___ >>> R-SIG-Mac mailing list >>> [email protected] >>>
Re: [R-SIG-Mac] Problems with R-4.4.2 in MacOS Sequoia with Intel chip
A couple of questions: - Is it the GUI or R itself that crashes? (i.e, can you run R in Terminal an still crash it? or even RStudio?) - Is it a Sequoia issue as such? I'm not seeing issues on Monterey/4.4.1 (I'm a little superstitious about upgrading production machines mid-semester and some of my machines are too old for Sequoia.) The source tarballs 4.4.2 were built and tested on Intel/Monterey. - pd > On 21 Nov 2024, at 16:07 , Gilberto Camara wrote: > > Dear Simon > > Since your message last week, I have been trying to reproduce the errors I am > finding with R-4.4.2 in an Intel MacMini to build a minimum testable example. > I am not succeeding in doing so. > > The error is a total collapse of R and occurs in a random fashion. Sometimes > calling another package (e.g. “xgboost”) produces the error. Sometimes making > a simple operation in a data table leads to error. > > The problem affects only Intel-based MacMinis with R-4.4.2. Running MacMinis > with R-4.2.3 work will. All is also well with R-4.4.2 in Macs with ARM, in > Windows and in Lunix/Ubuntu and Linux/Fedora. > > One issue I noticed is that the default C++ compiler in MacOS Sequoia is > compatible with c++-17, while the one used by CRAN is version c++-14.00. > > Do you have any ideas on how to proceed? Any test data set I could use? > > Many thanks > Gilberto > > Prof Dr Gilberto Camara > Senior Researcher > Getulio Vargas Foundation (FGV) > National Institute for Space Research (INPE), Brazil > https://gilbertocamara.org/ > = > > > > > >> On 12 Nov 2024, at 18:53, Simon Urbanek wrote: >> >> Gilberto, >> >> please read https://www.r-project.org/bugs.html first, in particular about >> how to report a problem. Just saying "cannot execute my scripts" is not >> helpful at all - please provide exact output, how it differs between the >> versions etc. For example, the problem may be in your code or packages used >> - we have no way of knowing without the necessary additional information. >> >> Thanks, >> Simon >> >> >>> On Nov 13, 2024, at 9:44 AM, Gilberto Camara >>> wrote: >>> >>> Dear R-SIG-MAC >>> >>> I have problems with R-4.4.2 in the latest Mac OS Sequoia (15.1) version in >>> a Mac mini with an Intel chip. R-4.4.2 cannot execute my scripts. No such >>> problems occur with R-4.4.2 in MacBook with ARM chip. >>> >>> I went back to R-4.2.3 and all is well in my environment (MacMini, Intel, >>> MacOS X 15.1). >>> >>> >>> I will try to provide an MWE to help those of you who know a lot about Mac >>> OS X. >>> >>> Best >>> Gilberto >>> >>> >>> Prof Dr Gilberto Camara >>> Senior Researcher >>> Getulio Vargas Foundation (FGV) >>> National Institute for Space Research (INPE), Brazil >>> https://gilbertocamara.org/ >>> = >>> >>> ___ >>> R-SIG-Mac mailing list >>> [email protected] >>> https://stat.ethz.ch/mailman/listinfo/r-sig-mac >>> >> >> ___ >> R-SIG-Mac mailing list >> [email protected] >> https://stat.ethz.ch/mailman/listinfo/r-sig-mac > > > [[alternative HTML version deleted]] > > ___ > R-SIG-Mac mailing list > [email protected] > https://stat.ethz.ch/mailman/listinfo/r-sig-mac -- Peter Dalgaard, Professor, Center for Statistics, Copenhagen Business School Solbjerg Plads 3, 2000 Frederiksberg, Denmark Phone: (+45)38153501 Office: A 4.23 Email: [email protected] Priv: [email protected] ___ R-SIG-Mac mailing list [email protected] https://stat.ethz.ch/mailman/listinfo/r-sig-mac
Re: [R-SIG-Mac] Problems with R-4.4.2 in MacOS Sequoia with Intel chip
Dear Simon Since your message last week, I have been trying to reproduce the errors I am finding with R-4.4.2 in an Intel MacMini to build a minimum testable example. I am not succeeding in doing so. The error is a total collapse of R and occurs in a random fashion. Sometimes calling another package (e.g. “xgboost”) produces the error. Sometimes making a simple operation in a data table leads to error. The problem affects only Intel-based MacMinis with R-4.4.2. Running MacMinis with R-4.2.3 work will. All is also well with R-4.4.2 in Macs with ARM, in Windows and in Lunix/Ubuntu and Linux/Fedora. One issue I noticed is that the default C++ compiler in MacOS Sequoia is compatible with c++-17, while the one used by CRAN is version c++-14.00. Do you have any ideas on how to proceed? Any test data set I could use? Many thanks Gilberto Prof Dr Gilberto Camara Senior Researcher Getulio Vargas Foundation (FGV) National Institute for Space Research (INPE), Brazil https://gilbertocamara.org/ = > On 12 Nov 2024, at 18:53, Simon Urbanek wrote: > > Gilberto, > > please read https://www.r-project.org/bugs.html first, in particular about > how to report a problem. Just saying "cannot execute my scripts" is not > helpful at all - please provide exact output, how it differs between the > versions etc. For example, the problem may be in your code or packages used - > we have no way of knowing without the necessary additional information. > > Thanks, > Simon > > >> On Nov 13, 2024, at 9:44 AM, Gilberto Camara wrote: >> >> Dear R-SIG-MAC >> >> I have problems with R-4.4.2 in the latest Mac OS Sequoia (15.1) version in >> a Mac mini with an Intel chip. R-4.4.2 cannot execute my scripts. No such >> problems occur with R-4.4.2 in MacBook with ARM chip. >> >> I went back to R-4.2.3 and all is well in my environment (MacMini, Intel, >> MacOS X 15.1). >> >> >> I will try to provide an MWE to help those of you who know a lot about Mac >> OS X. >> >> Best >> Gilberto >> >> >> Prof Dr Gilberto Camara >> Senior Researcher >> Getulio Vargas Foundation (FGV) >> National Institute for Space Research (INPE), Brazil >> https://gilbertocamara.org/ >> = >> >> ___ >> R-SIG-Mac mailing list >> [email protected] >> https://stat.ethz.ch/mailman/listinfo/r-sig-mac >> > > ___ > R-SIG-Mac mailing list > [email protected] > https://stat.ethz.ch/mailman/listinfo/r-sig-mac [[alternative HTML version deleted]] ___ R-SIG-Mac mailing list [email protected] https://stat.ethz.ch/mailman/listinfo/r-sig-mac
Re: [R-SIG-Mac] Problems with R-4.4.2 in MacOS Sequoia with Intel chip
Dear Simon Thanks! Point taken! Best Gilberto Prof Dr Gilberto Camara Senior Researcher Getulio Vargas Foundation (FGV) National Institute for Space Research (INPE), Brazil https://gilbertocamara.org/ = > On 12 Nov 2024, at 18:53, Simon Urbanek wrote: > > Gilberto, > > please read https://www.r-project.org/bugs.html first, in particular about > how to report a problem. Just saying "cannot execute my scripts" is not > helpful at all - please provide exact output, how it differs between the > versions etc. For example, the problem may be in your code or packages used - > we have no way of knowing without the necessary additional information. > > Thanks, > Simon > > >> On Nov 13, 2024, at 9:44 AM, Gilberto Camara wrote: >> >> Dear R-SIG-MAC >> >> I have problems with R-4.4.2 in the latest Mac OS Sequoia (15.1) version in >> a Mac mini with an Intel chip. R-4.4.2 cannot execute my scripts. No such >> problems occur with R-4.4.2 in MacBook with ARM chip. >> >> I went back to R-4.2.3 and all is well in my environment (MacMini, Intel, >> MacOS X 15.1). >> >> >> I will try to provide an MWE to help those of you who know a lot about Mac >> OS X. >> >> Best >> Gilberto >> >> >> Prof Dr Gilberto Camara >> Senior Researcher >> Getulio Vargas Foundation (FGV) >> National Institute for Space Research (INPE), Brazil >> https://gilbertocamara.org/ >> = >> >> ___ >> R-SIG-Mac mailing list >> [email protected] >> https://stat.ethz.ch/mailman/listinfo/r-sig-mac >> > > ___ > R-SIG-Mac mailing list > [email protected] > https://stat.ethz.ch/mailman/listinfo/r-sig-mac ___ R-SIG-Mac mailing list [email protected] https://stat.ethz.ch/mailman/listinfo/r-sig-mac
Re: [R-SIG-Mac] Problems with R-4.4.2 in MacOS Sequoia with Intel chip
Gilberto, please read https://www.r-project.org/bugs.html first, in particular about how to report a problem. Just saying "cannot execute my scripts" is not helpful at all - please provide exact output, how it differs between the versions etc. For example, the problem may be in your code or packages used - we have no way of knowing without the necessary additional information. Thanks, Simon > On Nov 13, 2024, at 9:44 AM, Gilberto Camara wrote: > > Dear R-SIG-MAC > > I have problems with R-4.4.2 in the latest Mac OS Sequoia (15.1) version in a > Mac mini with an Intel chip. R-4.4.2 cannot execute my scripts. No such > problems occur with R-4.4.2 in MacBook with ARM chip. > > I went back to R-4.2.3 and all is well in my environment (MacMini, Intel, > MacOS X 15.1). > > > I will try to provide an MWE to help those of you who know a lot about Mac OS > X. > > Best > Gilberto > > > Prof Dr Gilberto Camara > Senior Researcher > Getulio Vargas Foundation (FGV) > National Institute for Space Research (INPE), Brazil > https://gilbertocamara.org/ > = > > ___ > R-SIG-Mac mailing list > [email protected] > https://stat.ethz.ch/mailman/listinfo/r-sig-mac > ___ R-SIG-Mac mailing list [email protected] https://stat.ethz.ch/mailman/listinfo/r-sig-mac
