Re: [R-SIG-Mac] Apple Silicon aka M1 Macs
Thank you for the feedback, this is great! I kept investigating my particular issue and it turned out that I did not pay sufficient attention. I actually forgot to comment out the relevant include and lib paths that I set in the config.site file … my bad. As I proceeded, I encountered a few more issues. It turned out ./configure needed a newer version of curl. Installing it with brew did the job + linking the lib include and pkconfig to /opt/homebrew respectively. Currently my build efforts are stuck as libR.dylib references _libiconv but cannot find that symbol inside libiconv, which seems to be an old issue according to a quick google search, e.g (https://mw.gl/posts/libiconv_mac/). Installing and linking libiconv via brew did not work so far, I do not manage to tell the compiler to look up the right libiconv. On the other hand, the binary provided via brew seems to work fine. Installing R packages from the sources worked flawlessly and the packages seem usable. So far I've checked out tidymodels, readr, ggplot2, ranger, grf, glmnet (plust the dependencies). This looks very encouraging! Best wishes, Denis On Thu, Dec 24, 2020 at 10:50 PM Jeroen Ooms wrote: > > On Thu, Dec 24, 2020 at 7:32 PM Denis-Alexander Engemann > wrote: > > > > Hi everyone, > > > > I'm finally joining the party here and so far made good progress > > following Taras' instructions. > > > > Disclaimer - I'm new to working with R sources. > > But would be very happy to help with testing and reporting to > > accelerate support native M1 builds. > > > > I am using the R-4.0.3 sources with the latest gfortran release from > > François-Xavier Coudert: > > https://github.com/fxcoudert/gfortran-for-macOS/releases/tag/11-arm-alpha2 > > > > Currently I'm facing some issues at step 5. > > It seems that for some reasons liblzma cannot be found although it is > > well installed via xz at step 3. > > I also verified that the liblzma can be easily discovered along the lines > > of 3. > > > > The lines before ./configure stops are: > > > > ``` > > checking lzma.h usability... no > > checking lzma.h presence... no > > checking for lzma.h... no > > configure: error: "liblzma library and headers are required" > > ``` > > > > As mentioned, all is well installed in the include and lib directories at: > > /opt/homebrew/Cellar/xz/5.2.5 > > > > Would you have a suggestion where I should link the headers/library to > > be picked up by ./configure ? > > Homebrew now has an arm64 binary for R: > https://github.com/homebrew/homebrew-core/blob/master/Formula/r.rb . > Hence you can simply use: brew install r > > To tweak the build, use "brew edit r", save your changes, and then: > brew install r --build-from-source ___ R-SIG-Mac mailing list R-SIG-Mac@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-mac
Re: [R-SIG-Mac] Apple Silicon aka M1 Macs
On Thu, Dec 24, 2020 at 7:32 PM Denis-Alexander Engemann wrote: > > Hi everyone, > > I'm finally joining the party here and so far made good progress > following Taras' instructions. > > Disclaimer - I'm new to working with R sources. > But would be very happy to help with testing and reporting to > accelerate support native M1 builds. > > I am using the R-4.0.3 sources with the latest gfortran release from > François-Xavier Coudert: > https://github.com/fxcoudert/gfortran-for-macOS/releases/tag/11-arm-alpha2 > > Currently I'm facing some issues at step 5. > It seems that for some reasons liblzma cannot be found although it is > well installed via xz at step 3. > I also verified that the liblzma can be easily discovered along the lines of > 3. > > The lines before ./configure stops are: > > ``` > checking lzma.h usability... no > checking lzma.h presence... no > checking for lzma.h... no > configure: error: "liblzma library and headers are required" > ``` > > As mentioned, all is well installed in the include and lib directories at: > /opt/homebrew/Cellar/xz/5.2.5 > > Would you have a suggestion where I should link the headers/library to > be picked up by ./configure ? Homebrew now has an arm64 binary for R: https://github.com/homebrew/homebrew-core/blob/master/Formula/r.rb . Hence you can simply use: brew install r To tweak the build, use "brew edit r", save your changes, and then: brew install r --build-from-source ___ R-SIG-Mac mailing list R-SIG-Mac@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-mac
Re: [R-SIG-Mac] Apple Silicon aka M1 Macs
Hi everyone, I'm finally joining the party here and so far made good progress following Taras' instructions. Disclaimer - I'm new to working with R sources. But would be very happy to help with testing and reporting to accelerate support native M1 builds. I am using the R-4.0.3 sources with the latest gfortran release from François-Xavier Coudert: https://github.com/fxcoudert/gfortran-for-macOS/releases/tag/11-arm-alpha2 Currently I'm facing some issues at step 5. It seems that for some reasons liblzma cannot be found although it is well installed via xz at step 3. I also verified that the liblzma can be easily discovered along the lines of 3. The lines before ./configure stops are: ``` checking lzma.h usability... no checking lzma.h presence... no checking for lzma.h... no configure: error: "liblzma library and headers are required" ``` As mentioned, all is well installed in the include and lib directories at: /opt/homebrew/Cellar/xz/5.2.5 Would you have a suggestion where I should link the headers/library to be picked up by ./configure ? Best wishes, Denis On Wed, Nov 25, 2020 at 10:41 AM Taras Zakharko wrote: > > Creating a native Aarch64 build of R on my M1 machine was surprisingly > straightforward. Here is a step-by-step instruction of what I did in case > someone wants to replicate it: > > 1. Install native homebrew as described in this blog post under “Multiple > Homebrews” (https://soffes.blog/homebrew-on-apple-silicon). You will need to > manually chown a bunch of directories, keep an eye on brew output > > 2. Install the following brew packages > > brew install jpeg libpng libtiff pcre2 pkg-config tcl-tk xz zlib > > Again, pay attention to brew output, it is possible that you will have to > manually reset ownership on some directories > > 3. Make sure that zlib can be discovered by pkg-config by making it’s pc file > discoverable > > ln -s /opt/homebrew/Cellar/zlib/1.2.11/lib/pkgconfig/zlib.pc > /opt/homebrew/lib/pkgconfig/ > > 4. Install patched gfortran, I used the precompiled package from here: > > > https://github.com/fxcoudert/gfortran-for-macOS/releases/tag/11-arm-alpha1 > > 5. In config.site, add the paths to homebrew libraries: > > CPPFLAGS=-I/opt/homebrew/include > LDFLAGS=-L/opt/homebrew/lib > > Note: the R-admin manual tells to set CC, CXX, FC in config.sites, but it’s > not strictly necessary, the build will work without them > > 5. Run the configure script, I used > > ./configure --enable-R-shlib --without-x --enable-memory-profiling > --with-blas="-framework Accelerate” > > Check that the summary makes sense. If PNG is disabled, it’s probably > because you forgot step 3 > > 6. Build R > > make -j4 > > 7. Check that everything runs > > bin/R > >You should see something like "Platform: arm-apple-darwin20.2.0 (64-bit)” > in the startup message. > > All core tests seems to pass and the performance is excellent. > > I have tried building tidyverse and it blocked on testthat. It seems that > testthat uses a fairly old version of Catch which doesn’t detect platform > correctly and tries to use some x86 inline assembly during the arm build… > > > > On 23 Nov 2020, at 11:14, Prof Brian Ripley wrote: > > > > As a follow-up, I now have a preliminary native build of R (using a > > gfortran compiled from sources forked from GCC and using minor > > modifications of Tomas Kalibera's instructions). > > > > The check timing was 148s, with Aqua (not X11) Tcl/Tk built from the > > sources and no X11 support, so not quite 100% comparable. > > > > Building the compiler took 45m elapsed with 100% CPU most of the time: the > > machine (which has no fan) remained cool (unlike my MBP which has a fan but > > rarely runs it and does get warm to the touch). > > > > There is a preliminary write-up on 'arm64' Macs in R-devel's R-admin manual > > (the version on CRAN is as usual a few days behind). > > > > On 17/11/2020 14:57, Prof Brian Ripley wrote: > >> Mine (a 8GB MBA) arrived today, so I have started doing some comparisons. > >> For the CRAN build of R 4.0.3, §2.8 of R-admin recommends checking the > >> installation with > >> pdf("tests.pdf") ## optional, but prevents flashing graphics windows > >> Sys.setenv(LC_COLLATE = "C", LC_TIME = "C", LANGUAGE = "en") > >> tools::testInstalledBasic("both") > >> tools::testInstalledPackages(scope = "base") > >> tools::testInstalledPackages(scope = "recommended") > >> That took 454s (using Rosetta) against 895s for my late-2016 MBP (2.9GHz > >> i5): happily nothing untoward was reported (some recommended packages give > >> differences from reference output on both systems). > >> You need to install XQuartz to provide the X11() devices and support for > >> package Tcl/Tk: everything I tried using that worked as expected. > >> Having done that post-installation check I would happily use the Intel R > >> on an M1 machine. > >> We plan to check many of the Intel-compiled packages under R
Re: [R-SIG-Mac] Apple Silicon aka M1 Macs
Creating a native Aarch64 build of R on my M1 machine was surprisingly straightforward. Here is a step-by-step instruction of what I did in case someone wants to replicate it: 1. Install native homebrew as described in this blog post under “Multiple Homebrews” (https://soffes.blog/homebrew-on-apple-silicon). You will need to manually chown a bunch of directories, keep an eye on brew output 2. Install the following brew packages brew install jpeg libpng libtiff pcre2 pkg-config tcl-tk xz zlib Again, pay attention to brew output, it is possible that you will have to manually reset ownership on some directories 3. Make sure that zlib can be discovered by pkg-config by making it’s pc file discoverable ln -s /opt/homebrew/Cellar/zlib/1.2.11/lib/pkgconfig/zlib.pc /opt/homebrew/lib/pkgconfig/ 4. Install patched gfortran, I used the precompiled package from here: https://github.com/fxcoudert/gfortran-for-macOS/releases/tag/11-arm-alpha1 5. In config.site, add the paths to homebrew libraries: CPPFLAGS=-I/opt/homebrew/include LDFLAGS=-L/opt/homebrew/lib Note: the R-admin manual tells to set CC, CXX, FC in config.sites, but it’s not strictly necessary, the build will work without them 5. Run the configure script, I used ./configure --enable-R-shlib --without-x --enable-memory-profiling --with-blas="-framework Accelerate” Check that the summary makes sense. If PNG is disabled, it’s probably because you forgot step 3 6. Build R make -j4 7. Check that everything runs bin/R You should see something like "Platform: arm-apple-darwin20.2.0 (64-bit)” in the startup message. All core tests seems to pass and the performance is excellent. I have tried building tidyverse and it blocked on testthat. It seems that testthat uses a fairly old version of Catch which doesn’t detect platform correctly and tries to use some x86 inline assembly during the arm build… > On 23 Nov 2020, at 11:14, Prof Brian Ripley wrote: > > As a follow-up, I now have a preliminary native build of R (using a gfortran > compiled from sources forked from GCC and using minor modifications of Tomas > Kalibera's instructions). > > The check timing was 148s, with Aqua (not X11) Tcl/Tk built from the sources > and no X11 support, so not quite 100% comparable. > > Building the compiler took 45m elapsed with 100% CPU most of the time: the > machine (which has no fan) remained cool (unlike my MBP which has a fan but > rarely runs it and does get warm to the touch). > > There is a preliminary write-up on 'arm64' Macs in R-devel's R-admin manual > (the version on CRAN is as usual a few days behind). > > On 17/11/2020 14:57, Prof Brian Ripley wrote: >> Mine (a 8GB MBA) arrived today, so I have started doing some comparisons. >> For the CRAN build of R 4.0.3, §2.8 of R-admin recommends checking the >> installation with >> pdf("tests.pdf") ## optional, but prevents flashing graphics windows >> Sys.setenv(LC_COLLATE = "C", LC_TIME = "C", LANGUAGE = "en") >> tools::testInstalledBasic("both") >> tools::testInstalledPackages(scope = "base") >> tools::testInstalledPackages(scope = "recommended") >> That took 454s (using Rosetta) against 895s for my late-2016 MBP (2.9GHz >> i5): happily nothing untoward was reported (some recommended packages give >> differences from reference output on both systems). >> You need to install XQuartz to provide the X11() devices and support for >> package Tcl/Tk: everything I tried using that worked as expected. >> Having done that post-installation check I would happily use the Intel R on >> an M1 machine. >> We plan to check many of the Intel-compiled packages under Rosetta. >> There are many hours of work ahead to build/test a native toolchain: our >> goal is to have a native distribution for R 4.1.0 ca April 2021. > > > -- > Brian D. Ripley, rip...@stats.ox.ac.uk > Emeritus Professor of Applied Statistics, University of Oxford > > ___ > R-SIG-Mac mailing list > R-SIG-Mac@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-mac ___ R-SIG-Mac mailing list R-SIG-Mac@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-mac
Re: [R-SIG-Mac] Apple Silicon aka M1 Macs
As a follow-up, I now have a preliminary native build of R (using a gfortran compiled from sources forked from GCC and using minor modifications of Tomas Kalibera's instructions). The check timing was 148s, with Aqua (not X11) Tcl/Tk built from the sources and no X11 support, so not quite 100% comparable. Building the compiler took 45m elapsed with 100% CPU most of the time: the machine (which has no fan) remained cool (unlike my MBP which has a fan but rarely runs it and does get warm to the touch). There is a preliminary write-up on 'arm64' Macs in R-devel's R-admin manual (the version on CRAN is as usual a few days behind). On 17/11/2020 14:57, Prof Brian Ripley wrote: Mine (a 8GB MBA) arrived today, so I have started doing some comparisons. For the CRAN build of R 4.0.3, §2.8 of R-admin recommends checking the installation with pdf("tests.pdf") ## optional, but prevents flashing graphics windows Sys.setenv(LC_COLLATE = "C", LC_TIME = "C", LANGUAGE = "en") tools::testInstalledBasic("both") tools::testInstalledPackages(scope = "base") tools::testInstalledPackages(scope = "recommended") That took 454s (using Rosetta) against 895s for my late-2016 MBP (2.9GHz i5): happily nothing untoward was reported (some recommended packages give differences from reference output on both systems). You need to install XQuartz to provide the X11() devices and support for package Tcl/Tk: everything I tried using that worked as expected. Having done that post-installation check I would happily use the Intel R on an M1 machine. We plan to check many of the Intel-compiled packages under Rosetta. There are many hours of work ahead to build/test a native toolchain: our goal is to have a native distribution for R 4.1.0 ca April 2021. -- Brian D. Ripley, rip...@stats.ox.ac.uk Emeritus Professor of Applied Statistics, University of Oxford ___ R-SIG-Mac mailing list R-SIG-Mac@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-mac
Re: [R-SIG-Mac] Apple Silicon aka M1 Macs
Thank you for this update! I think that Apple Silicon has great potential for data science, the SPEC benchmarks reported by Anandtech look very encouraging. My M1 Mac should be arriving shortly, is there some way one could help out with the effort? — Taras > On 17 Nov 2020, at 15:57, Prof Brian Ripley wrote: > > Mine (a 8GB MBA) arrived today, so I have started doing some comparisons. > > For the CRAN build of R 4.0.3, §2.8 of R-admin recommends checking the > installation with > > pdf("tests.pdf") ## optional, but prevents flashing graphics windows > Sys.setenv(LC_COLLATE = "C", LC_TIME = "C", LANGUAGE = "en") > tools::testInstalledBasic("both") > tools::testInstalledPackages(scope = "base") > tools::testInstalledPackages(scope = "recommended") > > That took 454s (using Rosetta) against 895s for my late-2016 MBP (2.9GHz i5): > happily nothing untoward was reported (some recommended packages give > differences from reference output on both systems). > > You need to install XQuartz to provide the X11() devices and support for > package Tcl/Tk: everything I tried using that worked as expected. > > Having done that post-installation check I would happily use the Intel R on > an M1 machine. > > We plan to check many of the Intel-compiled packages under Rosetta. > > There are many hours of work ahead to build/test a native toolchain: our goal > is to have a native distribution for R 4.1.0 ca April 2021. > > -- > Brian D. Ripley, rip...@stats.ox.ac.uk > Emeritus Professor of Applied Statistics, University of Oxford > > ___ > R-SIG-Mac mailing list > R-SIG-Mac@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-mac ___ R-SIG-Mac mailing list R-SIG-Mac@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-mac
Re: [R-SIG-Mac] Apple Silicon aka M1 Macs
Prof. Ripley, Thanks for these updates and for the efforts by all involved in this process. I am holding off on moving to the M1 based Macs until next year, as besides waiting for a larger display variant, as I am also waiting on updates to other applications that will support that hardware in time. You reference XQuartz below, and unless I am mis-reading the tea leaves, there seems to be an indication in their list archives that there is no plan to update their binary release to support the new hardware. That release has not been updated since 2016. They seem to be suggesting a dependence upon MacPorts moving forward, possibly HomeBrew, for installing a native XQuartz on the new hardware. Either scenario introduces other issues idiosyncratic to those environments. Not sure if that influences any decisions here, and presume that you may already be aware of those dynamics. Regards, Marc Schwartz > On Nov 17, 2020, at 9:57 AM, Prof Brian Ripley wrote: > > Mine (a 8GB MBA) arrived today, so I have started doing some comparisons. > > For the CRAN build of R 4.0.3, §2.8 of R-admin recommends checking the > installation with > > pdf("tests.pdf") ## optional, but prevents flashing graphics windows > Sys.setenv(LC_COLLATE = "C", LC_TIME = "C", LANGUAGE = "en") > tools::testInstalledBasic("both") > tools::testInstalledPackages(scope = "base") > tools::testInstalledPackages(scope = "recommended") > > That took 454s (using Rosetta) against 895s for my late-2016 MBP (2.9GHz i5): > happily nothing untoward was reported (some recommended packages give > differences from reference output on both systems). > > You need to install XQuartz to provide the X11() devices and support for > package Tcl/Tk: everything I tried using that worked as expected. > > Having done that post-installation check I would happily use the Intel R on > an M1 machine. > > We plan to check many of the Intel-compiled packages under Rosetta. > > There are many hours of work ahead to build/test a native toolchain: our goal > is to have a native distribution for R 4.1.0 ca April 2021. > > -- > Brian D. Ripley, rip...@stats.ox.ac.uk > Emeritus Professor of Applied Statistics, University of Oxford ___ R-SIG-Mac mailing list R-SIG-Mac@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-mac
Re: [R-SIG-Mac] Apple Silicon aka M1 Macs
One thing that I noticed both on Silicon and the intel macs at GitHub Actions, is that the CRAN installer does not create the symlinks in `/usr/local/bin`. Probably some permission or entitlement issue. So you'll probably want to create them manually. E.g. on GHA it goes like this: https://github.com/r-lib/ps/blob/a24f2c4d1bdba63be14e7729b9ab81d0ed9f719e/.github/workflows/R-CMD-check.yaml#L42-L45 G. On Tue, Nov 17, 2020 at 3:15 PM Roy Mendelssohn - NOAA Federal via R-SIG-Mac wrote: > > Thank to all who are working on this. Looks like a lot of work. Also looks > like when and if R can run in native mode on the M1 it will be very fast. > > -Roy > > > > On Nov 17, 2020, at 6:57 AM, Prof Brian Ripley > > wrote: > > > > Mine (a 8GB MBA) arrived today, so I have started doing some comparisons. > > > > For the CRAN build of R 4.0.3, §2.8 of R-admin recommends checking the > > installation with > > > > pdf("tests.pdf") ## optional, but prevents flashing graphics windows > > Sys.setenv(LC_COLLATE = "C", LC_TIME = "C", LANGUAGE = "en") > > tools::testInstalledBasic("both") > > tools::testInstalledPackages(scope = "base") > > tools::testInstalledPackages(scope = "recommended") > > > > That took 454s (using Rosetta) against 895s for my late-2016 MBP (2.9GHz > > i5): happily nothing untoward was reported (some recommended packages give > > differences from reference output on both systems). > > > > You need to install XQuartz to provide the X11() devices and support for > > package Tcl/Tk: everything I tried using that worked as expected. > > > > Having done that post-installation check I would happily use the Intel R on > > an M1 machine. > > > > We plan to check many of the Intel-compiled packages under Rosetta. > > > > There are many hours of work ahead to build/test a native toolchain: our > > goal is to have a native distribution for R 4.1.0 ca April 2021. > > > > -- > > Brian D. Ripley, rip...@stats.ox.ac.uk > > Emeritus Professor of Applied Statistics, University of Oxford > > > > ___ > > R-SIG-Mac mailing list > > R-SIG-Mac@r-project.org > > https://stat.ethz.ch/mailman/listinfo/r-sig-mac > > ** > "The contents of this message do not reflect any position of the U.S. > Government or NOAA." > ** > Roy Mendelssohn > Supervisory Operations Research Analyst > NOAA/NMFS > Environmental Research Division > Southwest Fisheries Science Center > ***Note new street address*** > 110 McAllister Way > Santa Cruz, CA 95060 > Phone: (831)-420-3666 > Fax: (831) 420-3980 > e-mail: roy.mendelss...@noaa.gov www: https://www.pfeg.noaa.gov/ > > "Old age and treachery will overcome youth and skill." > "From those who have been given much, much will be expected" > "the arc of the moral universe is long, but it bends toward justice" -MLK Jr. > > ___ > R-SIG-Mac mailing list > R-SIG-Mac@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-mac ___ R-SIG-Mac mailing list R-SIG-Mac@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-mac
Re: [R-SIG-Mac] Apple Silicon aka M1 Macs
Thank to all who are working on this. Looks like a lot of work. Also looks like when and if R can run in native mode on the M1 it will be very fast. -Roy > On Nov 17, 2020, at 6:57 AM, Prof Brian Ripley wrote: > > Mine (a 8GB MBA) arrived today, so I have started doing some comparisons. > > For the CRAN build of R 4.0.3, §2.8 of R-admin recommends checking the > installation with > > pdf("tests.pdf") ## optional, but prevents flashing graphics windows > Sys.setenv(LC_COLLATE = "C", LC_TIME = "C", LANGUAGE = "en") > tools::testInstalledBasic("both") > tools::testInstalledPackages(scope = "base") > tools::testInstalledPackages(scope = "recommended") > > That took 454s (using Rosetta) against 895s for my late-2016 MBP (2.9GHz i5): > happily nothing untoward was reported (some recommended packages give > differences from reference output on both systems). > > You need to install XQuartz to provide the X11() devices and support for > package Tcl/Tk: everything I tried using that worked as expected. > > Having done that post-installation check I would happily use the Intel R on > an M1 machine. > > We plan to check many of the Intel-compiled packages under Rosetta. > > There are many hours of work ahead to build/test a native toolchain: our goal > is to have a native distribution for R 4.1.0 ca April 2021. > > -- > Brian D. Ripley, rip...@stats.ox.ac.uk > Emeritus Professor of Applied Statistics, University of Oxford > > ___ > R-SIG-Mac mailing list > R-SIG-Mac@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-mac ** "The contents of this message do not reflect any position of the U.S. Government or NOAA." ** Roy Mendelssohn Supervisory Operations Research Analyst NOAA/NMFS Environmental Research Division Southwest Fisheries Science Center ***Note new street address*** 110 McAllister Way Santa Cruz, CA 95060 Phone: (831)-420-3666 Fax: (831) 420-3980 e-mail: roy.mendelss...@noaa.gov www: https://www.pfeg.noaa.gov/ "Old age and treachery will overcome youth and skill." "From those who have been given much, much will be expected" "the arc of the moral universe is long, but it bends toward justice" -MLK Jr. ___ R-SIG-Mac mailing list R-SIG-Mac@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-mac