Thanks Simon.
Branching off here to discuss the toolchain in general.
I see the danger posting not 100% correct scripts and in blog posts in general
because they might get outdated and not always reflect the official
documentation.
On the other side: What is the middle way? Arguably installing a robust
toolchain (on Catalina especially but for macOS in general) is not just done
with the installation of the signed package.
Most normal users are not aware about all the little specialties on macOS
(missing openMP enabled compiler, SDK issues) and why adjustments might be
needed.
Even though the idea is to require as less manual configuration as possible,
this is not always practical in the end (unless all maintainers agree on
configuring their package without the need of an openMP enabled C compiler for
example).
And I also think that “installing from source” is not a dev-only thing.
1. By using the Apple internal clang as the default, will maintainers be
indirectly forced to remove the dependency on an openMP enabled compiler with
respect to getting source installations on macOS working?
2. Can there be an official section easily accessible for users which
recommends which compiler to use (and how to set the Makevars) if users really
need/want to do so?
a. Regarding openMP enabled compilers: llvm is probably the one with the
least differences to Apples clang. It can be installed via brew (brew install
llvm) and then linked in Makevars. Would that be an alternative?
3. SDK: I did not wanted to induce that setting the SDK vars is needed for the
CRAN build chain (maybe we got each other wrong here). However, as of now,
there are real issues with SDK 10.15 that result in errors that are really hard
to track down or even fall under the radar when testing (if someone does not
run Catalina). Therefore I am linking locally against SDK 10.13 since some days
and that resolved many issues (see again the Rcpp issue and the igraph issue).
Since Catalina is the latest release and many people run it there should be
test/instructions somewhere on mac.rproject that deal with this/suggest not to
use SDK 10.15 for R IMO. Maybe this issue deserves a special thread.
Cheers, Patrick
On 1. Apr 2020, 22:29 +0200, Simon Urbanek <simon.urba...@r-project.org>, wrote:
> Patrick,
>
> firstly, please don't post such things - they are often wrong (as are parts
> of what you included it this e-mail) and it's impossible for us to track all
> blogs that have incorrect advice. Unfortunately, a lot of the issues we see
> out there are due to people finding bad advice and using it. Ideally, it
> should be sufficient to point to the official R documentation since that is
> the canonical source. If you post a blog, it should point to the official
> documentation. If that documentation is missing or not covering a need,
> please post here so we can fix it. It is more important that the official
> source is complete and much better for the community than to have random
> tidbits that may or may not be correct floating around.
>
> As for the script, please do not use R tar balls - first, you're picking the
> wrong one (there is no R-devel for el-capitan. R 4.0.0 pre-releases are in
> R-4-0-branch and they are built for high sierra). Second, the tar ball
> doesn't have the entitlements enabled so it doesn't match the release. Please
> use the signed R package. This may change, but with the complications of the
> Apple notarization process that's what it is today.
>
> Then there is the question of the SDK. What are you trying to setup here?
> What you posted here has nothing to do with the CRAN setup. You don't need
> 10.13 SDK to compile igraph, it compiles just fine using stock command line
> tools on High Sierra which is what we use on CRAN. You can, of course, pick
> any SDK you like, but please don't tell people that's what they should do.
> One of the most annoying issues is that people are recommending messing with
> Makevars which is incredibly fragile and will cause things to break left and
> right with any update. It's ok to do that for a very special purpose, but no
> regular user should have Makevars in their home.
>
> So if you really want to serve the community, please suggest improvements to
> the official documentation and post them here. If you have special needs, ask
> about them here.
>
> Thanks,
> Simon
>
>
>
> > On 1/04/2020, at 9:47 PM, Patrick Schratz <patrick.schr...@gmail.com> wrote:
> >
> > Thanks Simon,
> >
> > This simplifies things a lot and clears up the process. While the
> > instructions on https://mac.r-project.org/ are more clear now I think there
> > is still simplification needed for the install process and custom
> > modifications that are needed.
> > This not only applies to the dev page but finally also to production end
> > users if the SDK 10.15 issues (see below) persists.
> >
> > I’ve scripted all installation steps (R-devel, gfortran, SDK 10.13) and
> > will probably write a blog post about it to make the R community more aware
> > about it. Let me know if anything is wrong about it - otherwise I’ll use
> > the code below in the blog post.
> >
> > # install R-devel
> > wget
> > https://mac.r-project.org/el-capitan/R-devel/R-devel-el-capitan-sa-x86_64.tar.gz
> > sudo tar fvxz R*.tar.gz -C /
> > rm R-devel-el-capitan-sa-x86_64.tar.gz
> >
> > # install gfortran
> > wget
> > https://github.com/fxcoudert/gfortran-for-macOS/releases/download/8.2/gfortran-8.2-Mojave.dmg
> > sudo hdiutil attach gfortran*.dmg
> > sudo installer -package /Volumes/gfortran*/gfortran*/gfortran*.pkg -target /
> > sudo hdiutil detach /Volumes/gfortran-8.2-Mojave
> >
> > # install SDK10.13
> > wget
> > https://github.com/phracker/MacOSX-SDKs/releases/download/10.15/MacOSX10.13.sdk.tar.xz
> > tar fvxz MacOSX10.13.sdk.tar.xz
> > sudo mv MacOSX10.13.sdk /Library/Developer/CommandLineTools/SDKs/
> > rm -rf MacOSX10.13*
> >
> > In addition people need to set custom CFLAGS in ~/.R/Makevars to actually
> > use it:
> >
> > CFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.13.sdk
> > CCFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.13.sdk
> > CXXFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.13.sdk
> > CPPFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.13.sdk
> >
> > In addition there really seem to be major problems regarding the 10.15 SDK.
> > For example, {igraph} which is a package on which lots of other packages
> > depend, cannot be installed from source with SDK 10.15 but works with SDK
> > 10.13.
> >
> > Patrick
> > On 1. Apr 2020, 09:38 +0200, Simon Urbanek <simon.urba...@r-project.org>,
> > wrote:
> > > Hervé,
> > >
> > >
> > > > On 1/04/2020, at 6:19 PM, Hervé Pagès <hpa...@fredhutch.org> wrote:
> > > >
> > > > Thanks Simon. A couple of days ago we've started to use the R 4.0.0
> > > > alpha build from https://mac.r-project.org/ for the Bioconductor build
> > > > system. Bioconductor packages depend on thousands of CRAN packages and
> > > > one thing that makes it hard for us and for our users to build/check
> > > > Bioconductor packages at the moment is the absence of Mac binary
> > > > packages for R 4.0.0 on CRAN. Do you have an estimate of when they will
> > > > become available?
> > > >
> > >
> > > Both the packages and R are now available. I always recommend using the
> > > Mac-master https://mac.R-project.org as "mirror" since that one is most
> > > up-to-date.
> > >
> > >
> > > > Another question is where they are going to be hosted on CRAN. We need
> > > > to follow CRAN's layout for Bioconductor package repositories.
> > > >
> > >
> > >
> > > We have removed the extra layer, the package type is back to
> > > "mac.binary", so bin/macosx/contrib is the location.
> > >
> > > Cheers,
> > > Simon
> > >
> > >
> > >
> > > > Thanks!
> > > > H.
> > > >
> > > >
> > > > On 3/31/20 21:27, Simon Urbanek wrote:
> > > > > Dear Mac users,
> > > > > R 4.0.0 will be using an entirely new toolchain, entirely new build
> > > > > system on entirely new macOS version and hardware. Therefore I would
> > > > > like to ask you kindly to test the binaries from
> > > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__mac.R-2Dproject.org&d=DwIFAg&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=-PNBcsZhHb1MLy6fZakmLP6WDLMwL-VigJVzeFtPxOI&s=FJYxweW9WiMXbNMPpMM0bHXy020Gx9P-dHMrWmt7YzE&e=
> > > > > before the release as much as you can. Raising any issues after the
> > > > > release is too late! So please, please, test the pre-releases. Report
> > > > > any issues either directly to me or this mailing list.
> > > > > The nightly builds are signed, but not necessarily notarized.
> > > > > However, the build fulfils Apple's conditions and is known to pass
> > > > > notarization (in fact the the package available for download today is
> > > > > actually notarized) so it should be a good test for the release which
> > > > > will be notarized and should work on Catalina.
> > > > > For those that want to replicate our setup - technical details: we
> > > > > are now building with macOS 10.13 (High Sierra) as target (i.e. the
> > > > > oldest supported system), regular Apple Xcode/command line tools and
> > > > > GNU Fortran 8.2. R builds are running on macOS 10.15 (Catalina) with
> > > > > Xcode 11.4 using macOS 10.13 target. Packages are built on macOS
> > > > > 10.13 VMs with just Apple command line tools (this should make it
> > > > > easy to replicate the setup using Travis, for example). All 3rd party
> > > > > libraries that CRAN uses are available in
> > > > > https://urldefense.proofpoint.com/v2/url?u=http-3A__mac.r-2Dproject.org_libs-2D4_&d=DwIFAg&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=-PNBcsZhHb1MLy6fZakmLP6WDLMwL-VigJVzeFtPxOI&s=3ZbcXBticg_8MfXltGKdQ9SFNqqeLqqwK_pUhEet7QY&e=
> > > > > The new R build system is in
> > > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__svn.r-2Dproject.org_R-2Ddev-2Dweb_trunk_QA_Simon_R4&d=DwIFAg&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=-PNBcsZhHb1MLy6fZakmLP6WDLMwL-VigJVzeFtPxOI&s=Mmt7_pVM5aUDR3hz93-e-PtadG7wSYpaZe8Ayi5Pv4I&e=
> > > > > Packages build system has not changed and is in
> > > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__svn.r-2Dproject.org_R-2Ddev-2Dweb_trunk_QA_Simon_packages&d=DwIFAg&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=-PNBcsZhHb1MLy6fZakmLP6WDLMwL-VigJVzeFtPxOI&s=eId6n08LVeIAMhMXe3jvLFKn9QGi4_by8owFreDDc84&e=
> > > > > We also plan to have a mac-builder available with similar function as
> > > > > the win-builder where pre-submission tests can be performed and
> > > > > potentially a Travis template.
> > > > > Please test R pre-releases and provide feedback!
> > > > > Thanks,
> > > > > Simon
> > > > > _______________________________________________
> > > > > R-SIG-Mac mailing list
> > > > > R-SIG-Mac@r-project.org
> > > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__stat.ethz.ch_mailman_listinfo_r-2Dsig-2Dmac&d=DwIFAg&c=eRAMFD45gAfqt84VtBcfhQ&r=BK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA&m=-PNBcsZhHb1MLy6fZakmLP6WDLMwL-VigJVzeFtPxOI&s=ubqivtpb1OZ2I0MEbVvGlGDK-tfti3DPtIv4LQ-Ks6A&e=
> > > >
> > > > --
> > > > Hervé Pagès
> > > >
> > > > Program in Computational Biology
> > > > Division of Public Health Sciences
> > > > Fred Hutchinson Cancer Research Center
> > > > 1100 Fairview Ave. N, M1-B514
> > > > P.O. Box 19024
> > > > Seattle, WA 98109-1024
> > > >
> > > > E-mail: hpa...@fredhutch.org
> > > > Phone: (206) 667-5791
> > > > Fax: (206) 667-1319
> > > >
> > > > _______________________________________________
> > > > R-SIG-Mac mailing list
> > > > R-SIG-Mac@r-project.org
> > > > https://stat.ethz.ch/mailman/listinfo/r-sig-mac
> > >
> > > _______________________________________________
> > > R-SIG-Mac mailing list
> > > R-SIG-Mac@r-project.org
> > > https://stat.ethz.ch/mailman/listinfo/r-sig-mac
> >
> > [[alternative HTML version deleted]]
> >
> > _______________________________________________
> > R-SIG-Mac mailing list
> > R-SIG-Mac@r-project.org
> > https://stat.ethz.ch/mailman/listinfo/r-sig-mac
>
[[alternative HTML version deleted]]
_______________________________________________
R-SIG-Mac mailing list
R-SIG-Mac@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-mac
