Petře, thanks, for the detailed analysis. It is rather curious that the issue appears only on _newer_ systems - we are more used to issues due to older CA chains and similar. It looks like an Apple bug on specific systems, so hopefully it will be fixed eventually. In general I was trying to avoid having to supply our own SSL library since that opens a whole can of worms - on one hand due the dependency issues (which libraries get compiled against what) and on the other hand we become responsible for security updates.
Thanks to Jeroen for the work-around (CURL_SSL_BACKEND=SecureTransport), using the native API is certainly preferred, there have been several issues with both OpenSSL and LibreSSL before. It seems that Apple has been flip-flopping with libcurl a lot - on El Capitan it was shipped with SecureTransport, on High-Sierra with LibreSSL, on Catalina and higher with both, but Libre the default. I am somewhat less apprehensive to use static libcurl for R than SSL libraries as the fallout is a bit smaller. As a trial I have added static curl[2] which is close to the Apple build minus MultiSSL to big-sur nightly builds of R[3] and as expected that solves the problem. It may not be entirely unproblematic for package space, because packages often forget to prepend --static when using static builds of libraries, and so do other dependencies that may use curl, but I'll see what comes out of it. Cheers, Šimon [1] - https://github.com/R-macos/recipes [2] - https://github.com/R-macos/recipes/blob/add-ons/recipes/curl [3] - https://mac.r-project.org/ > On 10/01/2022, at 11:22 PM, Petr Bouchal <pbouc...@gmail.com> wrote: > > Dear all, > > In brief: on Monterey, R cannot reach certain web domains due to a bug in > Libre SSL - and perhaps not relying on system curl/openssl in R would be a > systematic solution to this and símilar issues. > > Specifically: on MacOS Monterey 12.1 using R 4.1.2, download.file() and other > functions that rely on system-provided curl/openssl/Libre SSL (including in > the curl package) have been failing on specific domains. > > So running > > download.file(“https://www.czso.cz/”, tempfile()) > > returns: > > status was ‘SSL connect error’ > > the underlying error being > > error:06FFF089:digital envelope routines:CRYPTO_internal:bad key length. > > This is caused by the Libre SSL bundled in MacOS Monterey and also affects > several other domains, most notably https://libzip.org. > > It is clearly an OS bug but infortunately also a situation where it affects R > users because of how R relates to system libraries and is very difficult to > work around. > > It has manifested on CRAN (causing a package archival) and Github outside of > R, so is not caused by a specific machine. It can be replicated on both M1 > and Intel and also occurs when using curl in the system command line. > > The czso.cz domain is the Czech Statistical Office, which makes it quite > important for a number of users, also of a package I maintain (czso) which > relies on accessing this domain. I have reported this to the server admin but > since the problem is in the OS, I do not expect them to be able to help. I am > not an expert in web security so cannot tell if there is anything in the > certificates which could be causing this. In browsers, no such issue occurs > and the server is configured correctly as per ssllabs.com testing. I have > also reported to Apple but it is unclear whether they will fix this given the > rare nature of the issue. > > It is difficult to work around even on individual machines as replacing the > system curl/openssl requires steps beyond what a most users are comfortable > with (or should be doing to begin with). Using HTTP instead of HTTPS does not > work, nor does using curl —insecure and equivalents. > > This brings back the question of whether R on MacOS should include its own > openssl instead of relying on the system-provided library. This has been > discussed on the r-devel list: > https://stat.ethz.ch/pipermail/r-devel/2020-June/079657.html. > > Apple also recommends against relying on shared openssl, if I understand this > correctly: https://developer.apple.com/forums/thread/89051. Given Apple’s > approach to openssl/Libre SSL in MacOS (the bundled Libre SSL version is 3 > years old), such hard-to-handle issues are likely to reappear over time. (I > don’t have in-depth knowledge of how R is compiled, so apologies for any > inaccuracies; hopefully it is clear what I mean.) > > I’d be grateful for any thoughts on how this might be handled in the specific > case and perhaps generally. > > Kind regards > Petr Bouchal > > _______________________________________________ > R-SIG-Mac mailing list > R-SIG-Mac@r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-mac > _______________________________________________ R-SIG-Mac mailing list R-SIG-Mac@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-mac
