Re: [R-SIG-Mac] Forking issues in Mavericks
On Oct 30, 2013, at 6:13 PM, Jeroen Ooms jeroen.o...@stat.ucla.edu wrote: On Wed, Oct 30, 2013 at 2:57 PM, Simon Urbanek simon.urba...@r-project.org wrote: Ah, you're running it in R.app? Well this was the easiest way to show it; the same problem appears when forking form inside httpuv or so. AFAIR httpuv itself is not fork-safe, so it may be a different issue - in cases like this the exact way the process is setup plays an important role. This appears to be the case for another reason - see below. Anyway, my hunch was correct - it's libcurl crashing in SSL trying to use CF. I wish there was at least way to make situations like these fail more gracefully... it's fine if the misbehaved child proc is killed, but it's a bit painful that it also takes down its parent(s). There's nothing we can do to catch these situations before it is too late? I can't think of a way right now, because we have no idea that some 3rd party is calling CF. BTW: I was trying to cause a fork segfault with something like mcparallel(quartz()), but to my great astonishment it works in 10.9. So apparently it is now safe to call CF after fork() as long as you didn't initialize CF before forking. That's actually great news, because the server (e.g. in my use-case Rserve) just has to make sure that no CF is called and then all is well. Also if the child dies, it does NOT take down the parent - I suspect that there is more involved in your case, so we can take it offline if you want to provide the complete example of your issue. Cheers, Simon ___ R-SIG-Mac mailing list R-SIG-Mac@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-mac
Re: [R-SIG-Mac] Forking issues in Mavericks
On Oct 29, 2013, at 3:53 PM, Jeroen Ooms jeroen.o...@stat.ucla.edu wrote: On Tue, Oct 29, 2013 at 12:35 PM, Steve Lianoglou lianoglou.st...@gene.com wrote: Is this happening when you are running R from w/in R.app (which, I think, is not new), or are you running R from the terminal and still seeing this)? It happens when a forked child process tries to use some fork unsafe opratation, after the parent has already done so as well, see [1]. It is related to R.app in the sense that the GUI uses the CF event loop which is considered fork unsafe, and therefore a forked child process can't do any fork unsafe operations anymore. However it also happens in other contexts such as prefork in RApache [2]. But my actual question is: what is it about using httr::GET that is considered fork unsafe, which was not considered unsafe in 10.8? I didn't have time to look (will do later today), but a wild guess is that Apple was moving away from some common libraries like OpenSSL and replacing them with their own which could be CF-based and thus unsafe. Again, that's wild guess, I'll have to look at the trace to confirm that. Cheers, Simon [1] http://objectivistc.tumblr.com/post/16187948939/you-must-exec-a-core-foundation-fork-safety-tale [2] http://stackoverflow.com/questions/2344368/problem-configuring-rapache-on-os-x-10-5-8/2358834#2358834 ___ R-SIG-Mac mailing list R-SIG-Mac@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-mac ___ R-SIG-Mac mailing list R-SIG-Mac@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-mac
Re: [R-SIG-Mac] Forking issues in Mavericks
On Wed, Oct 30, 2013 at 10:08 AM, Simon Urbanek simon.urba...@r-project.org wrote: I didn't have time to look (will do later today), but a wild guess is that Apple was moving away from some common libraries like OpenSSL and replacing them with their own which could be CF-based and thus unsafe. Again, that's wild guess, I'll have to look at the trace to confirm that. Thanks. It's unfortunate though, these developments are limiting the usefulness of the parallel package on the mac, because a lot of basic functionality is now considered unsafe to use in a forked process, even though on Linux there is no problem at all. I also don't get why osx decides to kill both the parent and child process when the child attempt to do something unsafe. That doesn't seem the most graceful way of dealing with things. ___ R-SIG-Mac mailing list R-SIG-Mac@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-mac
Re: [R-SIG-Mac] Forking issues in Mavericks
Still, I do not get any error with package parallel in my Mavericks. I compiled though R with a gcc 4.8.2. On 30 Oct 2013, at 18:26, Jeroen Ooms jeroen.o...@stat.ucla.edu wrote: On Wed, Oct 30, 2013 at 10:08 AM, Simon Urbanek simon.urba...@r-project.org wrote: I didn't have time to look (will do later today), but a wild guess is that Apple was moving away from some common libraries like OpenSSL and replacing them with their own which could be CF-based and thus unsafe. Again, that's wild guess, I'll have to look at the trace to confirm that. Thanks. It's unfortunate though, these developments are limiting the usefulness of the parallel package on the mac, because a lot of basic functionality is now considered unsafe to use in a forked process, even though on Linux there is no problem at all. I also don't get why osx decides to kill both the parent and child process when the child attempt to do something unsafe. That doesn't seem the most graceful way of dealing with things. ___ R-SIG-Mac mailing list R-SIG-Mac@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-mac ___ R-SIG-Mac mailing list R-SIG-Mac@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-mac
Re: [R-SIG-Mac] Forking issues in Mavericks
Jeroen,
actually, it works just fine for me on 10.9 (with CRAN build of R):
library(parallel)
library(httr)
myfork - mcparallel(GET(https://api.github.com/users/hadley/repos;))
out - mccollect(myfork)
out
$`53129`
Response [https://api.github.com/users/hadley/repos]
Status: 200
Content-type: application/json; charset=utf-8
[{id:12241750,name:adv-r,full_name:hadley/adv-r,owner:{login:hadley,id:4196,avatar_url:https://2.gravatar.com/avatar/7ba164f40a50bc23dbb2aa825fb7bc16?d=https%3A%2F%2Fidenticons.gi
[...]
sessionInfo()
R version 3.0.2 Patched (2013-10-23 r64103)
Platform: x86_64-apple-darwin10.8.0 (64-bit)
locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
attached base packages:
[1] parallel stats graphics grDevices utils datasets methods
[8] base
other attached packages:
[1] httr_0.2
loaded via a namespace (and not attached):
[1] digest_0.6.3 RCurl_1.95-4.1 stringr_0.6.2 tools_3.0.2
system(uname -a)
Darwin ginaz.client.research.att.com 13.0.0 Darwin Kernel Version 13.0.0: Thu
Sep 19 22:22:27 PDT 2013; root:xnu-2422.1.72~6/RELEASE_X86_64 x86_64
BTW: if you really want to trace the crash, start the parent R and run this in
lldb:
pro at -n R -w
that will attach to the forked child process.
Cheers,
Simon
On Oct 29, 2013, at 3:08 PM, Jeroen Ooms jeroen.o...@stat.ucla.edu wrote:
The following works in Linux and OSX 10.8 but in OSX 10.9 it results
in a
__THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS_COREFOUNDATION_FUNCTIONALITY___YOU_MUST_EXEC__():
library(parallel)
library(httr)
myfork - mcparallel(GET(https://api.github.com/users/hadley/repos;))
out - mccollect(myfork)
I've seen this error before when trying to access X11 from inside a
forked process. In this case it seems to be related to SSL, but I'm
not quite sure how to further debug this in 10.9 (lldb anyone?). What
exactly might be triggering this error?
___
R-SIG-Mac mailing list
R-SIG-Mac@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-mac
___
R-SIG-Mac mailing list
R-SIG-Mac@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-mac
Re: [R-SIG-Mac] Forking issues in Mavericks
On Wed, Oct 30, 2013 at 1:41 PM, Simon Urbanek simon.urba...@r-project.org wrote: Jeroen, actually, it works just fine for me on 10.9 (with CRAN build of R): Hmm interesting. Here is a clip of what happens on my machine: http://youtu.be/GAtKa6P75Qs. Note that it shows the error messages from the child proc, but afterwards the parent process is also broken. I think it is no longer allowed to use fork() or exec(). This mac is an almost completely blank installation of Mavericks. I formatted and did a recovery of 10.8 before updating to 10.9. Afterwards installed R.app and xquarts. Other than that almost everything on the machine is factory settings. ___ R-SIG-Mac mailing list R-SIG-Mac@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-mac
Re: [R-SIG-Mac] Forking issues in Mavericks
On Oct 30, 2013, at 5:45 PM, Jeroen Ooms jeroen.o...@stat.ucla.edu wrote: On Wed, Oct 30, 2013 at 1:41 PM, Simon Urbanek simon.urba...@r-project.org wrote: Jeroen, actually, it works just fine for me on 10.9 (with CRAN build of R): Hmm interesting. Here is a clip of what happens on my machine: http://youtu.be/GAtKa6P75Qs. Note that it shows the error messages from the child proc, but afterwards the parent process is also broken. I think it is no longer allowed to use fork() or exec(). This mac is an almost completely blank installation of Mavericks. I formatted and did a recovery of 10.8 before updating to 10.9. Afterwards installed R.app and xquarts. Other than that almost everything on the machine is factory settings. Ah, you're running it in R.app? That less likely to work, because R.app is fully in CF by being an app, so anything that touches the event loop will die. R itself is trying to avoid that by disabling the EL calls in children, but libraries may not know. Anyway, my hunch was correct - it's libcurl crashing in SSL trying to use CF. There must be some code that is conditional in running in an app, so it thinks it's ok to use CF which fails (see below). A way around could be compiling static libcurl against openssl so it doesn't uses the system one. * thread #1: tid = 0x11e139, 0x7fff83381206 libdispatch.dylib`_dispatch_wakeup + 100, queue = 'com.apple.main-thread, stop reason = EXC_BAD_ACCESS (code=1, address=0x110) frame #0: 0x7fff83381206 libdispatch.dylib`_dispatch_wakeup + 100 frame #1: 0x7fff833817a8 libdispatch.dylib`_dispatch_queue_push_list_slow2 + 30 frame #2: 0x7fff83384145 libdispatch.dylib`_dispatch_mach_msg_send + 608 frame #3: 0x7fff83383e99 libdispatch.dylib`dispatch_mach_send + 136 frame #4: 0x7fff8198a864 libxpc.dylib`_xpc_connection_send_message_with_reply_f + 125 frame #5: 0x7fff8198a724 libxpc.dylib`xpc_connection_send_message_with_reply_sync + 180 frame #6: 0x7fff8c8cc193 CoreFoundation`-[CFPrefsPlistSource copyReplyForDaemonMessage:toConnection:error:] + 243 frame #7: 0x7fff8ca26820 CoreFoundation`__47-[CFPrefsPlistSource alreadylocked_synchronize]_block_invoke_2 + 352 frame #8: 0x7fff8c8cba9b CoreFoundation`withDaemonConnection + 299 frame #9: 0x7fff8c8cb4fb CoreFoundation`-[CFPrefsPlistSource alreadylocked_synchronize] + 587 frame #10: 0x7fff8c8cb1f3 CoreFoundation`_copyValueForKey + 131 frame #11: 0x7fff8c8cb147 CoreFoundation`-[CFPrefsPlistSource copyValueForKey:] + 71 frame #12: 0x7fff8c8f3730 CoreFoundation`___CFPreferencesCopyValueWithContainer_block_invoke + 32 frame #13: 0x7fff8c8ca097 CoreFoundation`+[CFPrefsSource withSourceForIdentifier:user:byHost:container:perform:] + 839 frame #14: 0x7fff8c8f36c7 CoreFoundation`_CFPreferencesCopyValueWithContainer + 231 frame #15: 0x7fff85c7e3de Security`_SSLContextReadDefault + 46 frame #16: 0x7fff8d52d8d6 libsystem_pthread.dylib`__pthread_once_handler + 65 frame #17: 0x7fff884b5156 libsystem_platform.dylib`_os_once + 73 frame #18: 0x7fff8d52d875 libsystem_pthread.dylib`pthread_once + 57 frame #19: 0x7fff85c7cec5 Security`SSLCreateContextWithRecordFuncs + 329 frame #20: 0x7fff85c7cd3f Security`SSLCreateContext + 25 frame #21: 0x7fff867a8fe4 libcurl.4.dylib`darwinssl_connect_common + 443 frame #22: 0x7fff8679a18c libcurl.4.dylib`Curl_ssl_connect_nonblocking + 36 frame #23: 0x7fff86778134 libcurl.4.dylib`Curl_http_connect + 77 frame #24: 0x7fff867847da libcurl.4.dylib`Curl_protocol_connect + 129 frame #25: 0x7fff867953cb libcurl.4.dylib`multi_runsingle + 840 frame #26: 0x7fff86794fd1 libcurl.4.dylib`curl_multi_perform + 166 frame #27: 0x7fff8678e944 libcurl.4.dylib`curl_easy_perform + 244 frame #28: 0x00010069a18a RCurl.so`R_curl_easy_perform(handle=0x000102ce6000, opts=0x000104026ca0, isProtected=0x00010090a848, encoding=0x00010087f000) + 106 at curl.c:93 ___ R-SIG-Mac mailing list R-SIG-Mac@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-mac
Re: [R-SIG-Mac] Forking issues in Mavericks
On Wed, Oct 30, 2013 at 2:57 PM, Simon Urbanek simon.urba...@r-project.org wrote: Ah, you're running it in R.app? Well this was the easiest way to show it; the same problem appears when forking form inside httpuv or so. Anyway, my hunch was correct - it's libcurl crashing in SSL trying to use CF. I wish there was at least way to make situations like these fail more gracefully... it's fine if the misbehaved child proc is killed, but it's a bit painful that it also takes down its parent(s). There's nothing we can do to catch these situations before it is too late? ___ R-SIG-Mac mailing list R-SIG-Mac@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-mac
Re: [R-SIG-Mac] Forking issues in Mavericks
Hi, On Tue, Oct 29, 2013 at 12:08 PM, Jeroen Ooms jeroen.o...@stat.ucla.edu wrote: The following works in Linux and OSX 10.8 but in OSX 10.9 it results in a __THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS_COREFOUNDATION_FUNCTIONALITY___YOU_MUST_EXEC__(): library(parallel) library(httr) myfork - mcparallel(GET(https://api.github.com/users/hadley/repos;)) out - mccollect(myfork) I've seen this error before when trying to access X11 from inside a forked process. In this case it seems to be related to SSL, but I'm not quite sure how to further debug this in 10.9 (lldb anyone?). What exactly might be triggering this error? Is this happening when you are running R from w/in R.app (which, I think, is not new), or are you running R from the terminal and still seeing this)? -steve -- Steve Lianoglou Computational Biologist Bioinformatics and Computational Biology Genentech ___ R-SIG-Mac mailing list R-SIG-Mac@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-mac
Re: [R-SIG-Mac] Forking issues in Mavericks
Same system, no error (on the shell). Debugging parallel processes is in general not easy - here is a thread on this using lldb: http://stackoverflow.com/questions/14746315/how-to-attach-to-child-process-in-lldb. But first hope for an answer of a more experienced list user, who possibly knows how the processes are actually forked. Are you sure, that you call just these commands? Or did you use these for a simple example? Usually the error comes up from the Core Foundation library, if a child process tries to fork unsafely. Regarding the your problems with X11 though the answer from Steve sounds like a very good guess. Best Simon On 29 Oct 2013, at 20:08, Jeroen Ooms jeroen.o...@stat.ucla.edu wrote: The following works in Linux and OSX 10.8 but in OSX 10.9 it results in a __THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS_COREFOUNDATION_FUNCTIONALITY___YOU_MUST_EXEC__(): library(parallel) library(httr) myfork - mcparallel(GET(https://api.github.com/users/hadley/repos;)) out - mccollect(myfork) I've seen this error before when trying to access X11 from inside a forked process. In this case it seems to be related to SSL, but I'm not quite sure how to further debug this in 10.9 (lldb anyone?). What exactly might be triggering this error? ___ R-SIG-Mac mailing list R-SIG-Mac@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-mac ___ R-SIG-Mac mailing list R-SIG-Mac@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-mac
Re: [R-SIG-Mac] Forking issues in Mavericks
On Tue, Oct 29, 2013 at 12:35 PM, Steve Lianoglou lianoglou.st...@gene.com wrote: Is this happening when you are running R from w/in R.app (which, I think, is not new), or are you running R from the terminal and still seeing this)? It happens when a forked child process tries to use some fork unsafe opratation, after the parent has already done so as well, see [1]. It is related to R.app in the sense that the GUI uses the CF event loop which is considered fork unsafe, and therefore a forked child process can't do any fork unsafe operations anymore. However it also happens in other contexts such as prefork in RApache [2]. But my actual question is: what is it about using httr::GET that is considered fork unsafe, which was not considered unsafe in 10.8? [1] http://objectivistc.tumblr.com/post/16187948939/you-must-exec-a-core-foundation-fork-safety-tale [2] http://stackoverflow.com/questions/2344368/problem-configuring-rapache-on-os-x-10-5-8/2358834#2358834 ___ R-SIG-Mac mailing list R-SIG-Mac@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-mac
