[R] makeSOCKcluster
Hi, I am attempting to implement a mixed (windows/linux) snow sockets parallelism in R, but am running into difficulties similar to a post made Aug 31, 2006 under the same subject heading. I feel like I may be one or two non-obvious steps away from getting it all working, but I'm stuck. If anyone can shed some light on this (I believe Prof. Tierney stated that he has successfully run a snow master on Windows with slave nodes on Linux using ssh.exe through Cygwin, which is exactly what I am attempting), I'd be most grateful. SNOW master on Windows information: WinXP 32bit R-2.5.1 I built a windows package (.zip) version of snow-0.2-3 for R using Rtools and installed it without trouble. Cygwin including ssh as an exe and in the path Linux slave nodes information: ROCKS compute nodes, each 64bit (I mention this difference between the win and linux platforms out of desperation but I don't think it's an issue since snow should be agnostic). (btw, the SOCK version of snow here is just for me to get used to parallelism in R before attempting MPI). R-2.5.1, local directory install (no su, global R install is older). Snow installed to R-2.5.1 instance Unattended ssh authentication through public/private key pairs. Each node allows since they are all NFS. Rgui commands: library(snow) cl-makeCluster(c(ip.path.to.node1,ip.path.to.node2),type=SOCK) Result: R becomes unresponsive and has to be forcibly closed. Sometimes before that happens message stating env: C:\pathToR\library\snow\RSOCKnode.sh no file found appears (if I make R redraw the interface screen) Troubleshooting step1: Interspersed print commands within C:\localRinstall\library\snow\R\snow file. Result of troubleshooting step1: It appears that this snow file hangs on line: system(paste(rshcmd, -l, user, machine, env, env, script)) Resolution of troubleshooting step1: Manually attempted paste commands in R: system(ssh -l me ip.path.to.node1 ls) Result: gets ls of remote machine; so the script path must be incorrect and the offending command Located RSOCKnode.sh on remote machine (for all remote machines, since it's NFS) Inserted line into C:\localRinstall\library\snow\R\snow file just before the offending line system(paste(rshcmd, -l, user, machine, env, env, script)). That inserted line hard-codes the path of the script and the script name like so: script-/rootpath/home/cluster/me/R-2.5.1/library/snow/RSOCKnode.sh. This worked, and the next print statement following the system call in the snow file now prints to the screen. But again R hangs, this time at the line: con - socketConnection(port = port, server=TRUE, blocking=TRUE, open=a+b). Troubleshooting step2: Attempted manual socketConnection via R commands: con-socketConnection(ip.path.to.node1,port=22) showConnections() description classmode text isopen can read can write 3 -ip.path.to.node1 socket a+ text opened yesyes *** so the socketConnection does work, but in SNOW it hangs at this last command, and I'm completely stuck. As a parallel test, I have installed SNOW on the R-2.5.1 instance on the linux cluster. If I access the head node and launch R, I can use SNOW with sockets successfully (from the linux head node as master to the linux compute nodes as slaves). However, running R more or less continually on the head node of a cluster is bad form. Ideally I would like to run a windows snow master to the linux node slaves, but I can't seem to get past this point. For reference, I include the original post below. I am stuck at one command in the snow file past where this last post had problems. I hope this is clear. I am intent on solving this problem, so feel free to ask questions if you have feedback but my description is not clear. I really appreciate any help! Yours, Michael Janis UCLA Bioinformatics *transcript from original post entitles makeSOCKcluster follows* makeSOCKcluster Click to flag this post by Hrishikesh Rajpathak Aug 31, 2006; 10:39pm :: Rate this Message: - Use ratings to moderate (?) Reply | Reply to Author | View Threaded | Show Only this Message Hi, I am a newbie to R and trying to implement parallelism in R. I am currently using R-2.3.1, and Cygwin to run R on Windows xp. ssh and all are working fine, When I try to create a socket connection as makeSOCKcluster(c(localhost,localhost)), it just waits for the other prcess on localhost to get created and respond. But this other process is not created. To debug, I put print statements in the snow file in library\snow\r of R after every statement that comes under Socket Implementation. I realized that it does the execution till system(a-paste(rshcmd, -l, user, machine, env, env, script)) part and then it goes in wait state. It cannot run the next command which is con
[R] 2nd R Console
Hi, I was reading a thread: [R] 2nd R console and had a similar question regarding having more than one R console open at a time. However, my question differs from that of the thread: Is it possible, or is there a wrapper that will allow one, to open an arbitrary number of R consoles which access the same R session (all objects in that session, etc.). This would be R on linux accessed through a shell - kind of like using GNU screen multi-user such that people could work collaboratively on a given session. The problem with screen is that all commands are interleaved in the same terminal, which is confusing and does not allow access to the command prompt at the same time, rather it would be sequential. I know there will be why questions but it is useful in an academic environment. Basically we have a memory machine for large genomic analysis - and we could set that up as an Rserver, but this placing R into a multi-user engine is better suited for our immediate needs. Does anybody have thoughts on this? Thanks for considering, Michael Janis UCLA Bioinformatics __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] 2nd R Console
Hi, I was reading a thread: [R] 2nd R console and had a similar question regarding having more than one R console open at a time. However, my question differs from that of the thread: Is it possible, or is there a wrapper that will allow one, to open an arbitrary number of R consoles which access the same R session (all objects in that session, etc.). This would be R on linux accessed through a shell - kind of like using GNU screen multi-user such that people could work collaboratively on a given session. The problem with screen is that all commands are interleaved in the same terminal, which is confusing and does not allow access to the command prompt at the same time, rather it would be sequential. I know there will be why questions but it is useful in an academic environment. Basically we have a memory machine for large genomic analysis - and we could set that up as an Rserver, but this placing R into a multi-user engine is better suited for our immediate needs. Does anybody have thoughts on this? Thanks for considering, Michael Janis UCLA Bioinformatics __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] 2nd R Console
Oh now that *is* something interesting! Thank you Greg, I'll have to give this a try. -Original Message- From: Greg Snow [mailto:[EMAIL PROTECTED] Sent: Monday, July 30, 2007 3:44 PM To: [EMAIL PROTECTED]; r-help@stat.math.ethz.ch Subject: RE: [R] 2nd R Console Have you looked at the nws package? It allows for a common workspace that multiple R sessions can all access. Hope this helps, -Original Message- From: Michael Janis [EMAIL PROTECTED] To: r-help@stat.math.ethz.ch r-help@stat.math.ethz.ch Sent: 7/30/07 7:49 AM Subject: [R] 2nd R Console Hi, I was reading a thread: [R] 2nd R console and had a similar question regarding having more than one R console open at a time. However, my question differs from that of the thread: Is it possible, or is there a wrapper that will allow one, to open an arbitrary number of R consoles which access the same R session (all objects in that session, etc.). This would be R on linux accessed through a shell - kind of like using GNU screen multi-user such that people could work collaboratively on a given session. The problem with screen is that all commands are interleaved in the same terminal, which is confusing and does not allow access to the command prompt at the same time, rather it would be sequential. I know there will be why questions but it is useful in an academic environment. Basically we have a memory machine for large genomic analysis - and we could set that up as an Rserver, but this placing R into a multi-user engine is better suited for our immediate needs. Does anybody have thoughts on this? Thanks for considering, Michael Janis UCLA Bioinformatics __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
