Hi,
I am attempting to implement a mixed (windows/linux) snow sockets
parallelism in R, but am running into difficulties similar to a post made
Aug 31, 2006 under the same subject heading. I feel like I may be one or
two non-obvious steps away from getting it all working, but I'm stuck. If
anyone can shed some light on this (I believe Prof. Tierney stated that he
has successfully run a snow master on Windows with slave nodes on Linux
using ssh.exe through Cygwin, which is exactly what I am attempting), I'd be
most grateful.
SNOW master on Windows information:
WinXP 32bit
R-2.5.1
I built a windows package (.zip) version of snow-0.2-3 for R using Rtools
and installed it without trouble.
Cygwin including ssh as an exe and in the path
Linux slave nodes information:
ROCKS compute nodes, each 64bit (I mention this difference between the win
and linux platforms out of desperation but I don't think it's an issue since
snow should be agnostic). (btw, the SOCK version of snow here is just for
me to get used to parallelism in R before attempting MPI).
R-2.5.1, local directory install (no su, global R install is older).
Snow installed to R-2.5.1 instance
Unattended ssh authentication through public/private key pairs. Each node
allows since they are all NFS.
Rgui commands:
>library(snow)
>cl<-makeCluster(c("ip.path.to.node1","ip.path.to.node2"),type="SOCK")
Result:
R becomes unresponsive and has to be forcibly closed. Sometimes before that
happens message stating env: C:\pathToR\library\snow\RSOCKnode.sh no file
found appears (if I make R redraw the interface screen)
Troubleshooting step1:
Interspersed print commands within C:\localRinstall\library\snow\R\snow
file.
Result of troubleshooting step1:
It appears that this snow file hangs on line:
system(paste(rshcmd, "-l", user, machine, "env", env, script))
Resolution of troubleshooting step1:
Manually attempted paste commands in R:
>system("ssh -l me ip.path.to.node1 ls")
Result: gets ls of remote machine; so the script path must be
incorrect and the offending command
Located RSOCKnode.sh on remote machine (for all remote machines, since
it's NFS)
Inserted line into C:\localRinstall\library\snow\R\snow file
just before the offending line system(paste(rshcmd, "-l", user, machine,
"env", env, script)). That inserted line hard-codes the path of the script
and the script name like so:
script<-"/rootpath/home/cluster/me/R-2.5.1/library/snow/RSOCKnode.sh". This
worked, and the next print statement following the system call in the snow
file now prints to the screen. But again R hangs, this time at the line:
con <- socketConnection(port = port, server=TRUE, blocking=TRUE,
open="a+b").
Troubleshooting step2:
Attempted manual socketConnection via R commands:
> con<-socketConnection("ip.path.to.node1",port=22)
> showConnections()
description class mode text isopen can read can
write
3 "->ip.path.to.node1" "socket" "a+" "text" "opened" "yes" "yes"
*** so the socketConnection does work, but in SNOW it hangs at this last
command, and I'm completely stuck.
As a parallel test, I have installed SNOW on the R-2.5.1 instance on the
linux cluster. If I access the head node and launch R, I can use SNOW with
sockets successfully (from the linux head node as master to the linux
compute nodes as slaves). However, running R more or less continually on the
head node of a cluster is bad form. Ideally I would like to run a windows
snow master to the linux node slaves, but I can't seem to get past this
point.
For reference, I include the original post below. I am stuck at one command
in the snow file past where this last post had problems.
I hope this is clear. I am intent on solving this problem, so feel free to
ask questions if you have feedback but my description is not clear. I
really appreciate any help!
Yours,
Michael Janis
UCLA Bioinformatics
*****transcript from original post entitles "makeSOCKcluster" follows*****
makeSOCKcluster
Click to flag this post
by Hrishikesh Rajpathak Aug 31, 2006; 10:39pm :: Rate this Message: - Use
ratings to moderate (?)
Reply | Reply to Author | View Threaded | Show Only this Message
Hi,
I am a newbie to R and trying to implement parallelism in R. I am
currently using R-2.3.1, and Cygwin to run R on Windows xp.
ssh and all are working fine,
When I try to create a socket connection as
makeSOCKcluster(c("localhost","localhost")),
it just waits for the other prcess on localhost to get created and
respond. But this other process is not created.
To debug, I put print statements in the "snow " file in library\snow\r of
R after every statement that comes under Socket Implementation. I realized
that it does the execution till
system(a<-paste(rshcmd, "-l", user, machine, "env", env, script))
part and then it goes in wait state. It cannot run the next command which
is
con <- socketConnection(port = port, server=TRUE, blocking=TRUE,
open="a+b")
can someone please tell me what exactly could be the problem?
Thank you,
Rishi
---------------------------------
Here's a new way to find what you're looking for - Yahoo! Answers
[[alternative HTML version deleted]]
______________________________________________
[EMAIL PROTECTED] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: makeSOCKcluster
Click to flag this post
by Prof Brian Ripley Aug 31, 2006; 10:59pm :: Rate this Message: - Use
ratings to moderate (?)
Reply | Reply to Author | View Threaded | Show Only this Message
On Fri, 1 Sep 2006, Hrishikesh Rajpathak wrote:
> Hi,
>
> I am a newbie to R and trying to implement parallelism in R. I am
> currently using R-2.3.1, and Cygwin to run R on Windows xp.
Did you build R under Cygwin (which is not a supported platform), or are
you running a native Windows build of R? If the latter, you may be being
optimistic to expect R to run Unix commands in the same way as under Unix.
I presume you mean this line in package snow:
system(paste(rshcmd, "-l", user, machine, "env", env, script))
which appears to be attempting to run a shell script: no shell is used by
system() on Windows R. As the return value is not checked, this will not
fail.
R under Windows differs from R under Unix in a number of ways, and
system() is a major one.
> ssh and all are working fine,
>
> When I try to create a socket connection as
>
> makeSOCKcluster(c("localhost","localhost")),
>
> it just waits for the other prcess on localhost to get created and
respond. But this other process is not created.
>
> To debug, I put print statements in the "snow " file in library\snow\r
of R after every statement that comes under Socket Implementation. I
realized that it does the execution till
>
> system(a<-paste(rshcmd, "-l", user, machine, "env", env, script))
>
> part and then it goes in wait state. It cannot run the next command
which is
>
> con <- socketConnection(port = port, server=TRUE, blocking=TRUE,
> open="a+b")
>
> can someone please tell me what exactly could be the problem?
>
> Thank you,
> Rishi
--
Brian D. Ripley, [EMAIL PROTECTED]
Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/
University of Oxford, Tel: +44 1865 272861 (self)
1 South Parks Road, +44 1865 272866 (PA)
Oxford OX1 3TG, UK Fax: +44 1865 272595
______________________________________________
[EMAIL PROTECTED] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: makeSOCKcluster
Click to flag this post
by Luke Tierney Sep 01, 2006; 04:33am :: Rate this Message: - Use ratings to
moderate (?)
Reply | Reply to Author | View Threaded | Show Only this Message
On Fri, 1 Sep 2006, Prof Brian Ripley wrote:
> On Fri, 1 Sep 2006, Hrishikesh Rajpathak wrote:
>
>> Hi,
>>
>> I am a newbie to R and trying to implement parallelism in R. I am
>> currently using R-2.3.1, and Cygwin to run R on Windows xp.
>
> Did you build R under Cygwin (which is not a supported platform), or are
> you running a native Windows build of R? If the latter, you may be being
> optimistic to expect R to run Unix commands in the same way as under Unix.
>
> I presume you mean this line in package snow:
>
> system(paste(rshcmd, "-l", user, machine, "env", env, script))
>
> which appears to be attempting to run a shell script: no shell is used by
> system() on Windows R. As the return value is not checked, this will not
> fail.
>
> R under Windows differs from R under Unix in a number of ways, and
> system() is a major one.
This runs a command of the form
ssh -l user machine env X=x Y=z script
so if your ssh is a .exe this should be OK. I have successfully run a
snow master on Windows with slave nodes on Linux with I believe the
ssh coming from cygwin, but that was a while ago.
My suggestion is that you look at the command that is generated by the
paste and try running it by hand and see if you can work out why it
hangs.
Best,
luke
>
>
>> ssh and all are working fine,
>>
>> When I try to create a socket connection as
>>
>> makeSOCKcluster(c("localhost","localhost")),
>>
>> it just waits for the other prcess on localhost to get created and
respond. But this other process is not created.
>>
>> To debug, I put print statements in the "snow " file in library\snow\r
of R after every statement that comes under Socket Implementation. I
realized that it does the execution till
>>
>> system(a<-paste(rshcmd, "-l", user, machine, "env", env, script))
>>
>> part and then it goes in wait state. It cannot run the next command
which is
>>
>> con <- socketConnection(port = port, server=TRUE, blocking=TRUE,
>> open="a+b")
>>
>> can someone please tell me what exactly could be the problem?
>>
>> Thank you,
>> Rishi
>
>
--
Luke Tierney
Chair, Statistics and Actuarial Science
Ralph E. Wareham Professor of Mathematical Sciences
University of Iowa Phone: 319-335-3386
Department of Statistics and Fax: 319-335-3017
Actuarial Science
241 Schaeffer Hall email: [EMAIL PROTECTED]
Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu
[[alternative HTML version deleted]]
______________________________________________
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
