Hi Stuti Could you check if Rscript is installed on all of the worker machines in the Spark cluster ? You can ssh into the machines and check if Rscript can be found in $PATH.
Thanks Shivaram On Mon, Aug 18, 2014 at 10:05 PM, Stuti Awasthi <[email protected]> wrote: > Hi All, > > > > I am using R 3.1 and Spark 0.9 and installed SparkR successfully. Now when > I execute the “pi.R” example using spark master as local, then script > executes fine. > > But when I try to execute same example using master as spark cluster > master, then in throws Rcript error. > > > > Error : > > java.io.IOException: Cannot run program "Rscript": java.io.IOException: > error=2, No such file or directory > > at java.lang.ProcessBuilder.start(ProcessBuilder.java:475) > > at edu.berkeley.cs.amplab.sparkr.RRDD.compute(RRDD.scala:113) > > at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:241) > > at org.apache.spark.rdd.RDD.iterator(RDD.scala:232) > > at > org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:109) > > at org.apache.spark.scheduler.Task.run(Task.scala:53) > > at > org.apache.spark.executor.Executor$TaskRunner$$anonfun$run$1.apply$mcV$sp(Executor.scala:213) > > at > org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:46) > > at > org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:45) > > at java.security.AccessController.doPrivileged(Native Method) > > at javax.security.auth.Subject.doAs(Subject.java:416) > > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121) > > at > org.apache.spark.deploy.SparkHadoopUtil.runAsUser(SparkHadoopUtil.scala:45) > > at > org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:178) > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > > at java.lang.Thread.run(Thread.java:679) > > > > I have checked, Rscript is present in my system and I also exported it in > CLASSPATH and PATH variables. Script is given the permission 777 as there > are multiple users of the clusters. > > $ which Rscript > > /usr/local/bin/Rscript > > > > $ type -a Rscript > > Rscript is /usr/local/bin/Rscript > > > > $ echo $PATH > > > /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/var/local/maven3/bin/:/var/local/ant/bin/:/usr/lib/jvm/java-6-openjdk:/usr/local/bin/Rscript > > > > $ echo $CLASSPATH > > :/usr/local/bin/Rscript > > > > Also I am getting same error if I open R prompt and then execute the > commands one after another OR if I execute the script. > > > > Please suggest > > > > Thanks > > Stuti Awasthi > > > > > > > > ::DISCLAIMER:: > > ---------------------------------------------------------------------------------------------------------------------------------------------------- > > The contents of this e-mail and any attachment(s) are confidential and > intended for the named recipient(s) only. > E-mail transmission is not guaranteed to be secure or error-free as > information could be intercepted, corrupted, > lost, destroyed, arrive late or incomplete, or may contain viruses in > transmission. The e mail and its contents > (with or without referred errors) shall therefore not attach any liability > on the originator or HCL or its affiliates. > Views or opinions, if any, presented in this email are solely those of the > author and may not necessarily reflect the > views or opinions of HCL or its affiliates. Any form of reproduction, > dissemination, copying, disclosure, modification, > distribution and / or publication of this message without the prior > written consent of authorized representative of > HCL is strictly prohibited. If you have received this email in error > please delete it and notify the sender immediately. > Before opening any email and/or attachments, please check them for viruses > and other defects. > > > ---------------------------------------------------------------------------------------------------------------------------------------------------- >
