[
https://issues.apache.org/jira/browse/SPARK-11304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ram Venkatesh resolved SPARK-11304.
-----------------------------------
Resolution: Duplicate
Same as SPARK-10500
> SparkR in yarn-client mode fails creating sparkr.zip
> ----------------------------------------------------
>
> Key: SPARK-11304
> URL: https://issues.apache.org/jira/browse/SPARK-11304
> Project: Spark
> Issue Type: Bug
> Components: SparkR
> Affects Versions: 1.5.1
> Reporter: Ram Venkatesh
>
> If you run sparkR in yarn-client mode and the spark installation directory is
> not writable by the current user, it fails with
> Exception in thread "main" java.io.FileNotFoundException:
> /usr/hdp/2.3.2.1-12/spark/R/lib/sparkr.zip (Permission denied)
> at java.io.FileOutputStream.open0(Native Method)
> at java.io.FileOutputStream.open(FileOutputStream.java:270)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
> at
> org.apache.spark.deploy.RPackageUtils$.zipRLibraries(RPackageUtils.scala:215)
> at
> org.apache.spark.deploy.SparkSubmit$.prepareSubmitEnvironment(SparkSubmit.scala:371)
> at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:153)
> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:120)
> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
> The behavior is the same with the pre-built spark-1.5.1-bin-hadoop2.6
> bits also.
> We need to either use an existing sparkr.zip if we find one in the R/lib
> directory, or create the file in a location accessible to the submitting user.
> Temporary hack workaround - create a world-writable file called sparkr.zip
> under R/lib. It will still fail if multiple users submit jobs at the same
> time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
