[
https://issues.apache.org/jira/browse/SPARK-19992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15936241#comment-15936241
]
Saisai Shao edited comment on SPARK-19992 at 3/22/17 12:51 PM:
---------------------------------------------------------------
Oh, I see. Check the code again, looks like "/*" cannot be worked with "local"
schema, only hadoop support schema like hdfs, file could support glob path.
I agree this probably just a setup / env problem.
was (Author: jerryshao):
Oh, I see. Check the code again, looks like "/*" cannot be worked with "local"
schema, only hadoop support schema like hdfs, file could support glob path.
> spark-submit on deployment-mode cluster
> ---------------------------------------
>
> Key: SPARK-19992
> URL: https://issues.apache.org/jira/browse/SPARK-19992
> Project: Spark
> Issue Type: Bug
> Components: Spark Submit
> Affects Versions: 2.0.2
> Environment: spark version 2.0.2
> hadoop version 2.6.0
> Reporter: narendra maru
>
> spark version 2.0.2
> hadoop version 2.6.0
> spark -submit command
> "spark-submit --class spark.mongohadoop.testing3 --master yarn --deploy-mode
> cluster --jars /home/ec2-user/jars/hgmongonew.jar,
> /home/ec2-user/jars/mongo-hadoop-spark-2.0.1.jar"
> after adding following in
> 1 Spark-default.conf
> spark.executor.extraJavaOptions -Dconfig.fuction.conf
> spark.yarn.jars=local:/usr/local/spark-2.0.2-bin-hadoop2.6/yarn/*
> spark.eventLog.dir=hdfs://localhost:9000/user/spark/applicationHistory
> spark.eventLog.enabled=true
> 2yarn-site.xml
> <property>
> <name>yarn.application.classpath</name>
> <value>
> /usr/local/hadoop-2.6.0/etc/hadoop,
> /usr/local/hadoop-2.6.0/,
> /usr/local/hadoop-2.6.0/lib/,
> /usr/local/hadoop-2.6.0/share/hadoop/common/,
> /usr/local/hadoop-2.6.0/share/hadoop/common/lib/
> /usr/local/hadoop-2.6.0/share/hadoop/hdfs/,
> /usr/local/hadoop-2.6.0/share/hadoop/hdfs/lib/,
> /usr/local/hadoop-2.6.0/share/hadoop/mapreduce/,
> /usr/local/hadoop-2.6.0/share/hadoop/mapreduce/lib/,
> /usr/local/hadoop-2.6.0/share/hadoop/tools/lib/,
> /usr/local/hadoop-2.6.0/share/hadoop/yarn/,
> /usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/*,
> /usr/local/spark-2.0.2-bin-hadoop2.6/jars/spark-yarn_2.11-2.0.2.jar
> </value>
> </property>
> Error on log:-
> Error: Could not find or load main class
> org.apache.spark.deploy.yarn.ApplicationMaster
> Error on terminal:-
> diagnostics: Application application_1489673977198_0002 failed 2 times due to
> AM Container for appattempt_1489673977198_0002_000002 exited with exitCode: 1
> For more detailed output, check application tracking
> page:http://bdg-hdp-sparkmaster:8088/proxy/application_1489673977198_0002/Then,
> click on links to logs of each attempt.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
