[
https://issues.apache.org/jira/browse/SPARK-19992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen resolved SPARK-19992.
-------------------------------
Resolution: Not A Problem
[~jerryshao] is "/*" even going to work?
This must be an env problem because YARN cluster mode certainly works, and it's
not clear the env is correct here. Here the Spark JAR is also being submitted
with --jars, and it's not a matching version even (though I doubt that's the
problem).
> spark-submit on deployment-mode cluster
> ---------------------------------------
>
> Key: SPARK-19992
> URL: https://issues.apache.org/jira/browse/SPARK-19992
> Project: Spark
> Issue Type: Bug
> Components: Spark Submit
> Affects Versions: 2.0.2
> Environment: spark version 2.0.2
> hadoop version 2.6.0
> Reporter: narendra maru
>
> spark version 2.0.2
> hadoop version 2.6.0
> spark -submit command
> "spark-submit --class spark.mongohadoop.testing3 --master yarn --deploy-mode
> cluster --jars /home/ec2-user/jars/hgmongonew.jar,
> /home/ec2-user/jars/mongo-hadoop-spark-2.0.1.jar"
> after adding following in
> 1 Spark-default.conf
> spark.executor.extraJavaOptions -Dconfig.fuction.conf
> spark.yarn.jars=local:/usr/local/spark-2.0.2-bin-hadoop2.6/yarn/*
> spark.eventLog.dir=hdfs://localhost:9000/user/spark/applicationHistory
> spark.eventLog.enabled=true
> 2yarn-site.xml
> <property>
> <name>yarn.application.classpath</name>
> <value>
> /usr/local/hadoop-2.6.0/etc/hadoop,
> /usr/local/hadoop-2.6.0/,
> /usr/local/hadoop-2.6.0/lib/,
> /usr/local/hadoop-2.6.0/share/hadoop/common/,
> /usr/local/hadoop-2.6.0/share/hadoop/common/lib/
> /usr/local/hadoop-2.6.0/share/hadoop/hdfs/,
> /usr/local/hadoop-2.6.0/share/hadoop/hdfs/lib/,
> /usr/local/hadoop-2.6.0/share/hadoop/mapreduce/,
> /usr/local/hadoop-2.6.0/share/hadoop/mapreduce/lib/,
> /usr/local/hadoop-2.6.0/share/hadoop/tools/lib/,
> /usr/local/hadoop-2.6.0/share/hadoop/yarn/,
> /usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/*,
> /usr/local/spark-2.0.2-bin-hadoop2.6/jars/spark-yarn_2.11-2.0.2.jar
> </value>
> </property>
> Error on log:-
> Error: Could not find or load main class
> org.apache.spark.deploy.yarn.ApplicationMaster
> Error on terminal:-
> diagnostics: Application application_1489673977198_0002 failed 2 times due to
> AM Container for appattempt_1489673977198_0002_000002 exited with exitCode: 1
> For more detailed output, check application tracking
> page:http://bdg-hdp-sparkmaster:8088/proxy/application_1489673977198_0002/Then,
> click on links to logs of each attempt.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
