]
Sent: Tuesday, November 15, 2016 9:27 PM
To: David Robison
Cc: user@spark.apache.org
Subject: Re: Problem submitting a spark job using yarn-client as master
you can set hdfs as defaults,
sparksession.sparkContext().hadoopConfiguration().set("fs.defaultFS",
“hdfs://master_node:8020”
you can set hdfs as defaults,
sparksession.sparkContext().hadoopConfiguration().set("fs.defaultFS",
“hdfs://master_node:8020”);
Regards
Rohit
On Nov 16, 2016, at 3:15 AM, David Robison
mailto:david.robi...@psgglobal.net>> wrote:
I am trying to submit a spark job through the yarn-client master
I am trying to submit a spark job through the yarn-client master setting. The
job gets created and submitted to the clients but immediately errors out. Here
is the relevant portion of the log:
15:39:37,385 INFO [org.apache.spark.deploy.yarn.Client] (default task-1)
Requesting a new application