Hi ,
Trying to access a property that I set under <configuration> of a Spark
action from within the Spark driver class returns null. I notice a
discussion similar to this in
http://mail-archives.apache.org/mod_mbox/oozie-user/201507.mbox/%3ccad-ua_jyv1uglgm1n8q30jtw_tceyiwgngu0yut2_umftve...@mail.gmail.com%3E
. I don't see the property (job.input.path) when I try to print all the
properties of SparkConf . Below is a snapshot of the spark action.
<action name="spark-helloworld">
<spark xmlns="uri:oozie:spark-action:0.1">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<configuration>
<property>
<name>job.input.path</name>
<value>${nameNode}/path</value>
</property>
</configuration>
<master>yarn-cluster</master>
<mode>cluster</mode>
<name>spark-hw</name>
<class>SparkHelloWorld</class>
<jar>${nameNode}/apps/helloworld.jar</jar>
<spark-opts> --queue oozie --num-executors 2 --executor-memory
2g --driver-memory 2g</spark-opts>
</spark>
<ok to="mark-job-end"/>
<error to="mark-job-fail"/>
</action>
I can access the property "job.input.path" by passing as an <arg> but
unable to when i set it under <configuration> . Is this a bug on the
version i have ?
The environment I am running on is CDH 5.4.8 which has Oozie
v4.1.0-cdh5.4.8 .
Regards
Ravi