Hi ,
  Trying to access a property that I set under <configuration> of a Spark
action from within the Spark driver class returns null.  I notice a
discussion similar to this in
http://mail-archives.apache.org/mod_mbox/oozie-user/201507.mbox/%3ccad-ua_jyv1uglgm1n8q30jtw_tceyiwgngu0yut2_umftve...@mail.gmail.com%3E
. I don't see the property (job.input.path) when I try to print all the
properties of SparkConf . Below is a snapshot of the spark action.

<action name="spark-helloworld">
    <spark xmlns="uri:oozie:spark-action:0.1">
        <job-tracker>${jobTracker}</job-tracker>
        <name-node>${nameNode}</name-node>
        <configuration>
            <property>
                <name>job.input.path</name>
                <value>${nameNode}/path</value>
            </property>
        </configuration>
        <master>yarn-cluster</master>
        <mode>cluster</mode>
        <name>spark-hw</name>
        <class>SparkHelloWorld</class>
        <jar>${nameNode}/apps/helloworld.jar</jar>
        <spark-opts> --queue oozie --num-executors 2 --executor-memory
2g --driver-memory 2g</spark-opts>
      </spark>
    <ok to="mark-job-end"/>
    <error to="mark-job-fail"/>
</action>


I can access the property "job.input.path" by passing as an <arg> but
unable to when i set it under <configuration>  .  Is this a bug on the
version i have ?

The environment I am running on is CDH 5.4.8 which has Oozie
v4.1.0-cdh5.4.8 .

Regards
Ravi

Reply via email to