Hi!
There is no easy and straightforward way of doing this for the Spark
action, but you can take advantage of the fact that Oozie 4.1.0 uses
MapReduce to launch Spark.
Just put "mapred.map.child.env" in the action configuration using the
format k1=v1,k2=v2. EL functions should also work here.
Gp
On Thu, May 10, 2018 at 6:39 PM, Richard Primera <
richard.prim...@woombatcg.com> wrote:
> Greetings,
>
> How can I set an environment variable to be accessible from either a .jar
> or .py script launched via a spark action?
>
> The idea is to set the environment variable with the output of the EL
> function ${wf:id()} from within the XML workflow definition, something
> along these lines:
>
> <jar>script.py</jar>
>
> <env>OOZIE_WORKFLOW_ID=${wf:id()}</env>
>
> And then have the ability to do wf_id = os.getenv("OOZIE_WORKFLOW_ID")
> from the script without having to pass them as command line arguments. The
> thing about command line arguments is that they don't scale as well because
> they rely on a specific ordering or some custom parsing implementation.
> This can be done easily it seems with a shell action, but I've been unable
> to find a similar straightforward way of doing it for a spark action.
>
> Oozie Version: 4.1.0-cdh5.12.1
>
>
--
*Peter Cseh *| Software Engineer
cloudera.com <https://www.cloudera.com>
[image: Cloudera] <https://www.cloudera.com/>
[image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
on LinkedIn] <https://www.linkedin.com/company/cloudera>
------------------------------
