Hi Maxime,
You could achieve the same by directly referring the varaibale as ${Myvar}
(without going through conf).
Regards,
Mohammad
________________________________
From: Maxime Petazzoni <[email protected]>
To: [email protected]
Sent: Tuesday, March 20, 2012 11:36 AM
Subject: Implementing coord:conf
Hi all,
I'm not sure how many people on this list also follow the #oozie IRC
channel, so here goes.
I found out yesterday that, unfortunately, the coord:conf EL function
was not implemented, meaning that it was impossible to refer to
properties that were not valid Java identifiers. For example, I want to
define the start and end times of my coordinators using two properties
that are passed via the command-line when the coordinator job is
submitted to Oozie:
$ cat coordinator.xml
<coordinator-app name="test"
start="${coord:conf(bob.schedule.start)}"
end="${coord:conf(bob.schedule.end)}"
timezone="US/Pacific">
<action>
<workflow>
<app-path>${coord:conf(bob.app.path)}</app-path>
</workflow>
</action>
</coordinator>
$ ~/opt/oozie/bin/oozie -config .../job.properties \
-Dbob.schedule.start=2012-03-20T11:15Z \
-Dbob.schedule.end=2012-03-27T00:00Z \
-run
With job.properties containing a few other configuration properties,
including of course bob.app.path (the path of the workflow application
in HDFS).
A similar function, wf:conf(), already exists for workflows, and is
implemented in DagELFunctions.java#wf_conf(String property): it gets the
Configuration object from the workflow instance, and accesses said
property by name.
I'm trying to implement the same functionality for coordinators, but I
can't figure out how to access the configuration from
CoordELFunctions.java. It's not easy to deep-dive into such a codebase
like this. From what I can gather, the job configuration properties are
set as variables into the ELEvaluator in
CoordELEvaluator.java#setConfigToEval(), which is called when the
ELEvaluator is created.
So I thought all I had to do was to implement coord_conf in
CoordELFunctions.java that gets variables from the ELEvaluator:
/**
* Return a job configuration property for the coordinator.
*
* @param property property name.
* @return the value of the property, <code>null</code> if the property is
undefined.
*/
public static String coord_conf(String property) {
ELEvaluator eval = ELEvaluator.getCurrent();
return (String) eval.getVariable(property);
}
Then declare it in oozie-default.xml. Unfortunately, even though it seems like
my coord_conf function is called correctly, it doesn't seem like the variable
is resolved correctly. This in what I get in the Oozie log when I try all this:
Caused by: java.lang.Exception: Unable to evaluate
:${coord:conf(bob.schedule.start)}:
at
org.apache.oozie.coord.CoordELFunctions.evalAndWrap(CoordELFunctions.java:551)
at
org.apache.oozie.command.coord.CoordSubmitXCommand.resolveAttribute(CoordSubmitXCommand.java:848)
... 33 more
Caused by: javax.servlet.jsp.el.ELException: variable [bob] cannot be resolved
at
org.apache.oozie.util.ELEvaluator$Context.resolveVariable(ELEvaluator.java:106)
I also don't understand why the variable name here is 'bob' and not
'bob.schedule.start'. And if I add -Dbob=foo to the command-line, the
exception is a bit different:
Caused by: javax.servlet.jsp.el.ELException: Unable to find a value for
"schedule" in object of class "java.lang.String" using operator "."
Which makes me think that maybe I don't understand at all how variables
in ELEvaluator work ;-)
If anyone more familiar with the Oozie codebase or how this particular
area works could shed some light on what's going on here, I'd be very
grateful! And hopefully I can wrap this thing up and send a proper
patch to contribute to Oozie.
In the meantime, my current, incomplete patch is attached.
Thanks in advance,
/Maxime
--
Maxime Petazzoni, Platform Engineer at Turn, Inc (www.turn.com)
