[
https://issues.apache.org/jira/browse/OOZIE-2187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Kanter updated OOZIE-2187:
---------------------------------
Attachment: OOZIE-2187.patch
As part of this, I also refactored and fixed some stuff in the parsing code,
especially with the global section:
- There were a bunch of nested {{if}} statements under {{else}} statements in
{{LiteWorkflowAppParser#validateAndParse}}. I changed these to {{if else}}
statements to make it easier to read. No logical changes here, other than
trivial changes to support the next item.
- We used to re-parse the global section for every action. I've now changed it
where we parse it once to populate a new data structure, and then use this data
structure to update actions
- The global section parsing code used to use a public boolean in the
{{ActionExecutor}} classes to determine if an action required a NameNode and
JobTracker, and would throw a {{WorkflowException}} if they didn't have one.
However, this is bad style; I've replaced it with a
{{requiresNameNodeJobTracker()}} method call.
- The global section parsing code used to add the {{<name-node>}},
{{<job-tracker>}}, {{<job-xml>}}, and {{<configuration>}} from {{<global>}} to
every action, even if they didn't want it (e.g. email action). While not
necessarily harmful, this could be problematic in the future and results in
larger database usage. It now only adds a {{<name-node>}} and
{{<job-tracker>}} if {{requiresNameNodeJobTracker()}} is {{true}} and only adds
a {{<job-xml>}} and {{<configuration>}} if a new
{{supportsConfigurationJobXML()}} is {{true}}.
- I added {{oozie.actions.default.name-node}} and
{{oozie.actions.default.job-tracker}} properties to oozie-site for the default
NameNode and JobTracker/ResourceManager. The parser puts these in if neither
the action nor the global section defines them and
{{requiresNameNodeJobTracker()}} is {{true}}.
- I added and updated a bunch of tests for all the stuff above.
> Add a way to specify a default JT/RM and NN
> -------------------------------------------
>
> Key: OOZIE-2187
> URL: https://issues.apache.org/jira/browse/OOZIE-2187
> Project: Oozie
> Issue Type: New Feature
> Components: core
> Reporter: Robert Kanter
> Assignee: Robert Kanter
> Attachments: OOZIE-2187.patch
>
>
> Oozie is cluster agnostic, which is why we require an RM/JT and NN per action
> in your workflow (or once via the <global> section). In practice, many users
> use one Oozie server per cluster, so it's an extra burden for them to have to
> specify this all the time. It would be convenient if we added configuration
> properties to oozie-site that would let you specify a default RM/JT and NN to
> use.
> This way, these users could completely omit the {{<job-tracker>}} and
> {{<name-node>}} fields from their workflows; as an added benefit, they can
> easily update these values if they ever rename/move their RM/JT or NN. We'd
> of course still allow specifying {{<job-tracker>}} and {{<name-node>}} in
> each action and {{<global>}} to allow individual workflows or actions to
> override the default.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
