[
https://issues.apache.org/jira/browse/OOZIE-2051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197440#comment-14197440
]
Venkat Ranganathan commented on OOZIE-2051:
-------------------------------------------
Thanks [~rkanter]
bq. Instead of XLog.getLog(...), you can just use LOG which is already defined
as that
Hmm. I used what was already in loadActionConf. I see your point though.
bq. For the action.xml files, we look for them once on startup; can we do that
with the action directories? Otherwise, it has to go through the whole action
directories thing every time an action is started
No sure I follow. loadActionConf() is called once when an action of that type
is created for the first time for a given cluster.
{code}
public XConfiguration createActionDefaultConf(String hostPort, String action) {
hostPort = (hostPort != null) ? hostPort.toLowerCase() : null;
Map<String, XConfiguration> hostPortActionConfigs =
actionConfigs.get(hostPort);
if (hostPortActionConfigs == null) {
hostPortActionConfigs = actionConfigs.get("*");
hostPort = "*";
}
XConfiguration actionConf = hostPortActionConfigs.get(action);
if (actionConf == null) {
// doing lazy loading as we don't know upfront all actions, no need
to synchronize
// as it is a read operation an in case of a race condition loading
and inserting
// into the Map is idempotent and the action-config Map is a
ConcurrentHashMap
actionConf = loadActionConf(hostPort, action);
hostPortActionConfigs.put(action, actionConf);
}
return new XConfiguration(actionConf.toProperties());
}
{code}
bq. The way this is now, properties specified in the the xml files in the
action directory has priority over the action.xml. I'm not sure if it should be
that way or the other way. Intuitively, I'd say the file should have priority
because it's more specific.
I see your point. If there is a conflict in the XML configuration, the job XML
used to win. With the setup, we are telling users not to use the job-xml for
configurations for providing default *-site files needed and use job-xml
elements for overriding purpose.
bq. Also, it's not providing any guarantee on the priority of properties
within the action directory. For example, if pig/a.xml and pig/b.xml both have
a property named "foo" with different values, it's not going to be necessarily
consistent with which has priority. I think that's going to cause problems for
the user. Unless you have a better idea, I think sorting the files
alphabetically (or reverse alphabetically) before loading them would be the
easiest. Make sure to mention the priority in the docs too.
Agreed. This is something I overlooked. Simply doing an Arrays.sort on the
file would be the easiest to provide predictable order
bq. Can you update the existing test, or add a new test, that verifies that it
works with multiple files? And the priority is correct between the action.xml
file, and the multiple files in the action directory?
Would do
bq. Can you add documentation on this feature? And improve/add documentation
on the original feature?
Yes we need to document this. Unfortunately there is no document on
action-conf yet in Oozie.
> Allows multiple default action configuration files per action
> -------------------------------------------------------------
>
> Key: OOZIE-2051
> URL: https://issues.apache.org/jira/browse/OOZIE-2051
> Project: Oozie
> Issue Type: Bug
> Affects Versions: 4.1.0
> Reporter: Venkat Ranganathan
> Assignee: Venkat Ranganathan
> Attachments: OOZIE-2051-2.patch, OOZIE-2051.patch
>
>
> Today, in oozie, we have a action-conf directory with one configuration file
> per action to provide default values. It is desirable to have a per action
> config directory where multiple configuration files can be either copied (or
> symlinked).
> This is useful for example, when we have an hive action with Tez engine (or
> Pig with tez engine) or other similar scenarios.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
