[
https://issues.apache.org/jira/browse/OOZIE-736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13223822#comment-13223822
]
[email protected] commented on OOZIE-736:
-----------------------------------------------------
bq. On 2012-03-06 10:13:38, Mohammad Islam wrote:
bq. > Hi Alejandro,
bq. > So how/who will the hadoop-config.xml be populated? Because it is
expected to be cluster specific configurations defined in *-site.xml (such as
core, hdfs etc).
bq. > Is it we have to create another file? Or we can **reuse** the cluster
specified *-site.xml as it is. Only thing we have to configure those file names
into oozie-site.xml.
bq. > right?
bq. >
bq. > Regards,
bq. > Mohammad
bq. >
bq. >
bq.
bq. Alejandro Abdelnur wrote:
bq. Mohammad,
bq.
bq. you have to create a hadoop-config.xml file per cluster, as you
indicated typically will have the information the *-site.xml files of the
cluster used by the clients. You could reuse the cluster specific ones by
renaming them, still you may have to combine the core/hdfs/mapred/yarn. And
yes, the only thing is to configure them in the the oozie-site.xml and copy
them into the oozie conf/ dir.
bq.
bq. Mohammad Islam wrote:
bq. can't we use them by reusing the file as it is without renaming and
without combining?
bq. This is another level of task and indirection.
bq. Renaming + combining could be handled through some configuration in
oozie-site.xml.
bq. Because there will be an impact on us.
bq.
bq.
bq.
Mohammad,
Are you suggesting that instead pointing to a file we should point to a
directory and from there load all the hadoop *-site.xml files ?
We could do that, yes it would be simpler for admins.
My concern here is that we would have to be aware from Oozie side on the order
of loading the *-site.xml files. That is why I've thought not to do it.
But you have point on usability.
Thoughts?
- Alejandro
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4186/#review5637
-----------------------------------------------------------
On 2012-03-06 05:39:54, Alejandro Abdelnur wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4186/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-03-06 05:39:54)
bq.
bq.
bq. Review request for oozie.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. hadoop.configurations= [<AUTHORITY>=<CONFIG_FILE>]*
bq.
bq. Where <AUTHORITY> is the HOST:PORT of the Hadoop JT/NN and <CONFIG_FILE>
is the Hadoop configuration file to load from the local filesystem with the
defaults for the corresponding JT/NN.
bq.
bq. In addition, a '*' wildcard would be support for the <AUTHORITY>, this
would be used as default if the job used authority does not have a custom
config.
bq.
bq. The Hadoop configuration file would be looked in the Oozie conf/ directory
bq.
bq. An example of this configuration would be:
bq.
bq. hadoop.configuration= *=hadoop-defaultconf.xml
myjobtracker:8021=hadoop-cluster1.xml, mynamenode:8020=hadoop-cluster1.xml
bq.
bq. Documentation is still missing, coming soon.
bq.
bq.
bq. This addresses bug OOZIE-736.
bq. https://issues.apache.org/jira/browse/OOZIE-736
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. trunk/core/src/main/conf/hadoop-config.xml PRE-CREATION
bq.
trunk/core/src/main/java/org/apache/oozie/service/HadoopAccessorService.java
1297338
bq. trunk/core/src/main/resources/oozie-default.xml 1297338
bq.
trunk/core/src/test/java/org/apache/oozie/service/TestHadoopAccessorService.java
1297338
bq. trunk/core/src/test/java/org/apache/oozie/test/XFsTestCase.java 1297338
bq. trunk/core/src/test/java/org/apache/oozie/test/XTestCase.java 1297338
bq. trunk/core/src/test/resources/hadoop-config.xml PRE-CREATION
bq. trunk/core/src/test/resources/test-hadoop-config.xml PRE-CREATION
bq. trunk/docs/src/site/twiki/AG_HadoopConfiguration.twiki PRE-CREATION
bq. trunk/docs/src/site/twiki/DG_UsingHadoopKerberos.twiki 1297338
bq. trunk/docs/src/site/twiki/index.twiki 1297338
bq.
bq. Diff: https://reviews.apache.org/r/4186/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. run all testcases with Hadoop 1
bq.
bq.
bq. Thanks,
bq.
bq. Alejandro
bq.
bq.
> Add support for configurations per JT/NN
> ----------------------------------------
>
> Key: OOZIE-736
> URL: https://issues.apache.org/jira/browse/OOZIE-736
> Project: Oozie
> Issue Type: New Feature
> Components: core
> Affects Versions: 3.2.0
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Priority: Critical
> Fix For: 3.2.0
>
>
> Currently Oozie does not support oozie-server-side configs for the JT/NN and
> users must add all values explicitly in their workflows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
