[
https://issues.apache.org/jira/browse/OOZIE-2343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14717181#comment-14717181
]
Rohini Palaniswamy commented on OOZIE-2343:
-------------------------------------------
[~rkanter],
I prefer that you make oozie.action.shell.setup.hadoop.conf.dir false by
default. With Hadoop 2.x, tarballs are used to setup the client side *-site.xml
files to support rolling upgrades. Hacking shell action to create the xml files
does not make sense in those cases. For Pig/Hive we already added option
(OOZIE-2205) to make the action xml not have the defaults and only ship
overriden settings and let the Pig/Hive launcher jobs pick the local
*-site.xml files from the mapreduce tarball. You can still have this option for
users who do not have mapreduce client tarball in hdfs and instead use
installation in node managers.
> Shell Action should take Oozie Action config and setup HADOOP_CONF_DIR
> ----------------------------------------------------------------------
>
> Key: OOZIE-2343
> URL: https://issues.apache.org/jira/browse/OOZIE-2343
> Project: Oozie
> Issue Type: Improvement
> Components: action
> Affects Versions: trunk
> Reporter: Robert Kanter
> Assignee: Robert Kanter
> Attachments: OOZIE-2343.001.patch
>
>
> Despite being not recommended, many users still run Hadoop ecosystem programs
> from the Shell Action (e.g. hive, beeline, sqoop, etc), even though we have
> Actions for those. One of the common problems we've been seeing here lately
> is that {{HADOOP_CONF_DIR}} is inherited from the NM, and is pointing to the
> NM's copy of the Hadoop *-site.xml files, which is often not well suited for
> client programs. This can lead to all kinds of weird problems, such as
> permissions errors, missing logs, etc.
> As a workaround, we've been telling users to set {{HADOOP_CONF_DIR}} to
> {{/etc/hadoop/conf}}, however, this has some disadvantages. Ideally, these
> client programs should pick up {{OOZIE_ACTION_CONF_XML}} instead. Given that
> {{OOZIE_ACTION_CONF_XML}} is an amalgamation of all of the Hadoop *-site.xml
> plus things Oozie and the user set, we can simply have the Shell Action
> copy-paste it to the correctly named *-site.xml files, and point
> {{HADOOP_CONF_DIR}} to there.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
