Arpit, Yes, I have copied the hadoop cluster config under oozie/conf/hadoop-conf-cluster-1 as explained in the JIRA. However, I need to add the following line (correction to my previous email) in the oozie/conf/hadoop-conf-cluster-1/mapred-site.xml:
<property> <name>mapreduce.job.hdfs-servers</name> <value>webhdfs://gridAnn01.grid.linkedin.com,webhdfs://girdBnn01.grid.linkedin.com</value> </property> this enabled gridB to do webhdfs calls to gridA and vice-versa. In the absence, i get authentication errors. It seems Oozie needs to get tokens for both the clusters before it can kick off the Falcon job that does the distcp. Thanks Venkat On Monday, July 14, 2014 9:17 AM, Arpit Gupta <[email protected]> wrote: hmm i dont recall having to set that. However take a look at the first comment here https://issues.apache.org/jira/browse/FALCON-389 I had to make sure NN and RM properties were set but that was when we had to do hcat replication. On Sun, Jul 13, 2014 at 10:28 PM, Venkat R <[email protected]> wrote: > After much of debugging, we had to add the following property to oozie's > hadoop confs in order to obtain kerberos tokens between secure hadoop > clusters. May be this can be added to the generated oozie job? > > <property> <name>oozie.launcher.mapreduce.job.hdfs-servers</name> > <value>${nameNode1},${nameNode2}</value> > </property> > > Venkat > > > On Friday, July 11, 2014 11:34 AM, Venkat R <[email protected]> > wrote: > > > > Venkatesh, > > * Switched to using webhdfs > * Using Hadoop-2/Yarn security enables > * both source and target oozie-site.xml points to oozie/conf/hadoop-conf > of all the clusters it talkes to. > > Venkat > -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
