Hi, I have the following configuration:
Cluster 1: Job Submission through Hue (hue1) to Cluster 1 (NN1 and RM1) via Oozie works. Cluster 2: Job Submission through Hue (hue2) to Cluster 2 (NN2 and RM2) via Oozie works. As Oozie is cluster agnostic - now I have tweaked my Hue code to submit job to cross cluster i.e. from Hue1 to Cluster2 (NN2 and RM2). In the workspace folder I do see job.properties with modified NN and JT (RM) Job is going to RM2 but it is failing there with the following warnings and error Warning: org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: fs.defaultFS; Ignoring. org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: fs.defaultFS; Ignoring. java.io.FileNotFoundException: File does not exist: hdfs://cluster1-nn1:8020/user/user1/.staging/job_1488573599716_0001/job.splitmetainfo org.apache.hadoop.security.token.SecretManager$InvalidToken): appattempt_1488573599716_0001_000001 not found in AMRMTokenSecretManager. Also it is trying to connect to RM1 as it shows the following message in the log INFO [main] org.apache.hadoop.yarn.client.RMProxy: Connecting to ResourceManager at RM1/xx.xx.xx.xx:8030 I have modified following in Oozie-site.xml oozie.service.ProxyUserService.proxyuser.hue.groups ==> * oozie.service.ProxyUserService.proxyuser.hue.hosts ==> * I would appreciate if someone can shed some light on how to make this work. Do I need to route job to cluster specific Oozie server? Why job.xml (sent to RM via Oozie) has Cluster-1's NN and RM information (if that is the case)? Do I require any other configuration setting to enable this cross cluster job submission work? Thanks.
