[jira] [Created] (YARN-3642) Hadoop2 yarn.resourcemanager.scheduler.address not loaded by RMProxy.java

2015-05-13 Thread Lee Hounshell (JIRA) 
Lee Hounshell created YARN-3642:
---

 Summary: Hadoop2 yarn.resourcemanager.scheduler.address not loaded 
by RMProxy.java
 Key: YARN-3642
 URL: https://issues.apache.org/jira/browse/YARN-3642
 Project: Hadoop YARN
  Issue Type: Bug
  Components: resourcemanager
Affects Versions: 2.7.0
 Environment: yarn-site.xml:
configuration

   property
  nameyarn.nodemanager.aux-services/name
  valuemapreduce_shuffle/value
   /property

   property
  nameyarn.nodemanager.aux-services.mapreduce.shuffle.class/name
  valueorg.apache.hadoop.mapred.ShuffleHandler/value
   /property

   property
  nameyarn.resourcemanager.hostname/name
  valueqadoop-nn001.apsalar.com/value
   /property

   property
  nameyarn.resourcemanager.scheduler.address/name
  valueqadoop-nn001.apsalar.com:8030/value
   /property

   property
  nameyarn.resourcemanager.address/name
  valueqadoop-nn001.apsalar.com:8032/value
   /property

   property
  nameyarn.resourcemanager.webap.address/name
  valueqadoop-nn001.apsalar.com:8088/value
   /property

   property
  nameyarn.resourcemanager.resource-tracker.address/name
  valueqadoop-nn001.apsalar.com:8031/value
   /property

   property
  nameyarn.resourcemanager.admin.address/name
  valueqadoop-nn001.apsalar.com:8033/value
   /property

   property
  nameyarn.log-aggregation-enable/name
  valuetrue/value
   /property

   property
  descriptionWhere to aggregate logs to./description
  nameyarn.nodemanager.remote-app-log-dir/name
  value/var/log/hadoop/apps/value
   /property

   property
  nameyarn.web-proxy.address/name
  valueqadoop-nn001.apsalar.com:8088/value
   /property

/configuration


core-site.xml:
configuration

   property
  namefs.defaultFS/name
  valuehdfs://qadoop-nn001.apsalar.com/value
   /property

   property
  namehadoop.proxyuser.hdfs.hosts/name
  value*/value
   /property

   property
  namehadoop.proxyuser.hdfs.groups/name
  value*/value
   /property

/configuration


hdfs-site.xml:
configuration

   property
  namedfs.replication/name
  value2/value
   /property

   property
  namedfs.namenode.name.dir/name
  valuefile:/hadoop/nn/value
   /property

   property
  namedfs.datanode.data.dir/name
  valuefile:/hadoop/dn/dfs/value
   /property

   property
  namedfs.http.address/name
  valueqadoop-nn001.apsalar.com:50070/value
   /property

   property
  namedfs.secondary.http.address/name
  valueqadoop-nn002.apsalar.com:50090/value
   /property

/configuration


mapred-site.xml:
configuration

   property 
  namemapred.job.tracker/name 
  valueqadoop-nn001.apsalar.com:8032/value 
   /property

   property
  namemapreduce.framework.name/name
  valueyarn/value
   /property

   property
  namemapreduce.jobhistory.address/name
  valueqadoop-nn001.apsalar.com:10020/value
  descriptionthe JobHistoryServer address./description
   /property

   property  
  namemapreduce.jobhistory.webapp.address/name  
  valueqadoop-nn001.apsalar.com:19888/value  
  descriptionthe JobHistoryServer web address/description
   /property

/configuration


hbase-site.xml:
configuration

property 
namehbase.master/name 
valueqadoop-nn001.apsalar.com:6/value 
/property 

property 
namehbase.rootdir/name 
valuehdfs://qadoop-nn001.apsalar.com:8020/hbase/value 
/property 

property 
namehbase.cluster.distributed/name 
valuetrue/value 
/property 

property
namehbase.zookeeper.property.dataDir/name
value/opt/local/zookeeper/value
/property 

property
namehbase.zookeeper.property.clientPort/name
value2181/value 
/property

property 
namehbase.zookeeper.quorum/name 
valueqadoop-nn001.apsalar.com/value 
/property 

property 
namezookeeper.session.timeout/name 
value18/value 
/property 

/configuration

Reporter: Lee Hounshell


There is an issue with Hadoop 2.7.0 when in distributed operation the datanode 
is unable to reach the yarn scheduler.  In our yarn-site.xml, we have defined 
this path to be:

   property
  nameyarn.resourcemanager.scheduler.address/name
  valueqadoop-nn001.apsalar.com:8030/value
   /property

But when running an oozie job, the problem manifests when looking at the job 
logs for the yarn container.
We see logs similar to the following showing the connection problem:

Showing 4096 bytes. Click here for full log
[main] org.apache.hadoop.http.HttpServer2: Jetty bound to port 64065
2015-05-13 17:49:33,930 INFO [main] org.mortbay.log: jetty-6.1.26
2015-05-13 17:49:33,971 INFO [main] org.mortbay.log: Extract

[jira] [Commented] (YARN-3642) Hadoop2 yarn.resourcemanager.scheduler.address not loaded by RMProxy.java

2015-05-14 Thread Lee Hounshell (JIRA) 

[ 
https://issues.apache.org/jira/browse/YARN-3642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14544459#comment-14544459
 ] 

Lee Hounshell commented on YARN-3642:
-

Here is a copy of our common /etc/hosts:

::1localhost
127.0.0.1  localhost loghost
10.1.0.220  35d47c6c-97c4-429c-a189-aeb5349eea43loghost
127.0.0.1   pgbackup
10.1.26.11  qadoop-batch.apsalar.com
10.1.26.1   qadoop-nn001.apsalar.com
10.1.26.2   qadoop-nn002.apsalar.com
10.1.26.3   qadoop-jt001.apsalar.com
10.1.26.4   qadoop-d001.apsalar.com
10.1.26.5   qadoop-d002.apsalar.com
10.1.26.6   qadoop-d003.apsalar.com
10.1.26.7   qadoop-master1a.apsalar.com
10.1.26.8   qadoop-shard1a.apsalar.com
10.1.26.9   qadoop-shard2a.apsalar.com


 Hadoop2 yarn.resourcemanager.scheduler.address not loaded by RMProxy.java
 -

 Key: YARN-3642
 URL: https://issues.apache.org/jira/browse/YARN-3642
 Project: Hadoop YARN
  Issue Type: Bug
  Components: resourcemanager
Affects Versions: 2.7.0
 Environment: yarn-site.xml:
 configuration
property
   nameyarn.nodemanager.aux-services/name
   valuemapreduce_shuffle/value
/property
property
   nameyarn.nodemanager.aux-services.mapreduce.shuffle.class/name
   valueorg.apache.hadoop.mapred.ShuffleHandler/value
/property
property
   nameyarn.resourcemanager.hostname/name
   valueqadoop-nn001.apsalar.com/value
/property
property
   nameyarn.resourcemanager.scheduler.address/name
   valueqadoop-nn001.apsalar.com:8030/value
/property
property
   nameyarn.resourcemanager.address/name
   valueqadoop-nn001.apsalar.com:8032/value
/property
property
   nameyarn.resourcemanager.webap.address/name
   valueqadoop-nn001.apsalar.com:8088/value
/property
property
   nameyarn.resourcemanager.resource-tracker.address/name
   valueqadoop-nn001.apsalar.com:8031/value
/property
property
   nameyarn.resourcemanager.admin.address/name
   valueqadoop-nn001.apsalar.com:8033/value
/property
property
   nameyarn.log-aggregation-enable/name
   valuetrue/value
/property
property
   descriptionWhere to aggregate logs to./description
   nameyarn.nodemanager.remote-app-log-dir/name
   value/var/log/hadoop/apps/value
/property
property
   nameyarn.web-proxy.address/name
   valueqadoop-nn001.apsalar.com:8088/value
/property
 /configuration
 core-site.xml:
 configuration
property
   namefs.defaultFS/name
   valuehdfs://qadoop-nn001.apsalar.com/value
/property
property
   namehadoop.proxyuser.hdfs.hosts/name
   value*/value
/property
property
   namehadoop.proxyuser.hdfs.groups/name
   value*/value
/property
 /configuration
 hdfs-site.xml:
 configuration
property
   namedfs.replication/name
   value2/value
/property
property
   namedfs.namenode.name.dir/name
   valuefile:/hadoop/nn/value
/property
property
   namedfs.datanode.data.dir/name
   valuefile:/hadoop/dn/dfs/value
/property
property
   namedfs.http.address/name
   valueqadoop-nn001.apsalar.com:50070/value
/property
property
   namedfs.secondary.http.address/name
   valueqadoop-nn002.apsalar.com:50090/value
/property
 /configuration
 mapred-site.xml:
 configuration
property 
   namemapred.job.tracker/name 
   valueqadoop-nn001.apsalar.com:8032/value 
/property
property
   namemapreduce.framework.name/name
   valueyarn/value
/property
property
   namemapreduce.jobhistory.address/name
   valueqadoop-nn001.apsalar.com:10020/value
   descriptionthe JobHistoryServer address./description
/property
property  
   namemapreduce.jobhistory.webapp.address/name  
   valueqadoop-nn001.apsalar.com:19888/value  
   descriptionthe JobHistoryServer web address/description
/property
 /configuration
 hbase-site.xml:
 configuration
 property 
 namehbase.master/name 
 valueqadoop-nn001.apsalar.com:6/value 
 /property 
 property 
 namehbase.rootdir/name 
 valuehdfs://qadoop-nn001.apsalar.com:8020/hbase/value 
 /property 
 property 
 namehbase.cluster.distributed/name 
 valuetrue/value 
 /property 
 property
 namehbase.zookeeper.property.dataDir/name
 value/opt/local/zookeeper/value
 /property 
 property
 namehbase.zookeeper.property.clientPort/name
 value2181/value 
 /property
 property 
 namehbase.zookeeper.quorum/name 
 valueqadoop-nn001.apsalar.com/value 
 /property 
 property

[jira] [Commented] (YARN-3642) Hadoop2 yarn.resourcemanager.scheduler.address not loaded by RMProxy.java

2015-05-14 Thread Lee Hounshell (JIRA) 

[ 
https://issues.apache.org/jira/browse/YARN-3642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14544450#comment-14544450
 ] 

Lee Hounshell commented on YARN-3642:
-

Yes, we have the same /etc/hosts on all machines.
We also have internal DNS with those.
The problem is not name-resolution.

Our yarn-site.conf is being read for all the other values.
only the yarn.resourcemanager.scheduler.address is not properly loaded into the 
conf object.

I won't be able to do a hostname test (in place of the hard-code IP) until 
sometime late next week.

 Hadoop2 yarn.resourcemanager.scheduler.address not loaded by RMProxy.java
 -

 Key: YARN-3642
 URL: https://issues.apache.org/jira/browse/YARN-3642
 Project: Hadoop YARN
  Issue Type: Bug
  Components: resourcemanager
Affects Versions: 2.7.0
 Environment: yarn-site.xml:
 configuration
property
   nameyarn.nodemanager.aux-services/name
   valuemapreduce_shuffle/value
/property
property
   nameyarn.nodemanager.aux-services.mapreduce.shuffle.class/name
   valueorg.apache.hadoop.mapred.ShuffleHandler/value
/property
property
   nameyarn.resourcemanager.hostname/name
   valueqadoop-nn001.apsalar.com/value
/property
property
   nameyarn.resourcemanager.scheduler.address/name
   valueqadoop-nn001.apsalar.com:8030/value
/property
property
   nameyarn.resourcemanager.address/name
   valueqadoop-nn001.apsalar.com:8032/value
/property
property
   nameyarn.resourcemanager.webap.address/name
   valueqadoop-nn001.apsalar.com:8088/value
/property
property
   nameyarn.resourcemanager.resource-tracker.address/name
   valueqadoop-nn001.apsalar.com:8031/value
/property
property
   nameyarn.resourcemanager.admin.address/name
   valueqadoop-nn001.apsalar.com:8033/value
/property
property
   nameyarn.log-aggregation-enable/name
   valuetrue/value
/property
property
   descriptionWhere to aggregate logs to./description
   nameyarn.nodemanager.remote-app-log-dir/name
   value/var/log/hadoop/apps/value
/property
property
   nameyarn.web-proxy.address/name
   valueqadoop-nn001.apsalar.com:8088/value
/property
 /configuration
 core-site.xml:
 configuration
property
   namefs.defaultFS/name
   valuehdfs://qadoop-nn001.apsalar.com/value
/property
property
   namehadoop.proxyuser.hdfs.hosts/name
   value*/value
/property
property
   namehadoop.proxyuser.hdfs.groups/name
   value*/value
/property
 /configuration
 hdfs-site.xml:
 configuration
property
   namedfs.replication/name
   value2/value
/property
property
   namedfs.namenode.name.dir/name
   valuefile:/hadoop/nn/value
/property
property
   namedfs.datanode.data.dir/name
   valuefile:/hadoop/dn/dfs/value
/property
property
   namedfs.http.address/name
   valueqadoop-nn001.apsalar.com:50070/value
/property
property
   namedfs.secondary.http.address/name
   valueqadoop-nn002.apsalar.com:50090/value
/property
 /configuration
 mapred-site.xml:
 configuration
property 
   namemapred.job.tracker/name 
   valueqadoop-nn001.apsalar.com:8032/value 
/property
property
   namemapreduce.framework.name/name
   valueyarn/value
/property
property
   namemapreduce.jobhistory.address/name
   valueqadoop-nn001.apsalar.com:10020/value
   descriptionthe JobHistoryServer address./description
/property
property  
   namemapreduce.jobhistory.webapp.address/name  
   valueqadoop-nn001.apsalar.com:19888/value  
   descriptionthe JobHistoryServer web address/description
/property
 /configuration
 hbase-site.xml:
 configuration
 property 
 namehbase.master/name 
 valueqadoop-nn001.apsalar.com:6/value 
 /property 
 property 
 namehbase.rootdir/name 
 valuehdfs://qadoop-nn001.apsalar.com:8020/hbase/value 
 /property 
 property 
 namehbase.cluster.distributed/name 
 valuetrue/value 
 /property 
 property
 namehbase.zookeeper.property.dataDir/name
 value/opt/local/zookeeper/value
 /property 
 property
 namehbase.zookeeper.property.clientPort/name
 value2181/value 
 /property
 property 
 namehbase.zookeeper.quorum/name 
 valueqadoop-nn001.apsalar.com/value 
 /property 
 property 
 namezookeeper.session.timeout/name 
 value18/value 
 /property 
 /configuration
Reporter: Lee Hounshell

 There is an issue with Hadoop 2.7.0 when in distributed operation the 
 datanode is unable to reach the yarn

[jira] [Commented] (YARN-3642) Hadoop2 yarn.resourcemanager.scheduler.address not loaded by RMProxy.java

2015-05-14 Thread Lee Hounshell (JIRA) 

[ 
https://issues.apache.org/jira/browse/YARN-3642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14544462#comment-14544462
 ] 

Lee Hounshell commented on YARN-3642:
-

Here is a copy of our yarn-site.xml:

?xml version=1.0?
!--
  Licensed under the Apache License, Version 2.0 (the License);
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an AS IS BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
--
configuration

   property
  nameyarn.nodemanager.aux-services/name
  valuemapreduce_shuffle/value
   /property

   property
  nameyarn.nodemanager.aux-services.mapreduce.shuffle.class/name
  valueorg.apache.hadoop.mapred.ShuffleHandler/value
   /property

   property
  nameyarn.resourcemanager.hostname/name
  valueqadoop-nn001.apsalar.com/value
   /property

   property
  nameyarn.resourcemanager.scheduler.address/name
  valueqadoop-nn001.apsalar.com:8030/value
   /property

   property
  nameyarn.resourcemanager.address/name
  valueqadoop-nn001.apsalar.com:8032/value
   /property

   property
  nameyarn.resourcemanager.webap.address/name
  valueqadoop-nn001.apsalar.com:8088/value
   /property

   property
  nameyarn.resourcemanager.resource-tracker.address/name
  valueqadoop-nn001.apsalar.com:8031/value
   /property

   property
  nameyarn.resourcemanager.admin.address/name
  valueqadoop-nn001.apsalar.com:8033/value
   /property

   property
  nameyarn.log-aggregation-enable/name
  valuetrue/value
   /property

   property
  descriptionWhere to aggregate logs to./description
  nameyarn.nodemanager.remote-app-log-dir/name
  value/var/log/hadoop/apps/value
   /property

   property
  nameyarn.web-proxy.address/name
  valueqadoop-nn001.apsalar.com:8088/value
   /property

/configuration


 Hadoop2 yarn.resourcemanager.scheduler.address not loaded by RMProxy.java
 -

 Key: YARN-3642
 URL: https://issues.apache.org/jira/browse/YARN-3642
 Project: Hadoop YARN
  Issue Type: Bug
  Components: resourcemanager
Affects Versions: 2.7.0
 Environment: yarn-site.xml:
 configuration
property
   nameyarn.nodemanager.aux-services/name
   valuemapreduce_shuffle/value
/property
property
   nameyarn.nodemanager.aux-services.mapreduce.shuffle.class/name
   valueorg.apache.hadoop.mapred.ShuffleHandler/value
/property
property
   nameyarn.resourcemanager.hostname/name
   valueqadoop-nn001.apsalar.com/value
/property
property
   nameyarn.resourcemanager.scheduler.address/name
   valueqadoop-nn001.apsalar.com:8030/value
/property
property
   nameyarn.resourcemanager.address/name
   valueqadoop-nn001.apsalar.com:8032/value
/property
property
   nameyarn.resourcemanager.webap.address/name
   valueqadoop-nn001.apsalar.com:8088/value
/property
property
   nameyarn.resourcemanager.resource-tracker.address/name
   valueqadoop-nn001.apsalar.com:8031/value
/property
property
   nameyarn.resourcemanager.admin.address/name
   valueqadoop-nn001.apsalar.com:8033/value
/property
property
   nameyarn.log-aggregation-enable/name
   valuetrue/value
/property
property
   descriptionWhere to aggregate logs to./description
   nameyarn.nodemanager.remote-app-log-dir/name
   value/var/log/hadoop/apps/value
/property
property
   nameyarn.web-proxy.address/name
   valueqadoop-nn001.apsalar.com:8088/value
/property
 /configuration
 core-site.xml:
 configuration
property
   namefs.defaultFS/name
   valuehdfs://qadoop-nn001.apsalar.com/value
/property
property
   namehadoop.proxyuser.hdfs.hosts/name
   value*/value
/property
property
   namehadoop.proxyuser.hdfs.groups/name
   value*/value
/property
 /configuration
 hdfs-site.xml:
 configuration
property
   namedfs.replication/name
   value2/value
/property
property
   namedfs.namenode.name.dir/name
   valuefile:/hadoop/nn/value
/property
property
   namedfs.datanode.data.dir/name
   valuefile:/hadoop/dn/dfs/value
/property
property
   namedfs.http.address/name
   valueqadoop-nn001.apsalar.com:50070/value
/property
property
   namedfs.secondary.http.address/name
   valueqadoop-nn002.apsalar.com:50090/value

[jira] [Commented] (YARN-3642) Hadoop2 yarn.resourcemanager.scheduler.address not loaded by RMProxy.java

2015-05-14 Thread Lee Hounshell (JIRA) 

[ 
https://issues.apache.org/jira/browse/YARN-3642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14544502#comment-14544502
 ] 

Lee Hounshell commented on YARN-3642:
-

Yes, we have a proper /etc/hosts
That is not the problem.

I am able to make it work by forcing the conf to have our yarn scheduler
address IP.



 Hadoop2 yarn.resourcemanager.scheduler.address not loaded by RMProxy.java
 -

 Key: YARN-3642
 URL: https://issues.apache.org/jira/browse/YARN-3642
 Project: Hadoop YARN
  Issue Type: Bug
  Components: resourcemanager
Affects Versions: 2.7.0
 Environment: yarn-site.xml:
 configuration
property
   nameyarn.nodemanager.aux-services/name
   valuemapreduce_shuffle/value
/property
property
   nameyarn.nodemanager.aux-services.mapreduce.shuffle.class/name
   valueorg.apache.hadoop.mapred.ShuffleHandler/value
/property
property
   nameyarn.resourcemanager.hostname/name
   valueqadoop-nn001.apsalar.com/value
/property
property
   nameyarn.resourcemanager.scheduler.address/name
   valueqadoop-nn001.apsalar.com:8030/value
/property
property
   nameyarn.resourcemanager.address/name
   valueqadoop-nn001.apsalar.com:8032/value
/property
property
   nameyarn.resourcemanager.webap.address/name
   valueqadoop-nn001.apsalar.com:8088/value
/property
property
   nameyarn.resourcemanager.resource-tracker.address/name
   valueqadoop-nn001.apsalar.com:8031/value
/property
property
   nameyarn.resourcemanager.admin.address/name
   valueqadoop-nn001.apsalar.com:8033/value
/property
property
   nameyarn.log-aggregation-enable/name
   valuetrue/value
/property
property
   descriptionWhere to aggregate logs to./description
   nameyarn.nodemanager.remote-app-log-dir/name
   value/var/log/hadoop/apps/value
/property
property
   nameyarn.web-proxy.address/name
   valueqadoop-nn001.apsalar.com:8088/value
/property
 /configuration
 core-site.xml:
 configuration
property
   namefs.defaultFS/name
   valuehdfs://qadoop-nn001.apsalar.com/value
/property
property
   namehadoop.proxyuser.hdfs.hosts/name
   value*/value
/property
property
   namehadoop.proxyuser.hdfs.groups/name
   value*/value
/property
 /configuration
 hdfs-site.xml:
 configuration
property
   namedfs.replication/name
   value2/value
/property
property
   namedfs.namenode.name.dir/name
   valuefile:/hadoop/nn/value
/property
property
   namedfs.datanode.data.dir/name
   valuefile:/hadoop/dn/dfs/value
/property
property
   namedfs.http.address/name
   valueqadoop-nn001.apsalar.com:50070/value
/property
property
   namedfs.secondary.http.address/name
   valueqadoop-nn002.apsalar.com:50090/value
/property
 /configuration
 mapred-site.xml:
 configuration
property 
   namemapred.job.tracker/name 
   valueqadoop-nn001.apsalar.com:8032/value 
/property
property
   namemapreduce.framework.name/name
   valueyarn/value
/property
property
   namemapreduce.jobhistory.address/name
   valueqadoop-nn001.apsalar.com:10020/value
   descriptionthe JobHistoryServer address./description
/property
property  
   namemapreduce.jobhistory.webapp.address/name  
   valueqadoop-nn001.apsalar.com:19888/value  
   descriptionthe JobHistoryServer web address/description
/property
 /configuration
 hbase-site.xml:
 configuration
 property 
 namehbase.master/name 
 valueqadoop-nn001.apsalar.com:6/value 
 /property 
 property 
 namehbase.rootdir/name 
 valuehdfs://qadoop-nn001.apsalar.com:8020/hbase/value 
 /property 
 property 
 namehbase.cluster.distributed/name 
 valuetrue/value 
 /property 
 property
 namehbase.zookeeper.property.dataDir/name
 value/opt/local/zookeeper/value
 /property 
 property
 namehbase.zookeeper.property.clientPort/name
 value2181/value 
 /property
 property 
 namehbase.zookeeper.quorum/name 
 valueqadoop-nn001.apsalar.com/value 
 /property 
 property 
 namezookeeper.session.timeout/name 
 value18/value 
 /property 
 /configuration
Reporter: Lee Hounshell

 There is an issue with Hadoop 2.7.0 when in distributed operation the 
 datanode is unable to reach the yarn scheduler.  In our yarn-site.xml, we 
 have defined this path to be:
 {code}
property
   nameyarn.resourcemanager.scheduler.address/name
   valueqadoop-nn001.apsalar.com:8030/value
/property
 {code}
 But when running

[jira] [Commented] (YARN-3642) Hadoop2 yarn.resourcemanager.scheduler.address not loaded by RMProxy.java

2015-05-15 Thread Lee Hounshell (JIRA) 

[ 
https://issues.apache.org/jira/browse/YARN-3642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14545770#comment-14545770
 ] 

Lee Hounshell commented on YARN-3642:
-

Rohith, Thank you very much for your help!  You are correct.  We are running 3 
nodemanagers.  After adding our yarn-site.xml to the CLASSPATH, (and putting 
back original code) we are able to successfully submit and run an oozie job.  
Our problem is solved.

 Hadoop2 yarn.resourcemanager.scheduler.address not loaded by RMProxy.java
 -

 Key: YARN-3642
 URL: https://issues.apache.org/jira/browse/YARN-3642
 Project: Hadoop YARN
  Issue Type: Bug
  Components: resourcemanager
Affects Versions: 2.7.0
 Environment: yarn-site.xml:
 configuration
property
   nameyarn.nodemanager.aux-services/name
   valuemapreduce_shuffle/value
/property
property
   nameyarn.nodemanager.aux-services.mapreduce.shuffle.class/name
   valueorg.apache.hadoop.mapred.ShuffleHandler/value
/property
property
   nameyarn.resourcemanager.hostname/name
   valueqadoop-nn001.apsalar.com/value
/property
property
   nameyarn.resourcemanager.scheduler.address/name
   valueqadoop-nn001.apsalar.com:8030/value
/property
property
   nameyarn.resourcemanager.address/name
   valueqadoop-nn001.apsalar.com:8032/value
/property
property
   nameyarn.resourcemanager.webap.address/name
   valueqadoop-nn001.apsalar.com:8088/value
/property
property
   nameyarn.resourcemanager.resource-tracker.address/name
   valueqadoop-nn001.apsalar.com:8031/value
/property
property
   nameyarn.resourcemanager.admin.address/name
   valueqadoop-nn001.apsalar.com:8033/value
/property
property
   nameyarn.log-aggregation-enable/name
   valuetrue/value
/property
property
   descriptionWhere to aggregate logs to./description
   nameyarn.nodemanager.remote-app-log-dir/name
   value/var/log/hadoop/apps/value
/property
property
   nameyarn.web-proxy.address/name
   valueqadoop-nn001.apsalar.com:8088/value
/property
 /configuration
 core-site.xml:
 configuration
property
   namefs.defaultFS/name
   valuehdfs://qadoop-nn001.apsalar.com/value
/property
property
   namehadoop.proxyuser.hdfs.hosts/name
   value*/value
/property
property
   namehadoop.proxyuser.hdfs.groups/name
   value*/value
/property
 /configuration
 hdfs-site.xml:
 configuration
property
   namedfs.replication/name
   value2/value
/property
property
   namedfs.namenode.name.dir/name
   valuefile:/hadoop/nn/value
/property
property
   namedfs.datanode.data.dir/name
   valuefile:/hadoop/dn/dfs/value
/property
property
   namedfs.http.address/name
   valueqadoop-nn001.apsalar.com:50070/value
/property
property
   namedfs.secondary.http.address/name
   valueqadoop-nn002.apsalar.com:50090/value
/property
 /configuration
 mapred-site.xml:
 configuration
property 
   namemapred.job.tracker/name 
   valueqadoop-nn001.apsalar.com:8032/value 
/property
property
   namemapreduce.framework.name/name
   valueyarn/value
/property
property
   namemapreduce.jobhistory.address/name
   valueqadoop-nn001.apsalar.com:10020/value
   descriptionthe JobHistoryServer address./description
/property
property  
   namemapreduce.jobhistory.webapp.address/name  
   valueqadoop-nn001.apsalar.com:19888/value  
   descriptionthe JobHistoryServer web address/description
/property
 /configuration
 hbase-site.xml:
 configuration
 property 
 namehbase.master/name 
 valueqadoop-nn001.apsalar.com:6/value 
 /property 
 property 
 namehbase.rootdir/name 
 valuehdfs://qadoop-nn001.apsalar.com:8020/hbase/value 
 /property 
 property 
 namehbase.cluster.distributed/name 
 valuetrue/value 
 /property 
 property
 namehbase.zookeeper.property.dataDir/name
 value/opt/local/zookeeper/value
 /property 
 property
 namehbase.zookeeper.property.clientPort/name
 value2181/value 
 /property
 property 
 namehbase.zookeeper.quorum/name 
 valueqadoop-nn001.apsalar.com/value 
 /property 
 property 
 namezookeeper.session.timeout/name 
 value18/value 
 /property 
 /configuration
Reporter: Lee Hounshell

 There is an issue with Hadoop 2.7.0 when in distributed operation the 
 datanode is unable to reach the yarn scheduler.  In our yarn-site.xml, we 
 have defined this path to be:
 {code}
property