[jira] [Updated] (YARN-3749) We should make a copy of configuration when init MiniYARNCluster with multiple RMs
[
https://issues.apache.org/jira/browse/YARN-3749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xuan Gong updated YARN-3749:
Attachment: YARN-3749.7.patch
Submit the same patch to kick the Jenkins.
We should make a copy of configuration when init MiniYARNCluster with
multiple RMs
--
Key: YARN-3749
URL: https://issues.apache.org/jira/browse/YARN-3749
Project: Hadoop YARN
Issue Type: Bug
Reporter: Chun Chen
Assignee: Chun Chen
Attachments: YARN-3749.2.patch, YARN-3749.3.patch, YARN-3749.4.patch,
YARN-3749.5.patch, YARN-3749.6.patch, YARN-3749.7.patch, YARN-3749.7.patch,
YARN-3749.patch
When I was trying to write a test case for YARN-2674, I found DS client
trying to connect to both rm1 and rm2 with the same address 0.0.0.0:18032
when RM failover. But I initially set
yarn.resourcemanager.address.rm1=0.0.0.0:18032,
yarn.resourcemanager.address.rm2=0.0.0.0:28032 After digging, I found it is
in ClientRMService where the value of yarn.resourcemanager.address.rm2
changed to 0.0.0.0:18032. See the following code in ClientRMService:
{code}
clientBindAddress = conf.updateConnectAddr(YarnConfiguration.RM_BIND_HOST,
YarnConfiguration.RM_ADDRESS,
YarnConfiguration.DEFAULT_RM_ADDRESS,
server.getListenerAddress());
{code}
Since we use the same instance of configuration in rm1 and rm2 and init both
RM before we start both RM, we will change yarn.resourcemanager.ha.id to rm2
during init of rm2 and yarn.resourcemanager.ha.id will become rm2 during
starting of rm1.
So I think it is safe to make a copy of configuration when init both of the
rm.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-3749) We should make a copy of configuration when init MiniYARNCluster with multiple RMs
[
https://issues.apache.org/jira/browse/YARN-3749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chun Chen updated YARN-3749:
Attachment: YARN-3749.2.patch
Upload a new patch to fix test cases.
Lots of previous tests The HA Configuration has multiple addresses that match
local node's address. is because I forgot to set YarnConfiguration.RM_HA_ID
before starting NM.
The patch also contains 2 minor fix, changed getting conf value of
RM_SCHEDULER_ADDRESS from serviceStart to serviceInit in
ApplicationMasterService, changed duplicates setRpcAddressForRM in tests to
HAUtil.
We should make a copy of configuration when init MiniYARNCluster with
multiple RMs
--
Key: YARN-3749
URL: https://issues.apache.org/jira/browse/YARN-3749
Project: Hadoop YARN
Issue Type: Bug
Reporter: Chun Chen
Assignee: Chun Chen
Attachments: YARN-3749.2.patch, YARN-3749.patch
When I was trying to write a test case for YARN-2674, I found DS client
trying to connect to both rm1 and rm2 with the same address 0.0.0.0:18032
when RM failover. But I initially set
yarn.resourcemanager.address.rm1=0.0.0.0:18032,
yarn.resourcemanager.address.rm2=0.0.0.0:28032 After digging, I found it is
in ClientRMService where the value of yarn.resourcemanager.address.rm2
changed to 0.0.0.0:18032. See the following code in ClientRMService:
{code}
clientBindAddress = conf.updateConnectAddr(YarnConfiguration.RM_BIND_HOST,
YarnConfiguration.RM_ADDRESS,
YarnConfiguration.DEFAULT_RM_ADDRESS,
server.getListenerAddress());
{code}
Since we use the same instance of configuration in rm1 and rm2 and init both
RM before we start both RM, we will change yarn.resourcemanager.ha.id to rm2
during init of rm2 and yarn.resourcemanager.ha.id will become rm2 during
starting of rm1.
So I think it is safe to make a copy of configuration when init both of the
rm.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-3749) We should make a copy of configuration when init MiniYARNCluster with multiple RMs
[
https://issues.apache.org/jira/browse/YARN-3749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chun Chen updated YARN-3749:
Attachment: YARN-3749.7.patch
We should make a copy of configuration when init MiniYARNCluster with
multiple RMs
--
Key: YARN-3749
URL: https://issues.apache.org/jira/browse/YARN-3749
Project: Hadoop YARN
Issue Type: Bug
Reporter: Chun Chen
Assignee: Chun Chen
Attachments: YARN-3749.2.patch, YARN-3749.3.patch, YARN-3749.4.patch,
YARN-3749.5.patch, YARN-3749.6.patch, YARN-3749.7.patch, YARN-3749.patch
When I was trying to write a test case for YARN-2674, I found DS client
trying to connect to both rm1 and rm2 with the same address 0.0.0.0:18032
when RM failover. But I initially set
yarn.resourcemanager.address.rm1=0.0.0.0:18032,
yarn.resourcemanager.address.rm2=0.0.0.0:28032 After digging, I found it is
in ClientRMService where the value of yarn.resourcemanager.address.rm2
changed to 0.0.0.0:18032. See the following code in ClientRMService:
{code}
clientBindAddress = conf.updateConnectAddr(YarnConfiguration.RM_BIND_HOST,
YarnConfiguration.RM_ADDRESS,
YarnConfiguration.DEFAULT_RM_ADDRESS,
server.getListenerAddress());
{code}
Since we use the same instance of configuration in rm1 and rm2 and init both
RM before we start both RM, we will change yarn.resourcemanager.ha.id to rm2
during init of rm2 and yarn.resourcemanager.ha.id will become rm2 during
starting of rm1.
So I think it is safe to make a copy of configuration when init both of the
rm.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-3749) We should make a copy of configuration when init MiniYARNCluster with multiple RMs
[
https://issues.apache.org/jira/browse/YARN-3749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chun Chen updated YARN-3749:
Attachment: YARN-3749.4.patch
We should make a copy of configuration when init MiniYARNCluster with
multiple RMs
--
Key: YARN-3749
URL: https://issues.apache.org/jira/browse/YARN-3749
Project: Hadoop YARN
Issue Type: Bug
Reporter: Chun Chen
Assignee: Chun Chen
Attachments: YARN-3749.2.patch, YARN-3749.3.patch, YARN-3749.4.patch,
YARN-3749.patch
When I was trying to write a test case for YARN-2674, I found DS client
trying to connect to both rm1 and rm2 with the same address 0.0.0.0:18032
when RM failover. But I initially set
yarn.resourcemanager.address.rm1=0.0.0.0:18032,
yarn.resourcemanager.address.rm2=0.0.0.0:28032 After digging, I found it is
in ClientRMService where the value of yarn.resourcemanager.address.rm2
changed to 0.0.0.0:18032. See the following code in ClientRMService:
{code}
clientBindAddress = conf.updateConnectAddr(YarnConfiguration.RM_BIND_HOST,
YarnConfiguration.RM_ADDRESS,
YarnConfiguration.DEFAULT_RM_ADDRESS,
server.getListenerAddress());
{code}
Since we use the same instance of configuration in rm1 and rm2 and init both
RM before we start both RM, we will change yarn.resourcemanager.ha.id to rm2
during init of rm2 and yarn.resourcemanager.ha.id will become rm2 during
starting of rm1.
So I think it is safe to make a copy of configuration when init both of the
rm.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-3749) We should make a copy of configuration when init MiniYARNCluster with multiple RMs
[
https://issues.apache.org/jira/browse/YARN-3749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chun Chen updated YARN-3749:
Attachment: YARN-3749.5.patch
We should make a copy of configuration when init MiniYARNCluster with
multiple RMs
--
Key: YARN-3749
URL: https://issues.apache.org/jira/browse/YARN-3749
Project: Hadoop YARN
Issue Type: Bug
Reporter: Chun Chen
Assignee: Chun Chen
Attachments: YARN-3749.2.patch, YARN-3749.3.patch, YARN-3749.4.patch,
YARN-3749.5.patch, YARN-3749.patch
When I was trying to write a test case for YARN-2674, I found DS client
trying to connect to both rm1 and rm2 with the same address 0.0.0.0:18032
when RM failover. But I initially set
yarn.resourcemanager.address.rm1=0.0.0.0:18032,
yarn.resourcemanager.address.rm2=0.0.0.0:28032 After digging, I found it is
in ClientRMService where the value of yarn.resourcemanager.address.rm2
changed to 0.0.0.0:18032. See the following code in ClientRMService:
{code}
clientBindAddress = conf.updateConnectAddr(YarnConfiguration.RM_BIND_HOST,
YarnConfiguration.RM_ADDRESS,
YarnConfiguration.DEFAULT_RM_ADDRESS,
server.getListenerAddress());
{code}
Since we use the same instance of configuration in rm1 and rm2 and init both
RM before we start both RM, we will change yarn.resourcemanager.ha.id to rm2
during init of rm2 and yarn.resourcemanager.ha.id will become rm2 during
starting of rm1.
So I think it is safe to make a copy of configuration when init both of the
rm.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-3749) We should make a copy of configuration when init MiniYARNCluster with multiple RMs
[
https://issues.apache.org/jira/browse/YARN-3749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chun Chen updated YARN-3749:
Attachment: YARN-3749.3.patch
We should make a copy of configuration when init MiniYARNCluster with
multiple RMs
--
Key: YARN-3749
URL: https://issues.apache.org/jira/browse/YARN-3749
Project: Hadoop YARN
Issue Type: Bug
Reporter: Chun Chen
Assignee: Chun Chen
Attachments: YARN-3749.2.patch, YARN-3749.3.patch, YARN-3749.patch
When I was trying to write a test case for YARN-2674, I found DS client
trying to connect to both rm1 and rm2 with the same address 0.0.0.0:18032
when RM failover. But I initially set
yarn.resourcemanager.address.rm1=0.0.0.0:18032,
yarn.resourcemanager.address.rm2=0.0.0.0:28032 After digging, I found it is
in ClientRMService where the value of yarn.resourcemanager.address.rm2
changed to 0.0.0.0:18032. See the following code in ClientRMService:
{code}
clientBindAddress = conf.updateConnectAddr(YarnConfiguration.RM_BIND_HOST,
YarnConfiguration.RM_ADDRESS,
YarnConfiguration.DEFAULT_RM_ADDRESS,
server.getListenerAddress());
{code}
Since we use the same instance of configuration in rm1 and rm2 and init both
RM before we start both RM, we will change yarn.resourcemanager.ha.id to rm2
during init of rm2 and yarn.resourcemanager.ha.id will become rm2 during
starting of rm1.
So I think it is safe to make a copy of configuration when init both of the
rm.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-3749) We should make a copy of configuration when init MiniYARNCluster with multiple RMs
[
https://issues.apache.org/jira/browse/YARN-3749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chun Chen updated YARN-3749:
Attachment: YARN-3749.6.patch
We should make a copy of configuration when init MiniYARNCluster with
multiple RMs
--
Key: YARN-3749
URL: https://issues.apache.org/jira/browse/YARN-3749
Project: Hadoop YARN
Issue Type: Bug
Reporter: Chun Chen
Assignee: Chun Chen
Attachments: YARN-3749.2.patch, YARN-3749.3.patch, YARN-3749.4.patch,
YARN-3749.5.patch, YARN-3749.6.patch, YARN-3749.patch
When I was trying to write a test case for YARN-2674, I found DS client
trying to connect to both rm1 and rm2 with the same address 0.0.0.0:18032
when RM failover. But I initially set
yarn.resourcemanager.address.rm1=0.0.0.0:18032,
yarn.resourcemanager.address.rm2=0.0.0.0:28032 After digging, I found it is
in ClientRMService where the value of yarn.resourcemanager.address.rm2
changed to 0.0.0.0:18032. See the following code in ClientRMService:
{code}
clientBindAddress = conf.updateConnectAddr(YarnConfiguration.RM_BIND_HOST,
YarnConfiguration.RM_ADDRESS,
YarnConfiguration.DEFAULT_RM_ADDRESS,
server.getListenerAddress());
{code}
Since we use the same instance of configuration in rm1 and rm2 and init both
RM before we start both RM, we will change yarn.resourcemanager.ha.id to rm2
during init of rm2 and yarn.resourcemanager.ha.id will become rm2 during
starting of rm1.
So I think it is safe to make a copy of configuration when init both of the
rm.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-3749) We should make a copy of configuration when init MiniYARNCluster with multiple RMs
[ https://issues.apache.org/jira/browse/YARN-3749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chun Chen updated YARN-3749: Summary: We should make a copy of configuration when init MiniYARNCluster with multiple RMs (was: We should make a copy of config MiniYARNCluster ) We should make a copy of configuration when init MiniYARNCluster with multiple RMs -- Key: YARN-3749 URL: https://issues.apache.org/jira/browse/YARN-3749 Project: Hadoop YARN Issue Type: Bug Reporter: Chun Chen -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3749) We should make a copy of configuration when init MiniYARNCluster with multiple RMs
[
https://issues.apache.org/jira/browse/YARN-3749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chun Chen updated YARN-3749:
Description:
When I was trying to write a test case for YARN-2674, I found DS client trying
to connect to both rm1 and rm2 with the same address 0.0.0.0:18032 when RM
failover. But I initially set yarn.resourcemanager.address.rm1=0.0.0.0:18032,
yarn.resourcemanager.address.rm2=0.0.0.0:28032 After digging, I found it is in
ClientRMService where the value of yarn.resourcemanager.address.rm2 changed to
0.0.0.0:18032. See the following code in ClientRMService:
{code}
clientBindAddress = conf.updateConnectAddr(YarnConfiguration.RM_BIND_HOST,
YarnConfiguration.RM_ADDRESS,
YarnConfiguration.DEFAULT_RM_ADDRESS,
server.getListenerAddress());
{code}
Since we use the same instance of configuration in rm1 and rm2 and init both RM
before we start both RM, we will change yarn.resourcemanager.ha.id to rm2
during init of rm2 and yarn.resourcemanager.ha.id will become rm2 during
starting of rm1.
So I think it is safe to make a copy of configuration when init both of the rm.
We should make a copy of configuration when init MiniYARNCluster with
multiple RMs
--
Key: YARN-3749
URL: https://issues.apache.org/jira/browse/YARN-3749
Project: Hadoop YARN
Issue Type: Bug
Reporter: Chun Chen
When I was trying to write a test case for YARN-2674, I found DS client
trying to connect to both rm1 and rm2 with the same address 0.0.0.0:18032
when RM failover. But I initially set
yarn.resourcemanager.address.rm1=0.0.0.0:18032,
yarn.resourcemanager.address.rm2=0.0.0.0:28032 After digging, I found it is
in ClientRMService where the value of yarn.resourcemanager.address.rm2
changed to 0.0.0.0:18032. See the following code in ClientRMService:
{code}
clientBindAddress = conf.updateConnectAddr(YarnConfiguration.RM_BIND_HOST,
YarnConfiguration.RM_ADDRESS,
YarnConfiguration.DEFAULT_RM_ADDRESS,
server.getListenerAddress());
{code}
Since we use the same instance of configuration in rm1 and rm2 and init both
RM before we start both RM, we will change yarn.resourcemanager.ha.id to rm2
during init of rm2 and yarn.resourcemanager.ha.id will become rm2 during
starting of rm1.
So I think it is safe to make a copy of configuration when init both of the
rm.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-3749) We should make a copy of configuration when init MiniYARNCluster with multiple RMs
[
https://issues.apache.org/jira/browse/YARN-3749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chun Chen updated YARN-3749:
Attachment: YARN-3749.patch
We should make a copy of configuration when init MiniYARNCluster with
multiple RMs
--
Key: YARN-3749
URL: https://issues.apache.org/jira/browse/YARN-3749
Project: Hadoop YARN
Issue Type: Bug
Reporter: Chun Chen
Assignee: Chun Chen
Attachments: YARN-3749.patch
When I was trying to write a test case for YARN-2674, I found DS client
trying to connect to both rm1 and rm2 with the same address 0.0.0.0:18032
when RM failover. But I initially set
yarn.resourcemanager.address.rm1=0.0.0.0:18032,
yarn.resourcemanager.address.rm2=0.0.0.0:28032 After digging, I found it is
in ClientRMService where the value of yarn.resourcemanager.address.rm2
changed to 0.0.0.0:18032. See the following code in ClientRMService:
{code}
clientBindAddress = conf.updateConnectAddr(YarnConfiguration.RM_BIND_HOST,
YarnConfiguration.RM_ADDRESS,
YarnConfiguration.DEFAULT_RM_ADDRESS,
server.getListenerAddress());
{code}
Since we use the same instance of configuration in rm1 and rm2 and init both
RM before we start both RM, we will change yarn.resourcemanager.ha.id to rm2
during init of rm2 and yarn.resourcemanager.ha.id will become rm2 during
starting of rm1.
So I think it is safe to make a copy of configuration when init both of the
rm.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
