Re: how to configure multi-homed management server?
Hi Yiping,
The host IP in global settings is set during first time start of
cloudstack-management service which starts soon after
cloudstack-setup-management script.
CloudStack picks the default NIC as host entry in global settings. Hence,
it doesnt pick the IP that you give along with cloudstack-setupp-databases
script.
In managemnet-server log, you'll notice ConfigurationServer saved
xx.xx.xx.xx as host entry that gets added with host ip addition.
Following source snippets form code may help you understand how it assigns
host ip better:
==
1) cloudstack/server/src/com/cloud/server/ConfigurationServerImpl.java
public void persistDefaultValues() throws InternalErrorException {
.
.
String hostIpAdr = NetUtils.getDefaultHostIp();
boolean needUpdateHostIp = true;
if (hostIpAdr != null) {
Boolean devel = Boolean.valueOf(_configDao.getValue(developer));
if (devel) {
String value =
_configDao.getValue(ApiServiceConfiguration.ManagementHostIPAdr.key());
if (value != null !value.equals(localhost)) {
needUpdateHostIp = false;
}
}
if (needUpdateHostIp) {
_configDepot.createOrUpdateConfigObject(ApiServiceConfiguration.class.getSimpleName(),
ApiServiceConfiguration.ManagementHostIPAdr, hostIpAdr);
s_logger.debug(ConfigurationServer saved \ + hostIpAdr + \ as host.);
}
}
2) cloudstack/utils/src/com/cloud/utils/net/NetUtils.java
.
.
public static String getDefaultHostIp() {
if (SystemUtils.IS_OS_WINDOWS) {
Pattern pattern =
Pattern.compile(\\s*0.0.0.0\\s*0.0.0.0\\s*(\\S*)\\s*(\\S*)\\s*);
try {
Process result = Runtime.getRuntime().exec(route print -4);
BufferedReader output = new BufferedReader(new
InputStreamReader(result.getInputStream()));
String line = output.readLine();
while (line != null) {
Matcher matcher = pattern.matcher(line);
if (matcher.find()) {
return matcher.group(2);
}
line = output.readLine();
}
} catch (IOException e) {
s_logger.debug(Caught IOException, e);
}
I hope this explanation helps your understanding.
Thanks,
Praveen
On Sat, Feb 14, 2015 at 2:40 AM, Yiping Zhang yzh...@marketo.com wrote:
Update:
Now my mgr¹s eth0 is configured with IP address on the lab network, the
hostname is set to the name of lab network IP. However, after I run
cloudstack-setup-databases and cloudstack-setup-management scripts, I
still find the wrong IP address in global setting param hosts, and wrong
value for param ³management.network.cidr².
It looks like that running cloudstack-setup-management script (with
immediate start of cloudstack-management service) populates DB tables. So
my question comes down to how does cloudstack-management service
determine these configuration values ?
Thanks
Yiping
On 2/11/15, 8:21 PM, Praveen B pbprave...@gmail.com wrote:
Hi Zhang,
host parameter in the global settings will decide system VMs to
communicate to MGMT ip address on port 8250. Since you have two IP
addresses on management server, CloudStack has picked up your corporate
network IP address.
As a fix, change the host parameter to your lab network IP address and
destroy system VMs. New system VMs will point to correct mgmt IP. Let me
know how it goes.
Thanks,
Praveen
On Thu, Feb 12, 2015 at 5:39 AM, Yiping Zhang yzh...@marketo.com wrote:
Hi, all:
My CS management server has two IP addresses: one IP address on our
corporate network (for general access) and one IP address on lab
network ,
which is used at CloudStack¹s management network.
When I run cloudstack-setup-database script, I have given ³‹mshost²
option
with its IP address on lab network. However, when CPVM comes up, the
cloud service is not running . Looking at /var/log/cloud.log file on
CPVM
indicates that it is trying to connect to port 8250 of management
server¹s
IP address on corporate network instead of the IP address on lab
network.
On SSVM, its cloud service also tries to connect to port 8250 of the
wrong
MS IP address
How does systemVMs decide which IP address on the management server to
use
when starting cloud service ? How do I make systemVM¹s to use proper
interface on management server ?
Thanks
Yiping
Re: how to configure multi-homed management server?
Update: Now my mgr¹s eth0 is configured with IP address on the lab network, the hostname is set to the name of lab network IP. However, after I run cloudstack-setup-databases and cloudstack-setup-management scripts, I still find the wrong IP address in global setting param hosts, and wrong value for param ³management.network.cidr². It looks like that running cloudstack-setup-management script (with immediate start of cloudstack-management service) populates DB tables. So my question comes down to how does cloudstack-management service determine these configuration values ? Thanks Yiping On 2/11/15, 8:21 PM, Praveen B pbprave...@gmail.com wrote: Hi Zhang, host parameter in the global settings will decide system VMs to communicate to MGMT ip address on port 8250. Since you have two IP addresses on management server, CloudStack has picked up your corporate network IP address. As a fix, change the host parameter to your lab network IP address and destroy system VMs. New system VMs will point to correct mgmt IP. Let me know how it goes. Thanks, Praveen On Thu, Feb 12, 2015 at 5:39 AM, Yiping Zhang yzh...@marketo.com wrote: Hi, all: My CS management server has two IP addresses: one IP address on our corporate network (for general access) and one IP address on lab network , which is used at CloudStack¹s management network. When I run cloudstack-setup-database script, I have given ³‹mshost² option with its IP address on lab network. However, when CPVM comes up, the cloud service is not running . Looking at /var/log/cloud.log file on CPVM indicates that it is trying to connect to port 8250 of management server¹s IP address on corporate network instead of the IP address on lab network. On SSVM, its cloud service also tries to connect to port 8250 of the wrong MS IP address How does systemVMs decide which IP address on the management server to use when starting cloud service ? How do I make systemVM¹s to use proper interface on management server ? Thanks Yiping
how to configure multi-homed management server?
Hi, all: My CS management server has two IP addresses: one IP address on our corporate network (for general access) and one IP address on lab network , which is used at CloudStack’s management network. When I run cloudstack-setup-database script, I have given “—mshost” option with its IP address on lab network. However, when CPVM comes up, the cloud service is not running . Looking at /var/log/cloud.log file on CPVM indicates that it is trying to connect to port 8250 of management server’s IP address on corporate network instead of the IP address on lab network. On SSVM, its cloud service also tries to connect to port 8250 of the wrong MS IP address How does systemVMs decide which IP address on the management server to use when starting cloud service ? How do I make systemVM’s to use proper interface on management server ? Thanks Yiping
Re: how to configure multi-homed management server?
Hi, Praveen: That worked, sort of. Now both CPVM and SSVM are trying to connect to the correct IP address on my management server, but the connect failed due to some sort of SSL error, even though I have not used SSL for this CS instance at all. Following error is logged in /var/log/cloud.log file repeatedly: 2015-02-12 04:50:58,053 INFO [utils.nio.NioClient] (Agent-Selector:null) Connecting to 10.0.100.11:8250 2015-02-12 04:50:58,060 ERROR [utils.nio.NioConnection] (Agent-Selector:null) Unable to initialize the threads. java.io.IOException: SSL: Fail to init SSL! java.io.IOException: Connection reset by peer at com.cloud.utils.nio.NioClient.init(NioClient.java:84) at com.cloud.utils.nio.NioConnection.run(NioConnection.java:108) at java.lang.Thread.run(Thread.java:701) Regarding ³host² global setting, if this variable determines which IP address on mgr to use, then I wonder do I still need to use ³‹mshost ip² option when running cloudstack-setup-databases script ? I am seeing some errors in catalina.out about cluster heartbeat failures which I have not seen before. Yiping On 2/11/15, 8:21 PM, Praveen B pbprave...@gmail.com wrote: Hi Zhang, host parameter in the global settings will decide system VMs to communicate to MGMT ip address on port 8250. Since you have two IP addresses on management server, CloudStack has picked up your corporate network IP address. As a fix, change the host parameter to your lab network IP address and destroy system VMs. New system VMs will point to correct mgmt IP. Let me know how it goes. Thanks, Praveen On Thu, Feb 12, 2015 at 5:39 AM, Yiping Zhang yzh...@marketo.com wrote: Hi, all: My CS management server has two IP addresses: one IP address on our corporate network (for general access) and one IP address on lab network , which is used at CloudStack¹s management network. When I run cloudstack-setup-database script, I have given ³‹mshost² option with its IP address on lab network. However, when CPVM comes up, the cloud service is not running . Looking at /var/log/cloud.log file on CPVM indicates that it is trying to connect to port 8250 of management server¹s IP address on corporate network instead of the IP address on lab network. On SSVM, its cloud service also tries to connect to port 8250 of the wrong MS IP address How does systemVMs decide which IP address on the management server to use when starting cloud service ? How do I make systemVM¹s to use proper interface on management server ? Thanks Yiping
Re: how to configure multi-homed management server?
Are you using custom SSL certificates and domain for your system VMs? Have you deployed HTTPS on your management server? Which version of CloudStack is this? I believe you can ignore heartbeat related errors. Only thing you need to maintain is time sync between the mgmt nodes. Also, I am not sure about mshost parameter of deploy-databases script. you can try giving other ip..see that works.. Thanks, Praveen On Thu, Feb 12, 2015 at 10:42 AM, Yiping Zhang yzh...@marketo.com wrote: Hi, Praveen: That worked, sort of. Now both CPVM and SSVM are trying to connect to the correct IP address on my management server, but the connect failed due to some sort of SSL error, even though I have not used SSL for this CS instance at all. Following error is logged in /var/log/cloud.log file repeatedly: 2015-02-12 04:50:58,053 INFO [utils.nio.NioClient] (Agent-Selector:null) Connecting to 10.0.100.11:8250 2015-02-12 04:50:58,060 ERROR [utils.nio.NioConnection] (Agent-Selector:null) Unable to initialize the threads. java.io.IOException: SSL: Fail to init SSL! java.io.IOException: Connection reset by peer at com.cloud.utils.nio.NioClient.init(NioClient.java:84) at com.cloud.utils.nio.NioConnection.run(NioConnection.java:108) at java.lang.Thread.run(Thread.java:701) Regarding ³host² global setting, if this variable determines which IP address on mgr to use, then I wonder do I still need to use ³‹mshost ip² option when running cloudstack-setup-databases script ? I am seeing some errors in catalina.out about cluster heartbeat failures which I have not seen before. Yiping On 2/11/15, 8:21 PM, Praveen B pbprave...@gmail.com wrote: Hi Zhang, host parameter in the global settings will decide system VMs to communicate to MGMT ip address on port 8250. Since you have two IP addresses on management server, CloudStack has picked up your corporate network IP address. As a fix, change the host parameter to your lab network IP address and destroy system VMs. New system VMs will point to correct mgmt IP. Let me know how it goes. Thanks, Praveen On Thu, Feb 12, 2015 at 5:39 AM, Yiping Zhang yzh...@marketo.com wrote: Hi, all: My CS management server has two IP addresses: one IP address on our corporate network (for general access) and one IP address on lab network , which is used at CloudStack¹s management network. When I run cloudstack-setup-database script, I have given ³‹mshost² option with its IP address on lab network. However, when CPVM comes up, the cloud service is not running . Looking at /var/log/cloud.log file on CPVM indicates that it is trying to connect to port 8250 of management server¹s IP address on corporate network instead of the IP address on lab network. On SSVM, its cloud service also tries to connect to port 8250 of the wrong MS IP address How does systemVMs decide which IP address on the management server to use when starting cloud service ? How do I make systemVM¹s to use proper interface on management server ? Thanks Yiping
Re: how to configure multi-homed management server?
Hi, Praveen: No, not using HTTPS for management server No, not using custom SSL certs or domain This is CloudStack 4.3.2 on RHEL 6.6 x86_64 server, and XenServer 6.2 hypervisor. This CS instance was installed the same way as my previous CS instances, and I have not changed any settings regarding SSL/HTTPS etc in any of my CS instances (including current one), yet this is the first time seeing this SSL related error. The differences between this CS instance from previous installations are mainly: 1. Current management node is a VM instance running on a separate standalone XenServer 6.2 outside of CloudStack. Previous management server was a physical server and it had only one IP address. 2. This CS instance is version 4.3.2, and my previous instances are either 4.3.0 or 4.3.1. I should also mention that there are seem to be some intermittent connectivity issues between VM instances hosted on this standalone XenServer host and VM instances hosted on other xenserver hosts managed by CloudStack. Have anyone seen this behavior before? I have not been able to find any references to this error in my google searches so far. Thanks, Yiping On 2/11/15, 10:01 PM, Praveen B pbprave...@gmail.com wrote: Are you using custom SSL certificates and domain for your system VMs? Have you deployed HTTPS on your management server? Which version of CloudStack is this? I believe you can ignore heartbeat related errors. Only thing you need to maintain is time sync between the mgmt nodes. Also, I am not sure about mshost parameter of deploy-databases script. you can try giving other ip..see that works.. Thanks, Praveen On Thu, Feb 12, 2015 at 10:42 AM, Yiping Zhang yzh...@marketo.com wrote: Hi, Praveen: That worked, sort of. Now both CPVM and SSVM are trying to connect to the correct IP address on my management server, but the connect failed due to some sort of SSL error, even though I have not used SSL for this CS instance at all. Following error is logged in /var/log/cloud.log file repeatedly: 2015-02-12 04:50:58,053 INFO [utils.nio.NioClient] (Agent-Selector:null) Connecting to 10.0.100.11:8250 2015-02-12 04:50:58,060 ERROR [utils.nio.NioConnection] (Agent-Selector:null) Unable to initialize the threads. java.io.IOException: SSL: Fail to init SSL! java.io.IOException: Connection reset by peer at com.cloud.utils.nio.NioClient.init(NioClient.java:84) at com.cloud.utils.nio.NioConnection.run(NioConnection.java:108) at java.lang.Thread.run(Thread.java:701) Regarding ³host² global setting, if this variable determines which IP address on mgr to use, then I wonder do I still need to use ³‹mshost ip² option when running cloudstack-setup-databases script ? I am seeing some errors in catalina.out about cluster heartbeat failures which I have not seen before. Yiping On 2/11/15, 8:21 PM, Praveen B pbprave...@gmail.com wrote: Hi Zhang, host parameter in the global settings will decide system VMs to communicate to MGMT ip address on port 8250. Since you have two IP addresses on management server, CloudStack has picked up your corporate network IP address. As a fix, change the host parameter to your lab network IP address and destroy system VMs. New system VMs will point to correct mgmt IP. Let me know how it goes. Thanks, Praveen On Thu, Feb 12, 2015 at 5:39 AM, Yiping Zhang yzh...@marketo.com wrote: Hi, all: My CS management server has two IP addresses: one IP address on our corporate network (for general access) and one IP address on lab network , which is used at CloudStack¹s management network. When I run cloudstack-setup-database script, I have given ³‹mshost² option with its IP address on lab network. However, when CPVM comes up, the cloud service is not running . Looking at /var/log/cloud.log file on CPVM indicates that it is trying to connect to port 8250 of management server¹s IP address on corporate network instead of the IP address on lab network. On SSVM, its cloud service also tries to connect to port 8250 of the wrong MS IP address How does systemVMs decide which IP address on the management server to use when starting cloud service ? How do I make systemVM¹s to use proper interface on management server ? Thanks Yiping
Re: how to configure multi-homed management server?
Hi Zhang, host parameter in the global settings will decide system VMs to communicate to MGMT ip address on port 8250. Since you have two IP addresses on management server, CloudStack has picked up your corporate network IP address. As a fix, change the host parameter to your lab network IP address and destroy system VMs. New system VMs will point to correct mgmt IP. Let me know how it goes. Thanks, Praveen On Thu, Feb 12, 2015 at 5:39 AM, Yiping Zhang yzh...@marketo.com wrote: Hi, all: My CS management server has two IP addresses: one IP address on our corporate network (for general access) and one IP address on lab network , which is used at CloudStack’s management network. When I run cloudstack-setup-database script, I have given “—mshost” option with its IP address on lab network. However, when CPVM comes up, the cloud service is not running . Looking at /var/log/cloud.log file on CPVM indicates that it is trying to connect to port 8250 of management server’s IP address on corporate network instead of the IP address on lab network. On SSVM, its cloud service also tries to connect to port 8250 of the wrong MS IP address How does systemVMs decide which IP address on the management server to use when starting cloud service ? How do I make systemVM’s to use proper interface on management server ? Thanks Yiping
