Re: cloudstack-management start failed (CloudStack 4.6.1 + CentOS7)

[giraffeg forestg]

s fine for me. This is due to tomcat 6/7 change.
>
> Regards, Remi
>
>
> Sent from my iPhone
>
> > On 06 Dec 2015, at 02:33, giraffeg forestg <giraffefore...@gmail.com>
> wrote:
> >
> > Hi all.
> >
> > My environment:
> >
> > CloudStack 4.6.1 , CentOS7
> > http://packages.shapeblue.com/cloudstack/upstream/centos7/4.6/
> >
> >
> > Steps to reproduce:
> >
> > 1)clean install CloudStack 4.6.1 on CentOS7
> >
> > use http://packages.shapeblue.com/cloudstack/upstream/centos7/4.6/
> >
> > [root@acs ~]# cat /etc/redhat-release
> > CentOS Linux release 7.1.1503 (Core)
> > [root@acs ~]#
> > [root@acs ~]# rpm -qa | grep cloudstack
> > cloudstack-management-4.6.1-shapeblue0.el7.centos.x86_64
> > cloudstack-common-4.6.1-shapeblue0.el7.centos.x86_64
> > [root@acs ~]#
> >
> > 2)cloudstack-setup-databases...(snip)
> >
> > 3)cloudstack-setup-management (Failed)
> >
> > [root@acs ~]# cloudstack-setup-management
> > Starting to configure CloudStack Management Server:
> > Configure Firewall ... [OK]
> > Configure CloudStack Management Server ...[Failed]
> > Cannot find /etc/cloudstack/management/server-nonssl.xml or
> > /etc/cloudstack/management/tomcat6-nonssl.conf, https enable failed
> > Try to restore your system:
> > Restore Firewall ... [OK]
> > Restore CloudStack Management Server ...[OK]
> > [root@acs ~]#
> >
> > 4)check log (/var/log/messages)
> >
> > Dec 6 10:08:51 acs server: WARNING: Unable to load server configuration
> > from [/usr/share/cloudstack-management/conf/server.xml]
> >
> > 5)check service status (failed)
> >
> > [root@acs ~]# systemctl status cloudstack-management.service
> > cloudstack-management.service - CloudStack Management Server
> > Loaded: loaded (/usr/lib/systemd/system/cloudstack-management.service;
> > enabled)
> > Active: failed (Result: exit-code) since Sun 2015-12-06 10:08:52 JST; 11s
> > ago
> > Process: 3061 ExecStop=/usr/libexec/tomcat/server stop (code=exited,
> > status=1/FAILURE)
> > Process: 3026 ExecStart=/usr/libexec/tomcat/server start (code=exited,
> > status=0/SUCCESS)
> > Main PID: 3026 (code=exited, status=0/SUCCESS)
> >
> > Dec 06 10:08:52 acs.dom.local server[3061]: at
> > java.io.FileInputStream.(FileInputStream.java:146)
> > Dec 06 10:08:52 acs.dom.local server[3061]: at
> > org.apache.catalina.startup.Catalina.stopServer(Catalina.java:466)
> > Dec 06 10:08:52 acs.dom.local server[3061]: at
> > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > Dec 06 10:08:52 acs.dom.local server[3061]: at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> > Dec 06 10:08:52 acs.dom.local server[3061]: at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorI...va:43)
> > Dec 06 10:08:52 acs.dom.local server[3061]: at
> > java.lang.reflect.Method.invoke(Method.java:606)
> > Dec 06 10:08:52 acs.dom.local server[3061]: at
> > org.apache.catalina.startup.Bootstrap.stopServer(Bootstrap.java:370)
> > Dec 06 10:08:52 acs.dom.local server[3061]: at
> > org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:457)
> > Dec 06 10:08:52 acs.dom.local systemd[1]: cloudstack-management.service:
> > control process exited, code=exited status=1
> > Dec 06 10:08:52 acs.dom.local systemd[1]: Unit
> > cloudstack-management.service entered failed state.
> > Hint: Some lines were ellipsized, use -l to show in full.
> > [root@acs ~]#
> >
> > workaround:
> > The correct do not know whether or not , but it worked.
> >
> > 6)check file
> >
> > [root@acs ~]# ls -la /etc/cloudstack/management
> > total 132
> > drwxr-xr-x. 3 root root 4096 Dec 5 17:59 .
> > drwxr-xr-x. 3 root root 4096 Dec 5 17:53 ..
> > drwxrwx---. 3 root cloud 4096 Dec 5 17:53 Catalina
> > rw-rr-. 1 root root 8945 Dec 1 20:22 catalina.policy
> > rw-rr-. 1 root root 3794 Dec 1 20:22 catalina.properties
> > rw-rr-. 1 root root 1653 Dec 1 20:22 classpath.conf
> > rw-rr-. 1 root root 1357 Dec 1 20:22 commons-logging.properties
> > rw-r. 1 root cloud 3137 Dec 5 17:59 db.properties
> > rw-rr-. 1 root root 979 Dec 1 20:22 environment.properties
> > rw-rr-. 1 root root 927 Dec 1 20:22 java.security.ciphers
> > rw-rr-. 1 root root 8 Dec 5 17:55 key
> > rw-rr-. 1 root root 7020 Dec 1 20:22 log4j-cloud.xml
> > rw-rr-. 1 root root 6722 Dec 1 20:22 server7-nonssl.xml
> > rw-rr-. 1 root root 7251 Dec 1 20:22 server7-ssl.xml
> > rw-rr-. 1 root ro

Default route is not configured on Redundant VPC VR (tier2)

[giraffeg forestg]

Hi all.

My environment:

CloudStack 4.6.1 , CentOS7
http://packages.shapeblue.com/cloudstack/upstream/centos7/4.6/

Hypervisor CentOS7 , KVM

SystemVM
http://cloudstack.apt-get.eu/systemvm/4.6/systemvm64template-4.6.0-kvm.qcow2.bz2


Steps to reproduce:

1)Create VPC (Redundant VPC offering)

2)Create tier1 & tier2

3)Create VM Instance on tier1 & tier2

4)Check VPC VR IP Address (no problem)

root@r-9-VM:~# ip a
1: lo:  mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
2: eth0:  mtu 1500 qdisc pfifo_fast state
UP qlen 1000
link/ether 0e:00:a9:fe:00:9a brd ff:ff:ff:ff:ff:ff
inet 169.254.0.154/16 brd 169.254.255.255 scope global eth0
3: eth1:  mtu 1500 qdisc pfifo_fast state
UP qlen 1000
link/ether 06:ac:80:00:00:21 brd ff:ff:ff:ff:ff:ff
inet 10.0.1.102/24 brd 10.0.1.255 scope global eth1
inet 10.0.1.103/24 brd 10.0.1.255 scope global secondary eth1
4: eth2:  mtu 1500 qdisc pfifo_fast state
UP qlen 1000
link/ether 02:00:7f:b8:00:05 brd ff:ff:ff:ff:ff:ff
inet 172.16.0.67/24 brd 172.16.0.255 scope global eth2
inet 172.16.0.1/24 brd 172.16.0.255 scope global secondary eth2
5: eth3:  mtu 1500 qdisc pfifo_fast state
UP qlen 1000
link/ether 02:00:03:56:00:04 brd ff:ff:ff:ff:ff:ff
inet 172.16.1.25/24 brd 172.16.1.255 scope global eth3
inet 172.16.1.1/24 brd 172.16.1.255 scope global secondary eth3
root@r-9-VM:~#

root@r-10-VM:~# ip a
1: lo:  mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
2: eth0:  mtu 1500 qdisc pfifo_fast state
UP qlen 1000
link/ether 0e:00:a9:fe:00:49 brd ff:ff:ff:ff:ff:ff
inet 169.254.0.73/16 brd 169.254.255.255 scope global eth0
3: eth1:  mtu 1500 qdisc noop state DOWN qlen 1000
link/ether 06:ac:80:00:00:21 brd ff:ff:ff:ff:ff:ff
inet 10.0.1.102/24 brd 10.0.1.255 scope global eth1
inet 10.0.1.103/24 brd 10.0.1.255 scope global secondary eth1
4: eth2:  mtu 1500 qdisc pfifo_fast state
UP qlen 1000
link/ether 02:00:19:11:00:06 brd ff:ff:ff:ff:ff:ff
inet 172.16.0.233/24 brd 172.16.0.255 scope global eth2
5: eth3:  mtu 1500 qdisc pfifo_fast state
UP qlen 1000
link/ether 02:00:20:19:00:05 brd ff:ff:ff:ff:ff:ff
inet 172.16.1.231/24 brd 172.16.1.255 scope global eth3
root@r-10-VM:~#


5)Reboot VPC VR r-9-VM
6)Check VPC VR IP Address (no problem)

root@r-9-VM:~# ip a
1: lo:  mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
2: eth0:  mtu 1500 qdisc pfifo_fast state
UP qlen 1000
link/ether 0e:00:a9:fe:01:28 brd ff:ff:ff:ff:ff:ff
inet 169.254.1.40/16 brd 169.254.255.255 scope global eth0
3: eth1:  mtu 1500 qdisc noop state DOWN qlen 1000
link/ether 06:ac:80:00:00:21 brd ff:ff:ff:ff:ff:ff
inet 10.0.1.103/24 brd 10.0.1.255 scope global eth1
inet 10.0.1.102/24 brd 10.0.1.255 scope global secondary eth1
4: eth2:  mtu 1500 qdisc pfifo_fast state
UP qlen 1000
link/ether 02:00:7f:b8:00:05 brd ff:ff:ff:ff:ff:ff
inet 172.16.0.67/24 brd 172.16.0.255 scope global eth2
5: eth3:  mtu 1500 qdisc pfifo_fast state
UP qlen 1000
link/ether 02:00:03:56:00:04 brd ff:ff:ff:ff:ff:ff
inet 172.16.1.25/24 brd 172.16.1.255 scope global eth3
root@r-9-VM:~#

root@r-10-VM:~# ip a
1: lo:  mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
2: eth0:  mtu 1500 qdisc pfifo_fast state
UP qlen 1000
link/ether 0e:00:a9:fe:00:49 brd ff:ff:ff:ff:ff:ff
inet 169.254.0.73/16 brd 169.254.255.255 scope global eth0
3: eth1:  mtu 1500 qdisc pfifo_fast state
UP qlen 1000
link/ether 06:ac:80:00:00:21 brd ff:ff:ff:ff:ff:ff
inet 10.0.1.102/24 brd 10.0.1.255 scope global eth1
inet 10.0.1.103/24 brd 10.0.1.255 scope global secondary eth1
4: eth2:  mtu 1500 qdisc pfifo_fast state
UP qlen 1000
link/ether 02:00:19:11:00:06 brd ff:ff:ff:ff:ff:ff
inet 172.16.0.233/24 brd 172.16.0.255 scope global eth2
inet 172.16.0.1/24 brd 172.16.0.255 scope global secondary eth2
5: eth3:  mtu 1500 qdisc pfifo_fast state
UP qlen 1000
link/ether 02:00:20:19:00:05 brd ff:ff:ff:ff:ff:ff
inet 172.16.1.231/24 brd 172.16.1.255 scope global eth3
inet 172.16.1.1/24 brd 172.16.1.255 scope global secondary eth3
root@r-10-VM:~#

cloudstack-management start failed (CloudStack 4.6.1 + CentOS7)

[giraffeg forestg]

Hi all.

My environment:

CloudStack 4.6.1 , CentOS7
http://packages.shapeblue.com/cloudstack/upstream/centos7/4.6/


Steps to reproduce:

1)clean install CloudStack 4.6.1 on CentOS7

use http://packages.shapeblue.com/cloudstack/upstream/centos7/4.6/

[root@acs ~]# cat /etc/redhat-release
CentOS Linux release 7.1.1503 (Core)
[root@acs ~]#
[root@acs ~]# rpm -qa | grep cloudstack
cloudstack-management-4.6.1-shapeblue0.el7.centos.x86_64
cloudstack-common-4.6.1-shapeblue0.el7.centos.x86_64
[root@acs ~]#

2)cloudstack-setup-databases...(snip)

3)cloudstack-setup-management (Failed)

[root@acs ~]# cloudstack-setup-management
Starting to configure CloudStack Management Server:
Configure Firewall ... [OK]
Configure CloudStack Management Server ...[Failed]
Cannot find /etc/cloudstack/management/server-nonssl.xml or
/etc/cloudstack/management/tomcat6-nonssl.conf, https enable failed
Try to restore your system:
Restore Firewall ... [OK]
Restore CloudStack Management Server ...[OK]
[root@acs ~]#

4)check log (/var/log/messages)

Dec 6 10:08:51 acs server: WARNING: Unable to load server configuration
from [/usr/share/cloudstack-management/conf/server.xml]

5)check service status (failed)

[root@acs ~]# systemctl status cloudstack-management.service
cloudstack-management.service - CloudStack Management Server
Loaded: loaded (/usr/lib/systemd/system/cloudstack-management.service;
enabled)
Active: failed (Result: exit-code) since Sun 2015-12-06 10:08:52 JST; 11s
ago
Process: 3061 ExecStop=/usr/libexec/tomcat/server stop (code=exited,
status=1/FAILURE)
Process: 3026 ExecStart=/usr/libexec/tomcat/server start (code=exited,
status=0/SUCCESS)
Main PID: 3026 (code=exited, status=0/SUCCESS)

Dec 06 10:08:52 acs.dom.local server[3061]: at
java.io.FileInputStream.(FileInputStream.java:146)
Dec 06 10:08:52 acs.dom.local server[3061]: at
org.apache.catalina.startup.Catalina.stopServer(Catalina.java:466)
Dec 06 10:08:52 acs.dom.local server[3061]: at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
Dec 06 10:08:52 acs.dom.local server[3061]: at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
Dec 06 10:08:52 acs.dom.local server[3061]: at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorI...va:43)
Dec 06 10:08:52 acs.dom.local server[3061]: at
java.lang.reflect.Method.invoke(Method.java:606)
Dec 06 10:08:52 acs.dom.local server[3061]: at
org.apache.catalina.startup.Bootstrap.stopServer(Bootstrap.java:370)
Dec 06 10:08:52 acs.dom.local server[3061]: at
org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:457)
Dec 06 10:08:52 acs.dom.local systemd[1]: cloudstack-management.service:
control process exited, code=exited status=1
Dec 06 10:08:52 acs.dom.local systemd[1]: Unit
cloudstack-management.service entered failed state.
Hint: Some lines were ellipsized, use -l to show in full.
[root@acs ~]#

workaround:
The correct do not know whether or not , but it worked.

6)check file

[root@acs ~]# ls -la /etc/cloudstack/management
total 132
drwxr-xr-x. 3 root root 4096 Dec 5 17:59 .
drwxr-xr-x. 3 root root 4096 Dec 5 17:53 ..
drwxrwx---. 3 root cloud 4096 Dec 5 17:53 Catalina
rw-rr-. 1 root root 8945 Dec 1 20:22 catalina.policy
rw-rr-. 1 root root 3794 Dec 1 20:22 catalina.properties
rw-rr-. 1 root root 1653 Dec 1 20:22 classpath.conf
rw-rr-. 1 root root 1357 Dec 1 20:22 commons-logging.properties
rw-r. 1 root cloud 3137 Dec 5 17:59 db.properties
rw-rr-. 1 root root 979 Dec 1 20:22 environment.properties
rw-rr-. 1 root root 927 Dec 1 20:22 java.security.ciphers
rw-rr-. 1 root root 8 Dec 5 17:55 key
rw-rr-. 1 root root 7020 Dec 1 20:22 log4j-cloud.xml
rw-rr-. 1 root root 6722 Dec 1 20:22 server7-nonssl.xml
rw-rr-. 1 root root 7251 Dec 1 20:22 server7-ssl.xml
rw-rr-. 1 root root 1383 Dec 1 20:22 tomcat-users.xml
rw-rr-. 1 root root 50475 Dec 1 20:22 web.xml
[root@acs ~]

7)copy file

[root@acs ~]# cd /etc/cloudstack/management
[root@acs management]# cp -p server7-nonssl.xml server.xml

8)start cloudstack-management service

[root@acs ~]# systemctl start cloudstack-management.service

9)check service status (running)

[root@acs ~]# systemctl status cloudstack-management.service
cloudstack-management.service - CloudStack Management Server
Loaded: loaded (/usr/lib/systemd/system/cloudstack-management.service;
enabled)
Active: active (running) since Sun 2015-12-06 10:12:23 JST; 1min 0s ago
Process: 3061 ExecStop=/usr/libexec/tomcat/server stop (code=exited,
status=1/FAILURE)
Main PID: 3164 (java)
CGroup: /system.slice/cloudstack-management.service
mq3164 java -Djava.awt.headless=true -Dcom.sun.management.jmxremote=false
-Xmx2g -XX:+HeapDumpOnOutOfMemoryError -X...

Dec 06 10:13:11 acs.dom.local server[3164]: INFO
[c.c.s.ConfigurationServerImpl] (localhost-startStop-1:null) Trying to...vm
iso
Dec 06 10:13:11 acs.dom.local sudo[3231]: cloud : TTY=unknown ;
PWD=/usr/share/cloudstack-management ; USER=root ; COMMA...vm_mnt
Dec 06 10:13:11 acs.dom.local sudo[3234]: cloud : 

Redundant VPC Virtual Router's state is BACKUP & BACKUP

[giraffeg forestg]

Hi all.

My environment:

CloudStack master(2015/10/31) 4.6.0-snapshot

Hypervisor CentOS6/KVM

SystemVM
build #654 (2015/10/22 19:27:55)
http://jenkins.buildacloud.org/job/build-systemvm64-master/lastSuccessfulBuild/artifact/tools/appliance/dist/systemvm64template-master-4.6.0-kvm.qcow2.bz2


Steps of reproduce.

1)Create VPC (Redundant VPC offering)

2)Create tier

3)Create VM Instance on this tier

4)Check Redundant state (good)
   r-14-VM Redundant state:MASTER
   r-15-VM Redundant state:BACKUP

5) r-14-VM Reboot Router

6)Check Redundant state (good)
   r-14-VM Redundant state:BACKUP
   r-15-VM Redundant state:MASTER

7) r-15-VM Reboot Router

8)Check Redundant state (bad)
   r-14-VM Redundant state:BACKUP
   r-15-VM Redundant state:BACKUP

9)Check Log(r-14-VM's /var/log/messages)

Nov  1 00:46:29 r-14-VM Keepalived_vrrp[3711]:
VRRP_Instance(inside_network) sending 0 priority
Nov  1 00:47:34 r-14-VM Keepalived_vrrp[2179]: Netlink: filter function
error
Nov  1 00:47:34 r-14-VM Keepalived_healthcheckers[2178]: Netlink: filter
function error
Nov  1 00:47:53 r-14-VM Keepalived_healthcheckers[2178]: Netlink: filter
function error
Nov  1 00:47:53 r-14-VM Keepalived_vrrp[2179]: Netlink: filter function
error
Nov  1 00:47:54 r-14-VM Keepalived_healthcheckers[2178]: Netlink: filter
function error
Nov  1 00:47:54 r-14-VM Keepalived_vrrp[2179]: Netlink: filter function
error

10)Check Log(r-15-VM's /var/log/messages)

Nov  1 00:49:19 r-15-VM Keepalived_vrrp[3682]:
VRRP_Instance(inside_network) sending 0 priority
Nov  1 00:50:25 r-15-VM Keepalived_vrrp[2179]: Netlink: filter function
error
Nov  1 00:50:25 r-15-VM Keepalived_healthcheckers[2178]: Netlink: filter
function error
Nov  1 00:50:45 r-15-VM Keepalived_healthcheckers[2178]: Netlink: filter
function error
Nov  1 00:50:45 r-15-VM Keepalived_vrrp[2179]: Netlink: filter function
error
Nov  1 00:50:45 r-15-VM Keepalived_healthcheckers[2178]: Netlink: filter
function error
Nov  1 00:50:45 r-15-VM Keepalived_vrrp[2179]: Netlink: filter function
error


Reported:
https://issues.apache.org/jira/browse/CLOUDSTACK-9015


Best regards.

---
Satoru Nakaya (Japan CloudStack User Group)

CloudStack HA & One VM startup failure

[giraffeg forestg]

Hi all.

My environment:
 CloudStack 4.3.0 (CentOS6)
 XenServer 6.2 SP1


1. One host failure.
2. VM is started on another host by CloudStack HA. (VM001,VM002,VM004,VM005)
3. VM003 only startup failure.

I want to know the cause of VM003 did not start by CloudStack HA.

Resources of the host is enough.
I could start VM003 by manual operation after several hours.

Although I think the following log is concerned, this means what?

2015-10-11 20:39:45,960 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(HostReservationReleaseChecker:ctx-a058591d) Cannot release reservation,
Found VM: VM[User|VM003] Stopped but reserved on host 11

Why ?

Best regards.


log (excerpt):

2015-10-11 20:28:55,770 DEBUG [c.c.h.HighAvailabilityManagerImpl]
(AgentTaskPool-11:ctx-bc69642d) Notifying HA Mgr of to restart vm 451-VM003
2015-10-11 20:28:55,776 INFO  [c.c.h.HighAvailabilityManagerImpl]
(AgentTaskPool-11:ctx-bc69642d) Schedule vm for HA:  VM[User|VM003]
2015-10-11 20:28:55,789 INFO  [c.c.h.HighAvailabilityManagerImpl]
(HA-Worker-0:ctx-71125e79 work-139) HA on VM[User|VM003]
2015-10-11 20:28:55,806 DEBUG [c.c.h.CheckOnAgentInvestigator]
(HA-Worker-0:ctx-71125e79 work-139) Unable to reach the agent for
VM[User|VM003]: Resource [Host:11] is unreachable: Host 11: Host with
specified id is not in the right state: Down
2015-10-11 20:28:55,806 INFO  [c.c.h.HighAvailabilityManagerImpl]
(HA-Worker-0:ctx-71125e79 work-139) SimpleInvestigator found
VM[User|VM003]to be alive? null
2015-10-11 20:28:58,135 INFO  [c.c.h.HighAvailabilityManagerImpl]
(HA-Worker-0:ctx-71125e79 work-139) XenServerInvestigator found
VM[User|VM003]to be alive? null
2015-10-11 20:28:58,135 DEBUG [c.c.h.UserVmDomRInvestigator]
(HA-Worker-0:ctx-71125e79 work-139) testing if VM[User|VM003] is alive
2015-10-11 20:29:10,585 DEBUG [c.c.h.UserVmDomRInvestigator]
(HA-Worker-0:ctx-71125e79 work-139) VM[User|VM003] could not be pinged,
returning that it is unknown
2015-10-11 20:29:15,040 DEBUG [c.c.h.UserVmDomRInvestigator]
(HA-Worker-0:ctx-71125e79 work-139) VM[User|VM003] could not be pinged,
returning that it is unknown
2015-10-11 20:29:15,040 DEBUG [c.c.h.UserVmDomRInvestigator]
(HA-Worker-0:ctx-71125e79 work-139) Returning null since we're unable to
determine state of VM[User|VM003]
2015-10-11 20:29:15,040 INFO  [c.c.h.HighAvailabilityManagerImpl]
(HA-Worker-0:ctx-71125e79 work-139) PingInvestigator found VM[User|VM003]to
be alive? null
2015-10-11 20:29:15,040 DEBUG [c.c.h.ManagementIPSystemVMInvestigator]
(HA-Worker-0:ctx-71125e79 work-139) Not a System Vm, unable to determine
state of VM[User|VM003] returning null
2015-10-11 20:29:15,040 DEBUG [c.c.h.ManagementIPSystemVMInvestigator]
(HA-Worker-0:ctx-71125e79 work-139) Testing if VM[User|VM003] is alive
2015-10-11 20:29:15,041 DEBUG [c.c.h.ManagementIPSystemVMInvestigator]
(HA-Worker-0:ctx-71125e79 work-139) Unable to find a management nic, cannot
ping this system VM, unable to determine state of VM[User|VM003] returning
null
2015-10-11 20:29:15,041 INFO  [c.c.h.HighAvailabilityManagerImpl]
(HA-Worker-0:ctx-71125e79 work-139) ManagementIPSysVMInvestigator found
VM[User|VM003]to be alive? null
2015-10-11 20:29:15,041 INFO  [c.c.h.HighAvailabilityManagerImpl]
(HA-Worker-0:ctx-71125e79 work-139) KVMInvestigator found VM[User|VM003]to
be alive? null
2015-10-11 20:29:15,041 INFO  [c.c.h.HighAvailabilityManagerImpl]
(HA-Worker-0:ctx-71125e79 work-139) HypervInvestigator found
VM[User|VM003]to be alive? null
2015-10-11 20:29:15,041 INFO  [c.c.h.HighAvailabilityManagerImpl]
(HA-Worker-0:ctx-71125e79 work-139) VMwareInvestigator found
VM[User|VM003]to be alive? null
2015-10-11 20:29:17,190 WARN  [c.c.v.VirtualMachineManagerImpl]
(HA-Worker-0:ctx-71125e79 work-139) Unable to actually stop VM[User|VM003]
but continue with release because it's a force stop
2015-10-11 20:29:17,194 DEBUG [c.c.v.VirtualMachineManagerImpl]
(HA-Worker-0:ctx-71125e79 work-139) VM[User|VM003] is stopped on the host.
Proceeding to release resource held.
2015-10-11 20:29:17,205 DEBUG [c.c.v.VirtualMachineManagerImpl]
(HA-Worker-0:ctx-71125e79 work-139) Successfully released network resources
for the vm VM[User|VM003]
2015-10-11 20:29:17,205 DEBUG [c.c.v.VirtualMachineManagerImpl]
(HA-Worker-0:ctx-71125e79 work-139) Successfully released storage resources
for the vm VM[User|VM003]
2015-10-11 20:29:17,276 DEBUG [c.c.v.VirtualMachineManagerImpl]
(HA-Worker-0:ctx-71125e79 work-139) Successfully transitioned to start
state for VM[User|VM003] reservation id =
74b7e3a7-80b4-4153-a6c8-993fd67c23b4
2015-10-11 20:29:17,280 DEBUG [c.c.v.VirtualMachineManagerImpl]
(HA-Worker-0:ctx-71125e79 work-139) Trying to deploy VM, vm has dcId: 1 and
podId: 1
2015-10-11 20:29:17,280 DEBUG [c.c.v.VirtualMachineManagerImpl]
(HA-Worker-0:ctx-71125e79 work-139) Deploy avoids pods: null, clusters:
null, hosts: null
2015-10-11 20:29:17,290 DEBUG [c.c.c.CapacityManagerImpl]
(HA-Worker-0:ctx-71125e79 work-139) VM state transitted from :Starting to
Stopped with event: 

Re: CS 4.5.2 Fails to Start.

[giraffeg forestg]

Hi


CLOUDSTACK-8212 might become your help.

https://issues.apache.org/jira/browse/CLOUDSTACK-8212


1. This is resolved by commenting out:
---
[root@acs ~]# vi
/usr/share/cloudstack-management/setup/db/schema-421to430.sql

 :

#realhostip changes, before changing table and adding default value
# UPDATE `cloud`.`configuration` SET value = CONCAT("*.",(SELECT
`temptable`.`value` FROM (SELECT * FROM `cloud`.`configuration` WHERE
`name`="consoleproxy.u
rl.domain") AS `temptable` WHERE
`temptable`.`name`="consoleproxy.url.domain")) WHERE
`name`="consoleproxy.url.domain";
# UPDATE `cloud`.`configuration` SET `value` = CONCAT("*.",(SELECT
`temptable`.`value` FROM (SELECT * FROM `cloud`.`configuration` WHERE
`name`="secstorage.ssl.cert.domain") AS `temptable` WHERE
`temptable`.`name`="secstorage.ssl.cert.domain")) WHERE
`name`="secstorage.ssl.cert.domain";
---

and,

2. stop cloudsack-management & cloudsack-usage & mariadb
3. initialize database
4. cloudstack-setup-databases
5. cloudstack-setup-management


I also the same problem has occurred.

My environment:
 CloudStack 4.5.2 (
http://packages.shapeblue.com/cloudstack/upstream/centos7/4.5)
 CentOS 7.1.1503
 mariadb-5.5.44-1



---
Satoru Nakaya

Japan CloudStack User Group
http://cloudstack.jp/


2015-09-11 23:11 GMT+09:00 Keerthiraja SJ :

> I really face this issue when I install on MariaDB. If I do the same on
> MySQL this pass it.
>
> On Fri, Sep 11, 2015 at 7:09 PM, Keerthiraja SJ 
> wrote:
>
> > Hi All,
> >
> > Today I tried to install the CS 4.5.2 on fresh box where I could able to
> > successfully deploy
> > the cloudstack-setup-databases.
> >
> > After I tried number of times the fresh installation I could see the same
> > issue been reproducing it.
> >
> > While stating the cloudstack servers I could see below error constantly.
> >
> >
> >
> >
> > *ERROR*ERROR [c.c.u.d.ScriptRunner] (main:null) Error executing: UPDATE
> > `cloud`.`configuration` SET value = CONCAT("*.",(SELECT
> `temptable`.`value`
> > FROM (SELECT * FROM `cloud`.`configuration` WHERE
> > `name`="consoleproxy.url.domain") AS `temptable` WHERE
> > `temptable`.`name`="consoleproxy.url.domain")) WHERE
> > `name`="consoleproxy.url.domain"
> > ERROR [c.c.u.d.ScriptRunner] (main:null)
> > com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications
> > link failure
> >
> > The last packet successfully received from the server was 25 milliseconds
> > ago.  The last packet sent successfully to the server was 25 milliseconds
> > ago.
> > ERROR [c.c.u.DatabaseUpgradeChecker] (main:null) Unable to execute
> upgrade
> > script: /usr/share/cloudstack-management/setup/db/schema-421to430.sql
> > com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications
> > link failure
> >
> >
> > Caused by: java.net.SocketException: Broken pipe
> > ... 61 more
> > ERROR [c.c.u.DatabaseUpgradeChecker] (main:null) Unable to upgrade the
> > database
> > com.cloud.utils.exception.CloudRuntimeException: Unable to execute
> upgrade
> > script: /usr/share/cloudstack-management/setup/db/schema-421to430.sql
> > at
> >
> com.cloud.upgrade.DatabaseUpgradeChecker.runScript(DatabaseUpgradeChecker.java:311)
> >
> > ERROR [c.c.u.d.DbUtil] (main:null) RELEASE_LOCK() throws exception
> > com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications
> > link failure
> >
> >
> > I did this test number of times by allocating fresh box.
> >
> > Thanks,
> > Keerthi
> >
>

Re: Couldn't change the nic driver of router-vm from E1000 to Vmxnet3

[giraffeg forestg]

Hi all.

Reply to myself.

This approach was successful.

API(addResourceDetail)
http://mail-archives.apache.org/mod_mbox/cloudstack-users/201409.mbox/%3c54213b18.7020...@gmail.com%3E


Perhaps we don't need to set global configuration ?

Best regards.

---
Satoru Nakaya (Japan CloudStack User Group)

2015-05-22 14:25 GMT+09:00 giraffeg forestg giraffefore...@gmail.com:

 Hi all.

 environment:

  management: Apache CloudStack 4.3.2 / CentOS 6.5 64bit
  host: VMware vSphere 5.1 (vNetwork Standard Switch)

 I want to change the nic driver of router-vm from E1000 to Vmxnet3.

 1.set the global configuration

  vmware.systemvm.nic.device.type : Vmxnet3

 2.management-server restart

 3.router-vm stop - start

   result : nic driver not changed.

 4.router-vm delete - create

   result : nic driver not changed.

 Please tell me a way of the changing.

 https://issues.apache.org/jira/browse/CLOUDSTACK-8503

 Best regards.



 ---
 Satoru Nakaya (Japan CloudStack User Group)

 This is new mascot character of jcsug.
 http://cloudstack.jp/wp-content/uploads/2015/05/jcsug-m.png
 illustration by wogura

Couldn't change the nic driver of router-vm from E1000 to Vmxnet3

[giraffeg forestg]

Hi all.

environment:

 management: Apache CloudStack 4.3.2 / CentOS 6.5 64bit
 host: VMware vSphere 5.1 (vNetwork Standard Switch)

I want to change the nic driver of router-vm from E1000 to Vmxnet3.

1.set the global configuration

 vmware.systemvm.nic.device.type : Vmxnet3

2.management-server restart

3.router-vm stop - start

  result : nic driver not changed.

4.router-vm delete - create

  result : nic driver not changed.

Please tell me a way of the changing.

https://issues.apache.org/jira/browse/CLOUDSTACK-8503

Best regards.



---
Satoru Nakaya (Japan CloudStack User Group)

This is new mascot character of jcsug.
http://cloudstack.jp/wp-content/uploads/2015/05/jcsug-m.png
illustration by wogura

Re: Cloudstack 4.4.3 RPMs

[giraffeg forestg]

Hi Dave

You can use ShapeBule repository.
(upstream 4.4 : Builds from official Apache CloudStack releases)

http://www.shapeblue.com/packages/


Satoru Nakaya

2015-04-29 1:06 GMT+09:00 Dave Garbus davegar...@spiceworks.com:

 Hey all,

 Does anyone know when the RPMs will be available for the 4.4.3 release?

 Thanks,
 Dave

Re: Please tell me about the attach of volume

[giraffeg forestg]

Mr. Koushik Das

Thank you for reply.

I did not find a change log of specification changes in git.

Thanks. :)

--
Satoru Nakaya ( @giraffeforestg )
Japan CloudStack User Group

2015-03-02 18:28 GMT+09:00 Koushik Das koushik@citrix.com:

 Based on the DB dump your analysis looks correct.

 -Original Message-
 From: giraffeg forestg [mailto:giraffefore...@gmail.com]
 Sent: Monday, 2 March 2015 14:46
 To: users@cloudstack.apache.org
 Subject: Please tell me about the attach of volume

 Hi All

 Please tell me about the attach of volume.
 The manual has been described below.

 

 http://docs.cloudstack.apache.org/projects/cloudstack-administration/en/4.4/storage.html
 Working With Volumes
 note

 CloudStack supports attaching up to 13 data disks to a VM on XenServer
 hypervisor versions 6.0 and above.
 For the VMs on other hypervisor types, the data disk limit is 6.
 

 The Manual is wrong.
 And the following is correct ?

 The specification has changed ?


 CloudStack supports attaching up to 13 data disks to a VM on the following
 type of hypervisor.
  1. XenServer hypervisor versions 6.0 and above.
  2. VMware versions 5.0 and above.

 For the VMs on other hypervisor types, the data disk limit is 6.


 CloudStack Version 4.3.2

 mysql select hypervisor_type,hypervisor_version,max_data_volumes_limit
 from cloud.hypervisor_capabilities;
 +-+++
 | hypervisor_type | hypervisor_version | max_data_volumes_limit |
 +-+++
 | XenServer   | default|  6 |
 | XenServer   | XCP 1.0|  6 |
 | XenServer   | 5.6|  6 |
 | XenServer   | 5.6 FP1|  6 |
 | XenServer   | 5.6 SP2|  6 |
 | XenServer   | 6.0| 13 |
 | XenServer   | 6.0.2  | 13 |
 | VMware  | default| 13 |
 | VMware  | 4.0| 13 |
 | VMware  | 4.1| 13 |
 | VMware  | 5.0| 13 |
 | KVM | default|  6 |
 | Ovm | default|  6 |
 | Ovm | 2.3|  6 |
 | XenServer   | 6.1.0  | 13 |
 | XenServer   | 6.2.0  | 13 |
 | VMware  | 5.1| 13 |
 | LXC | default|  6 |
 | Hyperv  | 6.2| 64 |
 | VMware  | 5.5| 13 |
 +-+++
 20 rows in set (0.23 sec)

 mysql



 Thanks

 --
 Satoru Nakaya ( @giraffeforestg )
 Japan CloudStack User Group

Please tell me about the attach of volume

[giraffeg forestg]

Hi All

Please tell me about the attach of volume.
The manual has been described below.


http://docs.cloudstack.apache.org/projects/cloudstack-administration/en/4.4/storage.html
Working With Volumes
note

CloudStack supports attaching up to 13 data disks to a VM on XenServer
hypervisor versions 6.0 and above.
For the VMs on other hypervisor types, the data disk limit is 6.


The Manual is wrong.
And the following is correct ?

The specification has changed ?


CloudStack supports attaching up to 13 data disks to a VM on the following
type of hypervisor.
 1. XenServer hypervisor versions 6.0 and above.
 2. VMware versions 5.0 and above.

For the VMs on other hypervisor types, the data disk limit is 6.


CloudStack Version 4.3.2

mysql select hypervisor_type,hypervisor_version,max_data_volumes_limit
from cloud.hypervisor_capabilities;
+-+++
| hypervisor_type | hypervisor_version | max_data_volumes_limit |
+-+++
| XenServer   | default|  6 |
| XenServer   | XCP 1.0|  6 |
| XenServer   | 5.6|  6 |
| XenServer   | 5.6 FP1|  6 |
| XenServer   | 5.6 SP2|  6 |
| XenServer   | 6.0| 13 |
| XenServer   | 6.0.2  | 13 |
| VMware  | default| 13 |
| VMware  | 4.0| 13 |
| VMware  | 4.1| 13 |
| VMware  | 5.0| 13 |
| KVM | default|  6 |
| Ovm | default|  6 |
| Ovm | 2.3|  6 |
| XenServer   | 6.1.0  | 13 |
| XenServer   | 6.2.0  | 13 |
| VMware  | 5.1| 13 |
| LXC | default|  6 |
| Hyperv  | 6.2| 64 |
| VMware  | 5.5| 13 |
+-+++
20 rows in set (0.23 sec)

mysql



Thanks

--
Satoru Nakaya ( @giraffeforestg )
Japan CloudStack User Group

The RPM file of version 4.3.2 will not be provided in the repository ?

[giraffeg forestg]

Hi All

CloudStack 4.3.2 I think has been released .
http://cloudstack-release-notes.readthedocs.org/en/4.3.2/

However , RPM file can not be found in the repository .
http://cloudstack.apt-get.eu/rhel/4.3/

The RPM file of version 4.3.2 will not be provided in the repository ?

Thanks

--
Satoru Nakaya

Performance degrades when I use the StaticNAT in Virtual Router. Is there a way to speed up?

[giraffeg forestg]

Hi All

Performance degrades when I use the StaticNAT in Virtual Router. Is there a
way to speed up?

Environment:
 Apache CloudStack 4.3.0
 SystemVM:systemvm64template-2014-01-14-master-xen.vhd
 HyperVisor:XenServer 6.2SP1
 Network:1Gbps (Catalyst3550-12G)

 System Offering:
  CPU:2000 MHz
  Memory:2048 MB
  Network Rate:1 MB

 Network Offering:
  Network Rate:1 MB


(Case-1, Direct Access) Good
PhysicalServer---VM Instance1
# ./nuttcp 192.168.11.2
 1122.3928 MB /  10.07 sec =  935.0451 Mbps 4 %TX 59 %RX 0 retrans 0.93
msRTT


(Case-2, VirtualRouter SourceNAT) Good
VM Instance2---VirtualRouter---PhysicalServer
# ./nuttcp 192.168.11.248
 1123.1641 MB /  10.03 sec =  938.9331 Mbps 12 %TX 15 %RX 0 retrans 3.11
msRTT


(Case-3, VirtualRouter StaticNAT) Not Good
PhysicalServer---VirtualRouter---VM Instance2
# ./nuttcp 192.168.11.173
  738. MB /  10.01 sec =  618.6374 Mbps 2 %TX 48 %RX 0 retrans 2.84
msRTT


Thanks.

--
Satoru Nakaya

Re: Performance degrades when I use the StaticNAT in Virtual Router. Is there a way to speed up?

[giraffeg forestg]

Thank you for advice. :)

# Isolation based on Security Groups in Advanced Zone
#
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Security+Groups+Isolation+in+Advanced+Zone

However, I'm want to improve the speed of StaticNAT in Virtual Router.

Trial-1:linux kernel parameter tuning of virural router

 /etc/sysctl.conf

 net.core.rmem_max = 16777216
 net.core.wmem_max = 16777216
 net.ipv4.tcp_rmem = 4096 87380 16777216
 net.ipv4.tcp_wmem = 4096 65536 16777216

 - Did not improve performance.  :(


Trial-2:disabling TSO of virural router

 /ethtool -K eth0 tso off
 /ethtool -K eth2 tso off

 - Did not improve performance.  :(


Thanks.

--
Satoru Nakaya



2014-06-24 1:46 GMT+09:00 Nux! n...@li.nux.ro:

 Depending on your circumstances, you could move to an Advanced Zone with
 Security Groups, this way traffic will bypass the Virtual Router and go
 directly to the VMs.

 HTH
 Lucian


 --
 Sent from the Delta quadrant using Borg technology!

 Nux!
 www.nux.ro


 - Original Message -
 From: giraffeg forestg giraffefore...@gmail.com
 To: users@cloudstack.apache.org
 Sent: Monday, 23 June, 2014 4:51:49 PM
 Subject: Performance degrades when I use the StaticNAT in Virtual Router.
 Is there a way to speed up?

 Hi All

 Performance degrades when I use the StaticNAT in Virtual Router. Is there a
 way to speed up?

 Environment:
  Apache CloudStack 4.3.0
  SystemVM:systemvm64template-2014-01-14-master-xen.vhd
  HyperVisor:XenServer 6.2SP1
  Network:1Gbps (Catalyst3550-12G)

  System Offering:
   CPU:2000 MHz
   Memory:2048 MB
   Network Rate:1 MB

  Network Offering:
   Network Rate:1 MB


 (Case-1, Direct Access) Good
 PhysicalServer---VM Instance1
 # ./nuttcp 192.168.11.2
  1122.3928 MB /  10.07 sec =  935.0451 Mbps 4 %TX 59 %RX 0 retrans 0.93
 msRTT


 (Case-2, VirtualRouter SourceNAT) Good
 VM Instance2---VirtualRouter---PhysicalServer
 # ./nuttcp 192.168.11.248
  1123.1641 MB /  10.03 sec =  938.9331 Mbps 12 %TX 15 %RX 0 retrans 3.11
 msRTT


 (Case-3, VirtualRouter StaticNAT) Not Good
 PhysicalServer---VirtualRouter---VM Instance2
 # ./nuttcp 192.168.11.173
   738. MB /  10.01 sec =  618.6374 Mbps 2 %TX 48 %RX 0 retrans 2.84
 msRTT


 Thanks.

 --
 Satoru Nakaya