KVM advanced zone without vLAN
Hi, How can I setup KVM Advanced zone without vLAN, I have all server node two interface like LAN & WAN. Please give me any idea for THIS. 2021-02-26 11:09:33,937 WARN [c.c.h.k.d.LibvirtServerDiscoverer] (qtp1026871825-329:ctx-f992bc1f ctx-f4dd472f) (logid:5ba8097b) can't setup agent, due to com.cloud.utils.exception.CloudRuntimeException: Failed to setup keystore on the KVM host: 188.214.105.15 - Failed to setup keystore on the KVM host: 188.214.105.15cat: /etc/cloudstack/agent/cloud.csr: No such file or directorychmod: cannot access ‘/etc/cloudstack/agent/cloud.csr’: No such file or directory Thanks. alamin
RE: Cloudstack 4.3.2 with KVM system template from shapeblue not supported by qemu version
Thanks Rohit Andrija, I hereby confirm that the command sent by Andrija worked fine and the template spinned on Centos 6.6, after running the following command from Ubuntu 14 qemu-img convert -f qcow2 systemvm64template-4.3-kvm.qcow2 -O qcow2 -o compat=0.10 systemvm64template-4.3-kvm.qcowold -Original Message- From: Rohit Yadav [mailto:rohit.ya...@shapeblue.com] Sent: Wednesday, May 27, 2015 10:45 PM To: us...@cloudstack.apache.org Cc: dev@cloudstack.apache.org Subject: Re: Cloudstack 4.3.2 with KVM system template from shapeblue not supported by qemu version I’ll fix our 4.3 systemvm template to be compatible with older versions. This has been fixed for 4.4 and 4.5. On 26-May-2015, at 10:30 am, Andrija Panic andrija.pa...@gmail.com wrote: Just convert the template (use some newer ubuntu version of qemu-img) to qcow2 version 2... They provide qcow2 version 3, you need version 2 for centos something like (but please google and confirm): qemu-img convert -f qcow2 /var/lib/libvirt/images/qcow2v3.img -O qcow2 -o compat=0.10 /var/lib/libvirt/images/qcow2v2.img On 26 May 2015 at 05:09, Amin Samir amin.sa...@opencloud.net.au wrote: Hello, We are using Cloudstack 4.3.2 from shapeblue repo, along with KVM template dated 2015-01-28, our environment is as follows: 1) Cloud stack management server Centos 6.6, with Cloudstack 4.3.2 2) Centos 6.6 KVM with the cloudstack-agent installed from shapeblue repo 3) FreeNAS 9.3 for NFS primary secondary storage. 4) Advanced zone We have configured cloubr0 on eth0 in a single physical network, modified the labels during the zone launch to match the bridge names on the host. However System vm cannot launch due the following error in the agent: uses a qcow2 feature which is not supported by this qemu version: QCOW version 3 Anyone please help with this issue, shall we use another system template. Thanks in advance. -- Andrija Panić Regards, Rohit Yadav Software Architect, ShapeBlue M. +91 88 262 30892 | rohit.ya...@shapeblue.com Blog: bhaisaab.org | Twitter: @_bhaisaab Find out more about ShapeBlue and our range of CloudStack related services IaaS Cloud Design Buildhttp://shapeblue.com/iaas-cloud-design-and-build// CSForge – rapid IaaS deployment frameworkhttp://shapeblue.com/csforge/ CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/ CloudStack Software Engineeringhttp://shapeblue.com/cloudstack-software-engineering/ CloudStack Infrastructure Supporthttp://shapeblue.com/cloudstack-infrastructure-support/ CloudStack Bootcamp Training Courseshttp://shapeblue.com/cloudstack-training/ This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
RE: Cloudstack 4.3.2 with KVM system template from shapeblue not supported by qemu version
Thanks very much Andrija will try and feedback with results. -Original Message- From: Andrija Panic [mailto:andrija.pa...@gmail.com] Sent: Tuesday, May 26, 2015 4:30 PM To: dev@cloudstack.apache.org Cc: us...@cloudstack.apache.org Subject: Re: Cloudstack 4.3.2 with KVM system template from shapeblue not supported by qemu version Just convert the template (use some newer ubuntu version of qemu-img) to qcow2 version 2... They provide qcow2 version 3, you need version 2 for centos something like (but please google and confirm): qemu-img convert -f qcow2 /var/lib/libvirt/images/qcow2v3.img -O qcow2 -o compat=0.10 /var/lib/libvirt/images/qcow2v2.img On 26 May 2015 at 05:09, Amin Samir amin.sa...@opencloud.net.au wrote: Hello, We are using Cloudstack 4.3.2 from shapeblue repo, along with KVM template dated 2015-01-28, our environment is as follows: 1) Cloud stack management server Centos 6.6, with Cloudstack 4.3.2 2) Centos 6.6 KVM with the cloudstack-agent installed from shapeblue repo 3) FreeNAS 9.3 for NFS primary secondary storage. 4) Advanced zone We have configured cloubr0 on eth0 in a single physical network, modified the labels during the zone launch to match the bridge names on the host. However System vm cannot launch due the following error in the agent: uses a qcow2 feature which is not supported by this qemu version: QCOW version 3 Anyone please help with this issue, shall we use another system template. Thanks in advance. -- Andrija Panić
Cloudstack 4.3.2 with KVM system template from shapeblue not supported by qemu version
Hello, We are using Cloudstack 4.3.2 from shapeblue repo, along with KVM template dated 2015-01-28, our environment is as follows: 1) Cloud stack management server Centos 6.6, with Cloudstack 4.3.2 2) Centos 6.6 KVM with the cloudstack-agent installed from shapeblue repo 3) FreeNAS 9.3 for NFS primary secondary storage. 4) Advanced zone We have configured cloubr0 on eth0 in a single physical network, modified the labels during the zone launch to match the bridge names on the host. However System vm cannot launch due the following error in the agent: uses a qcow2 feature which is not supported by this qemu version: QCOW version 3 Anyone please help with this issue, shall we use another system template. Thanks in advance.
Upgrade system vm template without upgrading management server
Hello, We need to upgrade our system vm without updating the Cloud Stack management server, we are planning to use the latest from shapeblue, our current environment is as follows: * Cloud Stack version 4.3.1, installed on CentOs 6.6 * Xen Server 6.2 patched till the latest as of 18th of December 2014 * Three Zones, two advanced and one basic Zone. Cannot find in the documentation anything related to this step except only when upgrading to a newer CS version. Can anyone please help? Thanks in advance.
RE: Upgrade system vm template without upgrading management server
Thanks very much abhi -Original Message- From: Abhinandan Prateek [mailto:abhinandan.prat...@shapeblue.com] Sent: Thursday, 18 December 2014 2:11 PM To: us...@cloudstack.apache.org Cc: dev@cloudstack.apache.org Subject: Re: Upgrade system vm template without upgrading management server There is an old thread on updating system VM templates independently here: http://mail-archives.apache.org/mod_mbox/cloudstack-users/201404.mbox/%3ccf7d4a6e.3bb21%25nitin.me...@citrix.com%3E -abhi On 18-Dec-2014, at 11:06 am, Amin a...@opencloud.net.au wrote: Hello, We need to upgrade our system vm without updating the Cloud Stack management server, we are planning to use the latest from shapeblue, our current environment is as follows: * Cloud Stack version 4.3.1, installed on CentOs 6.6 * Xen Server 6.2 patched till the latest as of 18th of December 2014 * Three Zones, two advanced and one basic Zone. Cannot find in the documentation anything related to this step except only when upgrading to a newer CS version. Can anyone please help? Thanks in advance. Find out more about ShapeBlue and our range of CloudStack related services IaaS Cloud Design Buildhttp://shapeblue.com/iaas-cloud-design-and-build// CSForge – rapid IaaS deployment frameworkhttp://shapeblue.com/csforge/ CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/ CloudStack Software Engineeringhttp://shapeblue.com/cloudstack-software-engineering/ CloudStack Infrastructure Supporthttp://shapeblue.com/cloudstack-infrastructure-support/ CloudStack Bootcamp Training Courseshttp://shapeblue.com/cloudstack-training/ This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
Re: CloudStack Quality Process
Hi Steve, I would like to be part of this too. I am new to CloudStack development but have good experience managing quality processes and CI in distributed teams. Amin Alid CTO, Safe Swiss Cloud AG skype: aminalid +41 44 586 0064 +41 43 541 5704 safeswisscloud.ch anolim.com On 12/01/2014 10:12 PM, Steve Wilson wrote: Hi Everyone, It was great to get to see a number of you at the recent CCC in Budapest. While I was there, I got to meet face to face with individuals working for several companies that have a real stake in the commercial success of the CloudStack project. After joining Citrix (and becoming involved in CloudStack) about a year ago, I’ve come to believe that we need to do more to mature our quality practices around this codebase. We all like to say #cloudstackworks (and it’s true), but this is a massive codebase that’s used in the most demanding situations. We have large telecommunications companies and enterprises who are betting their businesses on this software. It has to be great! There has been quite a bit of discussion on the mailing list in recent months about how we improve in this area. There is plenty of passion, but we haven’t made enough concrete progress as a community. In my discussions with key contributors as CCC, there was general agreement that the DEV list isn’t a good forum for hashing out these kinds of things. Email is too low-bandwidth and too impersonal. At CCC, I discussed with several people the idea that we commission a small sub team to go hash out a proposal for how we handle the following topics within the ACS community (which can then be brought back to the larger community for ratification): * Continuous integration and test automation * Gating of commits * Overall commit workflow We are looking for volunteers to commit to being part of this team. This would imply a serious commitment. We don’t want hangers on or observers. This will entail real work and late night meetings. We’re looking for people who are serious contributors to the codebase. From Citrix, David Nalley and Animesh Chaturvedi have booth told me they’re willing to commit to this project. They’ve both managed ACS releases and have a really good view into the current process — and I know both are passionate about improving our process. From my CCC discussions, I believe there are individuals from Schuberg Philis, Shape Blue and Cloud Ops who are willing to commit to this process. If you are willing to be part of this team to drive forward our community, please reply here. Thanks, -Steve Steve Wilson VP Product Unit Manager Cloud Software Citrix @virtualsteve
Provisioning of Windows using baremetal offering
Hello, Has anyone used the baremetal offering to provision windows operating system? the documentation states the baremetal offering is supported only for CentOs and Ubuntu (kickstart files) Thanks in advance for your response.
RE: Using console VM's without realhostip.com domain name
Thanks Wido, done so. -Original Message- From: Wido den Hollander [mailto:w...@widodh.nl] Sent: Friday, 10 October 2014 4:53 PM To: dev@cloudstack.apache.org Subject: Re: Using console VM's without realhostip.com domain name Hi, Go to the global settings and empty the console proxy and secondary storage domain settings. Those values should say realhostip.com now. Restart the management server and you're done. That will disable SSL. Wido On 10/10/2014 05:22 AM, Amin wrote: Hello, Can anyone please advise how to use cloudstack 4.3.1 (upgraded version from 2.2.14), how to use the console VM after deprecation of realhostip.com, we are not using SSL certificates. And we don't have domain name. Thanks in advance
Using console VM's without realhostip.com domain name
Hello, Can anyone please advise how to use cloudstack 4.3.1 (upgraded version from 2.2.14), how to use the console VM after deprecation of realhostip.com, we are not using SSL certificates. And we don't have domain name. Thanks in advance
RE: VM Storage migration only works once not vice versa CS 4.3 Xen 6.2 SP1
Hi Sanjeev,
Thank you for response, yes the host from the new cluster can mount the primary
storage from the first cluster, I think this is the error
VM state transitted from :Stopped to Migrating with event:
StorageMigrationRequested
vm's original host id: 73 new host id: null host id before state transition:
null
Cheers
-Original Message-
From: Sanjeev Neelarapu [mailto:sanjeev.neelar...@citrix.com]
Sent: Monday, 25 August 2014 12:31 PM
To: us...@cloudstack.apache.org; dev@cloudstack.apache.org
Subject: RE: VM Storage migration only works once not vice versa CS 4.3 Xen
6.2 SP1
Hi Amin,
There are no error messages in the log snippet you have provided.
Is the host in the 2nd cluster from which you are trying vm storage migration
(2nd time after successful migration) able to mount the primary storage in the
cluster1? If not you may have to resolve that issue. Otherwise open a defect.
Thanks,
Sanjeev
-Original Message-
From: Amin Samir [mailto:a...@opencloud.net.au]
Sent: Monday, August 25, 2014 7:21 AM
To: dev@cloudstack.apache.org; us...@cloudstack.apache.org
Subject: VM Storage migration only works once not vice versa CS 4.3 Xen 6.2
SP1
Hello,
We have two clusters, the environment is as follows:
? CS Mgmt server 4.3 on CentOS 6.5
? Hypervisors Xen Server 6.2 SP1.
? One Zone, Two clusters with different two primary storage servers.
? Primary Secondary storage are NFS based.
When attempting to move a VM from one cluster to another, we succeed only the
first time, the VM and its files are moved to a new hypervisor and new storage
server in the new cluster, however when attempting to migrate the VM to the
original cluster, host, and original storage, the migration fails, so it is
only one time migration.
The logs are as follows, we would appreciate if someone has faced this issue
before, it is very helpful feature in cloud stack to migrate across clusters
and storage servers, but it seems it is not working, cause we simulated the
environment on CS 4.3 from scratch installation and the behaviour is the same.
/mnt/DATA/VM-SEC-01/volumes/2/1071 is not being created.
2014-08-21 11:37:40,671 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(catalina-exec-21:ctx-da934e40 ctx-fb603895) submit async job-10947, details:
AsyncJobVO {id:10947, userId: 2, accountId:
2, instanceType: None, instanceId: null, cmd:
org.apache.cloudstack.api.command.admin.vm.MigrateVMCmd, cmdInfo:
{response:json,sessionkey:w6C9EF9N9Xi0AhEXZ3Cdc9nQNkI\u003d,virtu
almachineid:ae028abb-4181-49fb-913f-bda34f77d792,cmdEventType:VM.MIGRATE,ctxUserId:2,storageid:098d4f7a-a040-3462-add1-d618d6ec53c1,httpmethod:GET,_:1408592259844,
ctxAccountId:2,ctxStartEventId:1088984}, cmdVersion: 0, status:
IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid:
53350633438879, completeMsid: null, lastUpda
ted: null, lastPolled: null, created: null}
2014-08-21 11:37:40,671 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Job-Executor-14:ctx-57b36727) Add job-10947 into job monitoring
2014-08-21 11:37:40,671 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Job-Executor-14:ctx-57b36727) Executing AsyncJobVO {id:10947, userId: 2,
accountId: 2, instanceType: None, instanceId: nu
ll, cmd: org.apache.cloudstack.api.command.admin.vm.MigrateVMCmd, cmdInfo:
{response:json,sessionkey:w6C9EF9N9Xi0AhEXZ3Cdc9nQNkI\u003d,virtualmachineid:ae028abb-4181-49fb-913f
-bda34f77d792,cmdEventType:VM.MIGRATE,ctxUserId:2,storageid:098d4f7a-a040-3462-add1-d618d6ec53c1,httpmethod:GET,_:1408592259844,ctxAccountId:2,ctxStartEventId:
1088984}, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode:
0, result: null, initMsid: 53350633438879, completeMsid: null, lastUpdated:
null, lastPolled: null, created:
null}
2014-08-21 11:37:40,672 DEBUG [c.c.a.ApiServlet] (catalina-exec-21:ctx-da934e40
ctx-fb603895) ===END=== 103.4.132.4 -- GET
command=migrateVirtualMachinestorageid=098d4f7a-a040-3462-ad
d1-d618d6ec53c1virtualmachineid=ae028abb-4181-49fb-913f-bda34f77d792response=jsonsessionkey=w6C9EF9N9Xi0AhEXZ3Cdc9nQNkI%3D_=1408592259844
2014-08-21 11:37:40,738 DEBUG [c.c.c.CapacityManagerImpl]
(Job-Executor-14:ctx-57b36727 ctx-fb603895) VM state transitted from :Stopped
to Migrating with event: StorageMigrationRequested
vm's original host id: 73 new host id: null host id before state transition:
null
2014-08-21 11:37:40,943 DEBUG [o.a.c.s.m.AncientDataMotionStrategy]
(Job-Executor-14:ctx-57b36727 ctx-fb603895) copyAsync inspecting src type
VOLUME copyAsync inspecting dest type VOLUME
2014-08-21 11:37:40,946 DEBUG [o.a.c.s.c.a.StorageCacheRandomAllocator]
(Job-Executor-14:ctx-57b36727 ctx-fb603895) Can't find staging storage in zone:
1
2014-08-21 11:37:41,092 DEBUG [c.c.a.t.Request] (Job-Executor-14:ctx-57b36727
ctx-fb603895) Seq 73-1469252089: Sending { Cmd , MgmtId: 53350633438879, via:
73(XEN-08), Ver: v1, Flags: 1
00011
VM Storage migration only works once not vice versa CS 4.3 Xen 6.2 SP1
Hello,
We have two clusters, the environment is as follows:
? CS Mgmt server 4.3 on CentOS 6.5
? Hypervisors Xen Server 6.2 SP1.
? One Zone, Two clusters with different two primary storage servers.
? Primary Secondary storage are NFS based.
When attempting to move a VM from one cluster to another, we succeed only the
first time, the VM and its files are moved to a new hypervisor and new storage
server in the new cluster, however when attempting to migrate the VM to the
original cluster, host, and original storage, the migration fails, so it is
only one time migration.
The logs are as follows, we would appreciate if someone has faced this issue
before, it is very helpful feature in cloud stack to migrate across clusters
and storage servers, but it seems it is not working, cause we simulated the
environment on CS 4.3 from scratch installation and the behaviour is the same.
/mnt/DATA/VM-SEC-01/volumes/2/1071 is not being created.
2014-08-21 11:37:40,671 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(catalina-exec-21:ctx-da934e40 ctx-fb603895) submit async job-10947, details:
AsyncJobVO {id:10947, userId: 2, accountId:
2, instanceType: None, instanceId: null, cmd:
org.apache.cloudstack.api.command.admin.vm.MigrateVMCmd, cmdInfo:
{response:json,sessionkey:w6C9EF9N9Xi0AhEXZ3Cdc9nQNkI\u003d,virtu
almachineid:ae028abb-4181-49fb-913f-bda34f77d792,cmdEventType:VM.MIGRATE,ctxUserId:2,storageid:098d4f7a-a040-3462-add1-d618d6ec53c1,httpmethod:GET,_:1408592259844,
ctxAccountId:2,ctxStartEventId:1088984}, cmdVersion: 0, status:
IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid:
53350633438879, completeMsid: null, lastUpda
ted: null, lastPolled: null, created: null}
2014-08-21 11:37:40,671 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Job-Executor-14:ctx-57b36727) Add job-10947 into job monitoring
2014-08-21 11:37:40,671 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Job-Executor-14:ctx-57b36727) Executing AsyncJobVO {id:10947, userId: 2,
accountId: 2, instanceType: None, instanceId: nu
ll, cmd: org.apache.cloudstack.api.command.admin.vm.MigrateVMCmd, cmdInfo:
{response:json,sessionkey:w6C9EF9N9Xi0AhEXZ3Cdc9nQNkI\u003d,virtualmachineid:ae028abb-4181-49fb-913f
-bda34f77d792,cmdEventType:VM.MIGRATE,ctxUserId:2,storageid:098d4f7a-a040-3462-add1-d618d6ec53c1,httpmethod:GET,_:1408592259844,ctxAccountId:2,ctxStartEventId:
1088984}, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode:
0, result: null, initMsid: 53350633438879, completeMsid: null, lastUpdated:
null, lastPolled: null, created:
null}
2014-08-21 11:37:40,672 DEBUG [c.c.a.ApiServlet] (catalina-exec-21:ctx-da934e40
ctx-fb603895) ===END=== 103.4.132.4 -- GET
command=migrateVirtualMachinestorageid=098d4f7a-a040-3462-ad
d1-d618d6ec53c1virtualmachineid=ae028abb-4181-49fb-913f-bda34f77d792response=jsonsessionkey=w6C9EF9N9Xi0AhEXZ3Cdc9nQNkI%3D_=1408592259844
2014-08-21 11:37:40,738 DEBUG [c.c.c.CapacityManagerImpl]
(Job-Executor-14:ctx-57b36727 ctx-fb603895) VM state transitted from :Stopped
to Migrating with event: StorageMigrationRequested
vm's original host id: 73 new host id: null host id before state transition:
null
2014-08-21 11:37:40,943 DEBUG [o.a.c.s.m.AncientDataMotionStrategy]
(Job-Executor-14:ctx-57b36727 ctx-fb603895) copyAsync inspecting src type
VOLUME copyAsync inspecting dest type VOLUME
2014-08-21 11:37:40,946 DEBUG [o.a.c.s.c.a.StorageCacheRandomAllocator]
(Job-Executor-14:ctx-57b36727 ctx-fb603895) Can't find staging storage in zone:
1
2014-08-21 11:37:41,092 DEBUG [c.c.a.t.Request] (Job-Executor-14:ctx-57b36727
ctx-fb603895) Seq 73-1469252089: Sending { Cmd , MgmtId: 53350633438879, via:
73(XEN-08), Ver: v1, Flags: 1
00011,
[{org.apache.cloudstack.storage.command.CopyCommand:{srcTO:{org.apache.cloudstack.storage.to.VolumeObjectTO:{uuid:86417b6e-3fda-476e-b4d1-3bc77a67be89,volumeType:ROOT
,dataStore:{org.apache.cloudstack.storage.to.PrimaryDataStoreTO:{uuid:1a07254a-c9d4-3ae0-8090-461e6597d818,id:221,poolType:NetworkFilesystem,host:10.8.1.40,path:/mn
t/DRIVE/DATA,port:2049,url:NetworkFilesystem://10.8.1.40//mnt/DRIVE/DATA/?ROLE=PrimarySTOREUUID=1a07254a-c9d4-3ae0-8090-461e6597d818}},name:ROOT-710,size:42949672960,path
:c0669da9-1f91-4bdf-85aa-1d90bd9dc966,volumeId:1071,vmName:i-2-710-VM,accountId:2,format:VHD,id:1071,deviceId:0,hypervisorType:XenServer}},destTO:{org.apache.clo
udstack.storage.to.VolumeObjectTO:{uuid:86417b6e-3fda-476e-b4d1-3bc77a67be89,volumeType:ROOT,dataStore:{com.cloud.agent.api.to.NfsTO:{_url:nfs://10.8.1.30/mnt/DATA/VM-SEC
-01,_role:Image}},name:ROOT-710,size:42949672960,path:volumes/2/1071,volumeId:1071,vmName:i-2-710-VM,accountId:2,format:VHD,id:1071,deviceId:0,hypervisorT
ype:XenServer}},executeInSequence:false,options:{},wait:21600}}] }
2014-08-21 11:37:41,092 DEBUG [c.c.a.t.Request] (Job-Executor-14:ctx-57b36727
ctx-fb603895) Seq 73-1469252089: Executing: { Cmd , MgmtId: 53350633438879,
via: 73(XEN-08),
RE: Snapshot partial file creates are not cleaned up.
Thanks Sangeetha for your prompt response. Kind Regards -Original Message- From: Sangeetha Hariharan [mailto:sangeetha.hariha...@citrix.com] Sent: Wednesday, 20 August 2014 1:10 AM To: dev@cloudstack.apache.org; us...@cloudstack.apache.org Subject: RE: Snapshot partial file creates are not cleaned up. Hi, Following are couple of issues that have been logged to track this issue: CLOUDSTACK-5395 - When backup snapshot fails because of backup.snapshot.wait time exceeding , the vhd entries form the primary store is not getting cleared. This bug was reopened to address - cleaning up of snapshot that errors out from primary store and secondary store specific to KVM hypervisor. CLOUDSTACK-5446 - KVM-Secondary Store down-Even after secondary store is brought back up after being down for few hours,snapshot jobs do not get triggered with reason there is other active snapshot tasks on the instance to which the volume is attached Thanks Sangeetha -Original Message- From: Amin Samir [mailto:a...@opencloud.net.au] Sent: Monday, August 18, 2014 9:55 PM To: dev@cloudstack.apache.org Subject: Snapshot partial file creates are not cleaned up. Hello, When a volume snapshot creation times out ( based on the global setting), the partial file creates is not cleaned up. Has anyone faced this? Or filled a bug for this? Kind Regards
Snapshot partial file creates are not cleaned up.
Hello, When a volume snapshot creation times out ( based on the global setting), the partial file creates is not cleaned up. Has anyone faced this? Or filled a bug for this? Kind Regards
RE: Cancel VM migration when cluster is facing issues
Thanks Min, very much. Kind Regards Amin From: Min Chen [mailto:min.c...@citrix.com] Sent: Saturday, 16 August 2014 2:19 AM To: dev@cloudstack.apache.org Subject: Re: Cancel VM migration when cluster is facing issues There should be a global configuration for this: enable.ha.storage.migration. You can turn that off. Thanks -min From: Amin Samir a...@opencloud.net.au mailto:a...@opencloud.net.au Reply-To: dev@cloudstack.apache.org mailto:dev@cloudstack.apache.org dev@cloudstack.apache.org mailto:dev@cloudstack.apache.org Date: Thursday, August 14, 2014 7:39 PM To: dev@cloudstack.apache.org mailto:dev@cloudstack.apache.org dev@cloudstack.apache.org mailto:dev@cloudstack.apache.org Subject: Cancel VM migration when cluster is facing issues Hello, Has anyone know how to cancel the process of migrating VM from cluster to another, this happens automatically when a cluster does not have the adequate capacity or storage has issues, it is also attempts migrating the storage of VM to the destination cluster primary storage. Thank in advance for your help Kind Regards Kind Regards Amin Samir Open Cloud Pty. Sales Support Mobile: +61414356558 E-mail: a...@opencloud.net.au mailto:a...@opencloud.net.au http://www.linkedin.com/company/backonline?trk=company_name http://www.facebook.com/plugins/likebox.php?href=https%3A%2F%2Fwww.facebook.com%2Fpages%2FOpenCloud-Pty-Ltd%2F281961941864174amp;widthamp;height=290amp;colorscheme=lightamp;show_faces=trueamp;header=trueamp;stream=falseamp;show_border=true
Cancel VM migration when cluster is facing issues
Hello, Has anyone know how to cancel the process of migrating VM from cluster to another, this happens automatically when a cluster does not have the adequate capacity or storage has issues, it is also attempts migrating the storage of VM to the destination cluster primary storage. Thank in advance for your help Kind Regards Kind Regards Amin Samir Open Cloud Pty. Sales Support Mobile: +61414356558 E-mail: a...@opencloud.net.au http://www.linkedin.com/company/backonline?trk=company_name http://www.facebook.com/plugins/likebox.php?href=https%3A%2F%2Fwww.facebook.com%2Fpages%2FOpenCloud-Pty-Ltd%2F281961941864174amp;widthamp;height=290amp;colorscheme=lightamp;show_faces=trueamp;header=trueamp;stream=falseamp;show_border=true
VM's using custom offerings wont let us use the migrate host from UI (CS 4.3)!!!
Hello, We have cloud stack 4.3 over centos our hypervisors are xen server 6.2 SP1 with latest updates, the VM's that are created from custom offering don't let us use the migrate host icon from the UI, however you can move them from the xen center console. Has anyone faced this issue before? Kind Regards
RE: VPN site to site with no cisco nor juniper on the customer end
Thanks Chiradeep tried it with Pfsense and it worked, just had an issue where setting the Negotiation mode had to set it to main rather than aggressive. Kind Regards Amin -Original Message- From: Chiradeep Vittal [mailto:chiradeep.vit...@citrix.com] Sent: Friday, 16 May 2014 2:36 PM To: dev@cloudstack.apache.org Subject: Re: VPN site to site with no cisco nor juniper on the customer end It is standard IPSec, so try it out and blog about it! From: Amin Samir a...@opencloud.net.aumailto:a...@opencloud.net.au Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org Date: Wednesday, May 14, 2014 at 11:41 PM To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org Subject: VPN site to site with no cisco nor juniper on the customer end Hello, Can anyone please advise if we can use site to site VPN and there is no Cisco nor Juniper routers on the other (customer end) for instance if the customer has Pfsense as his router and firewall , what shall we do? Is it possible or not? Our environment is as follows: ? MS 4.3 installed on CentOs 6.5, implemented in an advanced networking mode. ? Hypervisors are Xen Server 6.2 SP1. Thanks in advance for your support Kind Regards Amin
VPN site to site with no cisco nor juniper on the customer end
Hello, Can anyone please advise if we can use site to site VPN and there is no Cisco nor Juniper routers on the other (customer end) for instance if the customer has Pfsense as his router and firewall , what shall we do? Is it possible or not? Our environment is as follows: ? MS 4.3 installed on CentOs 6.5, implemented in an advanced networking mode. ? Hypervisors are Xen Server 6.2 SP1. Thanks in advance for your support Kind Regards Amin
RE: Console Proxy Error!!
Follow step number three and you should be fine. http://shankerbalan.net/blog/apache-cloudstack-4-3-0-released/ Kind Regards Amin -Original Message- From: Prakash Rao Banuka [mailto:prakas...@simplifydc.com] Sent: Saturday, 3 May 2014 2:14 AM To: dev@cloudstack.apache.org Subject: Console Proxy Error!! Hi, After upgrading from cloudstack 4.2 to cloudstack 4.3, the console proxy VM is running but when we actually go to an instance and click on view console then it is giving an error (Firefox can't find the server at 10-10-10-40.*.realhostip.com.)Google Chrome error is (Unable to resolve the server's DNS address.) Any Ideas? Thank you Prakash
RE: Interesting 4.2.1. Issue...
Hi Michael, For whole three days now we have been using CentOS 6.5 as OS platform for our management server, and no signs of crashes nor did we need have to restart on a daily basis. No signs of java heap space errors neither in the Catalina.out nor in management.log. The environment in detail as follows: 1) Management server operating system CentOs 6.5 (updated as of 9th of April). 2) CloudStack Management version 4.2.1. 3) The default Tomcat version that comes with CentOs 6.5 is (6.0.24) 3) Primary Secondary Storage server Ubuntu12.04 LTS (NFS Service) 4) Hypervisors Xen Server 6.1 update with the latest patches as of today 14th April. If we come through any issues will sure keep you updated. Kind Regards Amin -Original Message- From: Michael Phillips [mailto:mphilli7...@hotmail.com] Sent: Thursday, 10 April 2014 11:10 PM To: dev@cloudstack.apache.org Subject: RE: Interesting 4.2.1. Issue... Amin, Any luck on your project taking your 3 mgmt servers to 4.3 on Centos 6.5? Subject: RE: Interesting 4.2.1. Issue... From: a...@opencloud.net.au To: dev@cloudstack.apache.org Date: Fri, 4 Apr 2014 11:10:28 +0800 Thanks and thanks for sharing the steps Kind Regards Amin -Original Message- From: Michael Phillips [mailto:mphilli7...@hotmail.com] Sent: Friday, 4 April 2014 11:02 AM To: dev@cloudstack.apache.org Subject: RE: Interesting 4.2.1. Issue... So I manually downloaded tomcat 6.0.33 herehttps://cwiki.apache.org/confluence/display/CLOUDSTACK/Setting+up+ CloudStack+Development+Environment+on+Linux Then did the following1. extracted 6.0.33 to /usr/share/tomcat6.0.33. 2. Changed symlink of /usr/share/cloudstack-managemet/bin to /usr/share/tomcat6.0.33/bin3. Changed symlink of /usr/share/cloudstack-management/lib to /usr/share/tomcat6.0.33/lib4. Verified cloudstack was running tomcat 6.0.33 by creating a tomcat_version.jsp file in /usr/share/cloudstack-management/webapps/client code for tomcat_version.jsp can be found herehttp://stackoverflow.com/questions/14925073/how-to-find-out-runnin g-tomcat-version I'll definitely let you know how it goes... Subject: RE: Interesting 4.2.1. Issue... From: a...@opencloud.net.au To: dev@cloudstack.apache.org Date: Fri, 4 Apr 2014 10:43:35 +0800 I tried but I failed to do so, each time cloudstack attempts to install to go fetches the 6.0.35 from the repo, maybe you have installed it after installing the cloudstack, if you managed to have a running cloudstack version above the 6.0.33 feedback with the results. Kind Regards Amin -Original Message- From: Michael Phillips [mailto:mphilli7...@hotmail.com] Sent: Friday, 4 April 2014 10:41 AM To: dev@cloudstack.apache.org Subject: RE: Interesting 4.2.1. Issue... So did you try changing your version of tomcat? Subject: RE: Interesting 4.2.1. Issue... From: a...@opencloud.net.au To: dev@cloudstack.apache.org Date: Fri, 4 Apr 2014 10:35:42 +0800 cd /usr/share/tomcat6/bin/ ./version.sh The output should be 6.0.33 instead of 6.0.35 Using CATALINA_BASE: /usr/share/tomcat6 Using CATALINA_HOME: /usr/share/tomcat6 Using CATALINA_TMPDIR: /usr/share/tomcat6/temp Using JRE_HOME:/usr Using CLASSPATH: /usr/share/tomcat6/bin/bootstrap.jar Server version: Apache Tomcat/6.0.35 Server built: Server number: 6.0.35.0 OS Name:Linux OS Version: 3.11.0-18-generic Architecture: amd64 JVM Version:1.6.0_30-b30 JVM Vendor: Sun Microsystems Inc. Kind Regards Amin -Original Message- From: Michael Phillips [mailto:mphilli7...@hotmail.com] Sent: Friday, 4 April 2014 10:31 AM To: dev@cloudstack.apache.org Subject: RE: Interesting 4.2.1. Issue... I've put tomcat 6.0.33 on our mgmt servers. I'm going to monitor it for the next few days to see if we get the error again. Do you know any way way to verify the version of tomcat that's running? Subject: RE: Interesting 4.2.1. Issue... From: a...@opencloud.net.au To: dev@cloudstack.apache.org Date: Thu, 3 Apr 2014 10:35:29 +0800 No we didn't, it wouldn't matter because the memory would still fill up, the problem is it opens a thread and it fails to close it so whatever you will increase soon or later the memory will fill up (if I understand right) The error in catalina is as follows: SEVERE: The web application [/client] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@1bd66d2d]) and a value of type [com.cloud.api.SerializationContext] (value [com.cloud.api.SerializationContext@2f6baed9]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak. If someone could help with this error generated in the catalina log, that would
RE: Interesting 4.2.1. Issue...
Change your browser, try another browser that you never used to log on to any other cloudstack, or clear the cache or any other history in your browser it will work. Kind Regards Amin -Original Message- From: Michael Phillips [mailto:mphilli7...@hotmail.com] Sent: Monday, 14 April 2014 10:33 AM To: dev@cloudstack.apache.org Subject: RE: Interesting 4.2.1. Issue... Excellent news! Quick question. I tried using centos 6.5 with 4.2.1, however after installing I could never log in with the default username\password combo of admin\password. Did you experience that? Subject: RE: Interesting 4.2.1. Issue... From: a...@opencloud.net.au To: dev@cloudstack.apache.org Date: Mon, 14 Apr 2014 10:08:16 +0800 Hi Michael, For whole three days now we have been using CentOS 6.5 as OS platform for our management server, and no signs of crashes nor did we need have to restart on a daily basis. No signs of java heap space errors neither in the Catalina.out nor in management.log. The environment in detail as follows: 1) Management server operating system CentOs 6.5 (updated as of 9th of April). 2) CloudStack Management version 4.2.1. 3) The default Tomcat version that comes with CentOs 6.5 is (6.0.24) 3) Primary Secondary Storage server Ubuntu12.04 LTS (NFS Service) 4) Hypervisors Xen Server 6.1 update with the latest patches as of today 14th April. If we come through any issues will sure keep you updated. Kind Regards Amin -Original Message- From: Michael Phillips [mailto:mphilli7...@hotmail.com] Sent: Thursday, 10 April 2014 11:10 PM To: dev@cloudstack.apache.org Subject: RE: Interesting 4.2.1. Issue... Amin, Any luck on your project taking your 3 mgmt servers to 4.3 on Centos 6.5? Subject: RE: Interesting 4.2.1. Issue... From: a...@opencloud.net.au To: dev@cloudstack.apache.org Date: Fri, 4 Apr 2014 11:10:28 +0800 Thanks and thanks for sharing the steps Kind Regards Amin -Original Message- From: Michael Phillips [mailto:mphilli7...@hotmail.com] Sent: Friday, 4 April 2014 11:02 AM To: dev@cloudstack.apache.org Subject: RE: Interesting 4.2.1. Issue... So I manually downloaded tomcat 6.0.33 herehttps://cwiki.apache.org/confluence/display/CLOUDSTACK/Setting+u p+ CloudStack+Development+Environment+on+Linux Then did the following1. extracted 6.0.33 to /usr/share/tomcat6.0.33. 2. Changed symlink of /usr/share/cloudstack-managemet/bin to /usr/share/tomcat6.0.33/bin3. Changed symlink of /usr/share/cloudstack-management/lib to /usr/share/tomcat6.0.33/lib4. Verified cloudstack was running tomcat 6.0.33 by creating a tomcat_version.jsp file in /usr/share/cloudstack-management/webapps/client code for tomcat_version.jsp can be found herehttp://stackoverflow.com/questions/14925073/how-to-find-out-runn in g-tomcat-version I'll definitely let you know how it goes... Subject: RE: Interesting 4.2.1. Issue... From: a...@opencloud.net.au To: dev@cloudstack.apache.org Date: Fri, 4 Apr 2014 10:43:35 +0800 I tried but I failed to do so, each time cloudstack attempts to install to go fetches the 6.0.35 from the repo, maybe you have installed it after installing the cloudstack, if you managed to have a running cloudstack version above the 6.0.33 feedback with the results. Kind Regards Amin -Original Message- From: Michael Phillips [mailto:mphilli7...@hotmail.com] Sent: Friday, 4 April 2014 10:41 AM To: dev@cloudstack.apache.org Subject: RE: Interesting 4.2.1. Issue... So did you try changing your version of tomcat? Subject: RE: Interesting 4.2.1. Issue... From: a...@opencloud.net.au To: dev@cloudstack.apache.org Date: Fri, 4 Apr 2014 10:35:42 +0800 cd /usr/share/tomcat6/bin/ ./version.sh The output should be 6.0.33 instead of 6.0.35 Using CATALINA_BASE: /usr/share/tomcat6 Using CATALINA_HOME: /usr/share/tomcat6 Using CATALINA_TMPDIR: /usr/share/tomcat6/temp Using JRE_HOME:/usr Using CLASSPATH: /usr/share/tomcat6/bin/bootstrap.jar Server version: Apache Tomcat/6.0.35 Server built: Server number: 6.0.35.0 OS Name:Linux OS Version: 3.11.0-18-generic Architecture: amd64 JVM Version:1.6.0_30-b30 JVM Vendor: Sun Microsystems Inc. Kind Regards Amin -Original Message- From: Michael Phillips [mailto:mphilli7...@hotmail.com] Sent: Friday, 4 April 2014 10:31 AM To: dev@cloudstack.apache.org Subject: RE: Interesting 4.2.1. Issue... I've put tomcat 6.0.33 on our mgmt servers. I'm going to monitor it for the next few days to see if we get the error again. Do you know any way way to verify the version of tomcat that's running? Subject: RE: Interesting 4.2.1. Issue... From: a...@opencloud.net.au
RE: Interesting 4.2.1. Issue...
Michael, Just give me two days, will feedback you. I have to leave it till late night where is the Ubuntu server crash times. Kind Regards Amin -Original Message- From: Michael Phillips [mailto:mphilli7...@hotmail.com] Sent: Thursday, 10 April 2014 11:10 PM To: dev@cloudstack.apache.org Subject: RE: Interesting 4.2.1. Issue... Amin, Any luck on your project taking your 3 mgmt servers to 4.3 on Centos 6.5? Subject: RE: Interesting 4.2.1. Issue... From: a...@opencloud.net.au To: dev@cloudstack.apache.org Date: Fri, 4 Apr 2014 11:10:28 +0800 Thanks and thanks for sharing the steps Kind Regards Amin -Original Message- From: Michael Phillips [mailto:mphilli7...@hotmail.com] Sent: Friday, 4 April 2014 11:02 AM To: dev@cloudstack.apache.org Subject: RE: Interesting 4.2.1. Issue... So I manually downloaded tomcat 6.0.33 herehttps://cwiki.apache.org/confluence/display/CLOUDSTACK/Setting+up+ CloudStack+Development+Environment+on+Linux Then did the following1. extracted 6.0.33 to /usr/share/tomcat6.0.33. 2. Changed symlink of /usr/share/cloudstack-managemet/bin to /usr/share/tomcat6.0.33/bin3. Changed symlink of /usr/share/cloudstack-management/lib to /usr/share/tomcat6.0.33/lib4. Verified cloudstack was running tomcat 6.0.33 by creating a tomcat_version.jsp file in /usr/share/cloudstack-management/webapps/client code for tomcat_version.jsp can be found herehttp://stackoverflow.com/questions/14925073/how-to-find-out-runnin g-tomcat-version I'll definitely let you know how it goes... Subject: RE: Interesting 4.2.1. Issue... From: a...@opencloud.net.au To: dev@cloudstack.apache.org Date: Fri, 4 Apr 2014 10:43:35 +0800 I tried but I failed to do so, each time cloudstack attempts to install to go fetches the 6.0.35 from the repo, maybe you have installed it after installing the cloudstack, if you managed to have a running cloudstack version above the 6.0.33 feedback with the results. Kind Regards Amin -Original Message- From: Michael Phillips [mailto:mphilli7...@hotmail.com] Sent: Friday, 4 April 2014 10:41 AM To: dev@cloudstack.apache.org Subject: RE: Interesting 4.2.1. Issue... So did you try changing your version of tomcat? Subject: RE: Interesting 4.2.1. Issue... From: a...@opencloud.net.au To: dev@cloudstack.apache.org Date: Fri, 4 Apr 2014 10:35:42 +0800 cd /usr/share/tomcat6/bin/ ./version.sh The output should be 6.0.33 instead of 6.0.35 Using CATALINA_BASE: /usr/share/tomcat6 Using CATALINA_HOME: /usr/share/tomcat6 Using CATALINA_TMPDIR: /usr/share/tomcat6/temp Using JRE_HOME:/usr Using CLASSPATH: /usr/share/tomcat6/bin/bootstrap.jar Server version: Apache Tomcat/6.0.35 Server built: Server number: 6.0.35.0 OS Name:Linux OS Version: 3.11.0-18-generic Architecture: amd64 JVM Version:1.6.0_30-b30 JVM Vendor: Sun Microsystems Inc. Kind Regards Amin -Original Message- From: Michael Phillips [mailto:mphilli7...@hotmail.com] Sent: Friday, 4 April 2014 10:31 AM To: dev@cloudstack.apache.org Subject: RE: Interesting 4.2.1. Issue... I've put tomcat 6.0.33 on our mgmt servers. I'm going to monitor it for the next few days to see if we get the error again. Do you know any way way to verify the version of tomcat that's running? Subject: RE: Interesting 4.2.1. Issue... From: a...@opencloud.net.au To: dev@cloudstack.apache.org Date: Thu, 3 Apr 2014 10:35:29 +0800 No we didn't, it wouldn't matter because the memory would still fill up, the problem is it opens a thread and it fails to close it so whatever you will increase soon or later the memory will fill up (if I understand right) The error in catalina is as follows: SEVERE: The web application [/client] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@1bd66d2d]) and a value of type [com.cloud.api.SerializationContext] (value [com.cloud.api.SerializationContext@2f6baed9]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak. If someone could help with this error generated in the catalina log, that would be much appreicated. Kind Regards Amin -Original Message- From: Michael Phillips [mailto:mphilli7...@hotmail.com] Sent: Thursday, 3 April 2014 9:34 AM To: dev@cloudstack.apache.org Subject: RE: Interesting 4.2.1. Issue... A few other articles also mention setting the initial heap size -Xms to the same value as the heap size, to go ahead and fully commit the heap. Have you tried that? One other thing I am curious of is have you fiddled with the Perm Size XX:Permsize setting? Subject
RE: Interesting 4.2.1. Issue...
Xen Server 6.1 with the latest updates As of today 11-April-2014 Kind Regards Amin -Original Message- From: Michael Phillips [mailto:mphilli7...@hotmail.com] Sent: Friday, 11 April 2014 9:16 AM To: dev@cloudstack.apache.org Subject: RE: Interesting 4.2.1. Issue... Gotcha... What hypervisor are you using with CS? Subject: RE: Interesting 4.2.1. Issue... From: a...@opencloud.net.au To: dev@cloudstack.apache.org Date: Fri, 11 Apr 2014 09:05:29 +0800 Michael, Just give me two days, will feedback you. I have to leave it till late night where is the Ubuntu server crash times. Kind Regards Amin -Original Message- From: Michael Phillips [mailto:mphilli7...@hotmail.com] Sent: Thursday, 10 April 2014 11:10 PM To: dev@cloudstack.apache.org Subject: RE: Interesting 4.2.1. Issue... Amin, Any luck on your project taking your 3 mgmt servers to 4.3 on Centos 6.5? Subject: RE: Interesting 4.2.1. Issue... From: a...@opencloud.net.au To: dev@cloudstack.apache.org Date: Fri, 4 Apr 2014 11:10:28 +0800 Thanks and thanks for sharing the steps Kind Regards Amin -Original Message- From: Michael Phillips [mailto:mphilli7...@hotmail.com] Sent: Friday, 4 April 2014 11:02 AM To: dev@cloudstack.apache.org Subject: RE: Interesting 4.2.1. Issue... So I manually downloaded tomcat 6.0.33 herehttps://cwiki.apache.org/confluence/display/CLOUDSTACK/Setting+u p+ CloudStack+Development+Environment+on+Linux Then did the following1. extracted 6.0.33 to /usr/share/tomcat6.0.33. 2. Changed symlink of /usr/share/cloudstack-managemet/bin to /usr/share/tomcat6.0.33/bin3. Changed symlink of /usr/share/cloudstack-management/lib to /usr/share/tomcat6.0.33/lib4. Verified cloudstack was running tomcat 6.0.33 by creating a tomcat_version.jsp file in /usr/share/cloudstack-management/webapps/client code for tomcat_version.jsp can be found herehttp://stackoverflow.com/questions/14925073/how-to-find-out-runn in g-tomcat-version I'll definitely let you know how it goes... Subject: RE: Interesting 4.2.1. Issue... From: a...@opencloud.net.au To: dev@cloudstack.apache.org Date: Fri, 4 Apr 2014 10:43:35 +0800 I tried but I failed to do so, each time cloudstack attempts to install to go fetches the 6.0.35 from the repo, maybe you have installed it after installing the cloudstack, if you managed to have a running cloudstack version above the 6.0.33 feedback with the results. Kind Regards Amin -Original Message- From: Michael Phillips [mailto:mphilli7...@hotmail.com] Sent: Friday, 4 April 2014 10:41 AM To: dev@cloudstack.apache.org Subject: RE: Interesting 4.2.1. Issue... So did you try changing your version of tomcat? Subject: RE: Interesting 4.2.1. Issue... From: a...@opencloud.net.au To: dev@cloudstack.apache.org Date: Fri, 4 Apr 2014 10:35:42 +0800 cd /usr/share/tomcat6/bin/ ./version.sh The output should be 6.0.33 instead of 6.0.35 Using CATALINA_BASE: /usr/share/tomcat6 Using CATALINA_HOME: /usr/share/tomcat6 Using CATALINA_TMPDIR: /usr/share/tomcat6/temp Using JRE_HOME:/usr Using CLASSPATH: /usr/share/tomcat6/bin/bootstrap.jar Server version: Apache Tomcat/6.0.35 Server built: Server number: 6.0.35.0 OS Name:Linux OS Version: 3.11.0-18-generic Architecture: amd64 JVM Version:1.6.0_30-b30 JVM Vendor: Sun Microsystems Inc. Kind Regards Amin -Original Message- From: Michael Phillips [mailto:mphilli7...@hotmail.com] Sent: Friday, 4 April 2014 10:31 AM To: dev@cloudstack.apache.org Subject: RE: Interesting 4.2.1. Issue... I've put tomcat 6.0.33 on our mgmt servers. I'm going to monitor it for the next few days to see if we get the error again. Do you know any way way to verify the version of tomcat that's running? Subject: RE: Interesting 4.2.1. Issue... From: a...@opencloud.net.au To: dev@cloudstack.apache.org Date: Thu, 3 Apr 2014 10:35:29 +0800 No we didn't, it wouldn't matter because the memory would still fill up, the problem is it opens a thread and it fails to close it so whatever you will increase soon or later the memory will fill up (if I understand right) The error in catalina is as follows: SEVERE: The web application [/client] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@1bd66d2d]) and a value of type [com.cloud.api.SerializationContext] (value [com.cloud.api.SerializationContext@2f6baed9]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak. If someone could help with this error generated in the catalina
RE: Interesting 4.2.1. Issue...
I am sorry to hear that, by next week we will be rebuilding ours using CentOS 6.5, will feedback you with the status. Kind Regards Amin -Original Message- From: Michael Phillips [mailto:mphilli7...@hotmail.com] Sent: Sunday, 6 April 2014 1:24 AM To: dev@cloudstack.apache.org Subject: RE: Interesting 4.2.1. Issue... Took about 2 days, but the heap space issue has re-occurred. ...sigh... So switching to tomcat 6.0.33 had zero effect.. Anyone else have any ideas?? Subject: RE: Interesting 4.2.1. Issue... From: a...@opencloud.net.au To: dev@cloudstack.apache.org Date: Fri, 4 Apr 2014 11:10:28 +0800 Thanks and thanks for sharing the steps Kind Regards Amin -Original Message- From: Michael Phillips [mailto:mphilli7...@hotmail.com] Sent: Friday, 4 April 2014 11:02 AM To: dev@cloudstack.apache.org Subject: RE: Interesting 4.2.1. Issue... So I manually downloaded tomcat 6.0.33 herehttps://cwiki.apache.org/confluence/display/CLOUDSTACK/Setting+up+ CloudStack+Development+Environment+on+Linux Then did the following1. extracted 6.0.33 to /usr/share/tomcat6.0.33. 2. Changed symlink of /usr/share/cloudstack-managemet/bin to /usr/share/tomcat6.0.33/bin3. Changed symlink of /usr/share/cloudstack-management/lib to /usr/share/tomcat6.0.33/lib4. Verified cloudstack was running tomcat 6.0.33 by creating a tomcat_version.jsp file in /usr/share/cloudstack-management/webapps/client code for tomcat_version.jsp can be found herehttp://stackoverflow.com/questions/14925073/how-to-find-out-runnin g-tomcat-version I'll definitely let you know how it goes... Subject: RE: Interesting 4.2.1. Issue... From: a...@opencloud.net.au To: dev@cloudstack.apache.org Date: Fri, 4 Apr 2014 10:43:35 +0800 I tried but I failed to do so, each time cloudstack attempts to install to go fetches the 6.0.35 from the repo, maybe you have installed it after installing the cloudstack, if you managed to have a running cloudstack version above the 6.0.33 feedback with the results. Kind Regards Amin -Original Message- From: Michael Phillips [mailto:mphilli7...@hotmail.com] Sent: Friday, 4 April 2014 10:41 AM To: dev@cloudstack.apache.org Subject: RE: Interesting 4.2.1. Issue... So did you try changing your version of tomcat? Subject: RE: Interesting 4.2.1. Issue... From: a...@opencloud.net.au To: dev@cloudstack.apache.org Date: Fri, 4 Apr 2014 10:35:42 +0800 cd /usr/share/tomcat6/bin/ ./version.sh The output should be 6.0.33 instead of 6.0.35 Using CATALINA_BASE: /usr/share/tomcat6 Using CATALINA_HOME: /usr/share/tomcat6 Using CATALINA_TMPDIR: /usr/share/tomcat6/temp Using JRE_HOME:/usr Using CLASSPATH: /usr/share/tomcat6/bin/bootstrap.jar Server version: Apache Tomcat/6.0.35 Server built: Server number: 6.0.35.0 OS Name:Linux OS Version: 3.11.0-18-generic Architecture: amd64 JVM Version:1.6.0_30-b30 JVM Vendor: Sun Microsystems Inc. Kind Regards Amin -Original Message- From: Michael Phillips [mailto:mphilli7...@hotmail.com] Sent: Friday, 4 April 2014 10:31 AM To: dev@cloudstack.apache.org Subject: RE: Interesting 4.2.1. Issue... I've put tomcat 6.0.33 on our mgmt servers. I'm going to monitor it for the next few days to see if we get the error again. Do you know any way way to verify the version of tomcat that's running? Subject: RE: Interesting 4.2.1. Issue... From: a...@opencloud.net.au To: dev@cloudstack.apache.org Date: Thu, 3 Apr 2014 10:35:29 +0800 No we didn't, it wouldn't matter because the memory would still fill up, the problem is it opens a thread and it fails to close it so whatever you will increase soon or later the memory will fill up (if I understand right) The error in catalina is as follows: SEVERE: The web application [/client] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@1bd66d2d]) and a value of type [com.cloud.api.SerializationContext] (value [com.cloud.api.SerializationContext@2f6baed9]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak. If someone could help with this error generated in the catalina log, that would be much appreicated. Kind Regards Amin -Original Message- From: Michael Phillips [mailto:mphilli7...@hotmail.com] Sent: Thursday, 3 April 2014 9:34 AM To: dev@cloudstack.apache.org Subject: RE: Interesting 4.2.1. Issue... A few other articles also mention setting the initial heap size -Xms to the same value as the heap size, to go ahead and fully commit the heap. Have you tried that? One other thing I am curious of is have you fiddled
RE: Cloud Stack 4.2.1 issue (Unable to start instance due to DB Exception)
Hi Yitao, Thanks for your response, we managed to solve it, the issue was with the user_statistics the id was not set to auto increment, we removed the records and altered the table to have the id column to auto increment and it worked. Kind Regards Amin -Original Message- From: Yitao Jiang [mailto:willier...@gmail.com] Sent: Friday, 4 April 2014 6:42 PM To: dev@cloudstack.apache.org Subject: RE: Cloud Stack 4.2.1 issue (Unable to start instance due to DB Exception) Hi, Amin Can u make sure that the vrouter of new network created ?never saw this exception before. More related logs will be helpful. On Apr 3, 2014 9:13 AM, Amin Samir a...@opencloud.net.au wrote: Hello, Can anyone please help with the below issue? Kind Regards Amin -Original Message- From: Amin Samir Sent: Tuesday, 1 April 2014 4:18 PM To: dev@cloudstack.apache.org Subject: Cloud Stack 4.2.1 issue (Unable to start instance due to DB Exception) Hello, We are facing an issue when we spin up a new instance with a new network (router VM), it gives the following error: Resource [Host:59] is unreachable: Host 59: Unable to start instance due to DB Exception on: com.mysql.jdbc.JDBC4PreparedStatement@5126808amailto: com.mysql.jdbc.JDBC4PreparedStatement@5126808a : INSERT INTO user_statistics (user_statistics.data_center_id, user_statistics.account_id, user_statistics.public_ip_address, user_statistics.device_id, user_statistics.device_type, user_statistics.network_id, user_statistics.net_bytes_received, user_statistics.net_bytes_sent, user_statistics.current_bytes_received, user_statistics.current_bytes_sent, user_statistics.agg_bytes_received, user_statistics.agg_bytes_sent) VALUES (1, 28, null, 360, _binary'DomainRouter', 237, 0, 0, 0, 0, 0, 0) Also we are facing this error every now and then: (DirectAgent-93:null) GetStorageStats Exception:SR_BACKEND_FAILURE_40The SR scan failed [opterr=uuid=870ed96e-d62b-428a-88dc-41afc75d311a]host:13944281-3f7e-4065-8580-1dc5b64cfdb1storageid: 098d4f7a-a040-3462-add1-d618d6ec53c1 However when spinning a new instance with an existing network (router VM) it works fine!! Has anyone faced this issue, your help would be much appreciated. Thanks Kind Regards Amin
RE: Interesting 4.2.1. Issue...
cd /usr/share/tomcat6/bin/ ./version.sh The output should be 6.0.33 instead of 6.0.35 Using CATALINA_BASE: /usr/share/tomcat6 Using CATALINA_HOME: /usr/share/tomcat6 Using CATALINA_TMPDIR: /usr/share/tomcat6/temp Using JRE_HOME:/usr Using CLASSPATH: /usr/share/tomcat6/bin/bootstrap.jar Server version: Apache Tomcat/6.0.35 Server built: Server number: 6.0.35.0 OS Name:Linux OS Version: 3.11.0-18-generic Architecture: amd64 JVM Version:1.6.0_30-b30 JVM Vendor: Sun Microsystems Inc. Kind Regards Amin -Original Message- From: Michael Phillips [mailto:mphilli7...@hotmail.com] Sent: Friday, 4 April 2014 10:31 AM To: dev@cloudstack.apache.org Subject: RE: Interesting 4.2.1. Issue... I've put tomcat 6.0.33 on our mgmt servers. I'm going to monitor it for the next few days to see if we get the error again. Do you know any way way to verify the version of tomcat that's running? Subject: RE: Interesting 4.2.1. Issue... From: a...@opencloud.net.au To: dev@cloudstack.apache.org Date: Thu, 3 Apr 2014 10:35:29 +0800 No we didn't, it wouldn't matter because the memory would still fill up, the problem is it opens a thread and it fails to close it so whatever you will increase soon or later the memory will fill up (if I understand right) The error in catalina is as follows: SEVERE: The web application [/client] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@1bd66d2d]) and a value of type [com.cloud.api.SerializationContext] (value [com.cloud.api.SerializationContext@2f6baed9]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak. If someone could help with this error generated in the catalina log, that would be much appreicated. Kind Regards Amin -Original Message- From: Michael Phillips [mailto:mphilli7...@hotmail.com] Sent: Thursday, 3 April 2014 9:34 AM To: dev@cloudstack.apache.org Subject: RE: Interesting 4.2.1. Issue... A few other articles also mention setting the initial heap size -Xms to the same value as the heap size, to go ahead and fully commit the heap. Have you tried that? One other thing I am curious of is have you fiddled with the Perm Size XX:Permsize setting? Subject: RE: Interesting 4.2.1. Issue... From: a...@opencloud.net.au To: dev@cloudstack.apache.org Date: Thu, 3 Apr 2014 09:26:31 +0800 Believe or not!! We have tried setting them in both formats and still the catalina log produces java heap space error Kind Regards Amin -Original Message- From: Michael Phillips [mailto:mphilli7...@hotmail.com] Sent: Thursday, 3 April 2014 9:24 AM To: dev@cloudstack.apache.org Subject: RE: Interesting 4.2.1. Issue... This may be a silly question, but in all of the docs I am reading online in regards to increasing the java heap size, they are specifying it as -XmxsizeMB example -Xmx2048m vs -Xmx2g Is it possible that it's not reading the 2g variable as 2GB? Subject: RE: Interesting 4.2.1. Issue... From: a...@opencloud.net.au To: dev@cloudstack.apache.org Date: Thu, 3 Apr 2014 09:06:17 +0800 It is 6.0.35 and it still produces this error, even after increasing the Xmx to 4g, we have installed tomcat 6.0.33 and each time we install the cloudstack it does not sense the installed 6.0.33 and attempts to install 6.0.35 as it is dependent on it. Silly solution is that we scheduled a daily restart @ 2PM through a cron job But first you have to killall jvsc then restart the management server. What we are considering is to migrate the management server to CentOS 6.5 it comes with tomcat 6.0.24 and mysql 5.1, we attempted to restore the dump on a pilot environment and it worked. If someone else has a better solution than this would you please share it? Kind Regards Amin -Original Message- From: Michael Phillips [mailto:mphilli7...@hotmail.com] Sent: Thursday, 3 April 2014 5:31 AM To: dev@cloudstack.apache.org Subject: RE: Interesting 4.2.1. Issue... So I just checked my tomcat version and we are running 6.0.35 which must be the default that comes with Ubuntu 12.04 out of the box. Our memory settings are as follows: JAVA_OPTS=-Djava.awt.headless=true -Dcom.sun.management.jmxremote.port=45219 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Xmx4g -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/log/cloudstack/management/ -XX:PermSize=512M -XX:MaxPermSize=800m So what version of tomcat are you running 6.0.35 or 6.0.33? Subject: RE: Interesting 4.2.1. Issue... From: a...@opencloud.net.au To: dev@cloudstack.apache.org Date: Tue, 1 Apr 2014 11:23:57 +0800 We have the same issue, after an upgrade from 2.2.14 to 4.2.1
RE: Interesting 4.2.1. Issue...
I tried but I failed to do so, each time cloudstack attempts to install to go fetches the 6.0.35 from the repo, maybe you have installed it after installing the cloudstack, if you managed to have a running cloudstack version above the 6.0.33 feedback with the results. Kind Regards Amin -Original Message- From: Michael Phillips [mailto:mphilli7...@hotmail.com] Sent: Friday, 4 April 2014 10:41 AM To: dev@cloudstack.apache.org Subject: RE: Interesting 4.2.1. Issue... So did you try changing your version of tomcat? Subject: RE: Interesting 4.2.1. Issue... From: a...@opencloud.net.au To: dev@cloudstack.apache.org Date: Fri, 4 Apr 2014 10:35:42 +0800 cd /usr/share/tomcat6/bin/ ./version.sh The output should be 6.0.33 instead of 6.0.35 Using CATALINA_BASE: /usr/share/tomcat6 Using CATALINA_HOME: /usr/share/tomcat6 Using CATALINA_TMPDIR: /usr/share/tomcat6/temp Using JRE_HOME:/usr Using CLASSPATH: /usr/share/tomcat6/bin/bootstrap.jar Server version: Apache Tomcat/6.0.35 Server built: Server number: 6.0.35.0 OS Name:Linux OS Version: 3.11.0-18-generic Architecture: amd64 JVM Version:1.6.0_30-b30 JVM Vendor: Sun Microsystems Inc. Kind Regards Amin -Original Message- From: Michael Phillips [mailto:mphilli7...@hotmail.com] Sent: Friday, 4 April 2014 10:31 AM To: dev@cloudstack.apache.org Subject: RE: Interesting 4.2.1. Issue... I've put tomcat 6.0.33 on our mgmt servers. I'm going to monitor it for the next few days to see if we get the error again. Do you know any way way to verify the version of tomcat that's running? Subject: RE: Interesting 4.2.1. Issue... From: a...@opencloud.net.au To: dev@cloudstack.apache.org Date: Thu, 3 Apr 2014 10:35:29 +0800 No we didn't, it wouldn't matter because the memory would still fill up, the problem is it opens a thread and it fails to close it so whatever you will increase soon or later the memory will fill up (if I understand right) The error in catalina is as follows: SEVERE: The web application [/client] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@1bd66d2d]) and a value of type [com.cloud.api.SerializationContext] (value [com.cloud.api.SerializationContext@2f6baed9]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak. If someone could help with this error generated in the catalina log, that would be much appreicated. Kind Regards Amin -Original Message- From: Michael Phillips [mailto:mphilli7...@hotmail.com] Sent: Thursday, 3 April 2014 9:34 AM To: dev@cloudstack.apache.org Subject: RE: Interesting 4.2.1. Issue... A few other articles also mention setting the initial heap size -Xms to the same value as the heap size, to go ahead and fully commit the heap. Have you tried that? One other thing I am curious of is have you fiddled with the Perm Size XX:Permsize setting? Subject: RE: Interesting 4.2.1. Issue... From: a...@opencloud.net.au To: dev@cloudstack.apache.org Date: Thu, 3 Apr 2014 09:26:31 +0800 Believe or not!! We have tried setting them in both formats and still the catalina log produces java heap space error Kind Regards Amin -Original Message- From: Michael Phillips [mailto:mphilli7...@hotmail.com] Sent: Thursday, 3 April 2014 9:24 AM To: dev@cloudstack.apache.org Subject: RE: Interesting 4.2.1. Issue... This may be a silly question, but in all of the docs I am reading online in regards to increasing the java heap size, they are specifying it as -XmxsizeMB example -Xmx2048m vs -Xmx2g Is it possible that it's not reading the 2g variable as 2GB? Subject: RE: Interesting 4.2.1. Issue... From: a...@opencloud.net.au To: dev@cloudstack.apache.org Date: Thu, 3 Apr 2014 09:06:17 +0800 It is 6.0.35 and it still produces this error, even after increasing the Xmx to 4g, we have installed tomcat 6.0.33 and each time we install the cloudstack it does not sense the installed 6.0.33 and attempts to install 6.0.35 as it is dependent on it. Silly solution is that we scheduled a daily restart @ 2PM through a cron job But first you have to killall jvsc then restart the management server. What we are considering is to migrate the management server to CentOS 6.5 it comes with tomcat 6.0.24 and mysql 5.1, we attempted to restore the dump on a pilot environment and it worked. If someone else has a better solution than this would you please share it? Kind Regards Amin -Original Message- From: Michael Phillips [mailto:mphilli7...@hotmail.com] Sent: Thursday, 3 April 2014 5:31 AM To: dev@cloudstack.apache.org Subject: RE: Interesting 4.2.1
RE: Interesting 4.2.1. Issue...
Thanks and thanks for sharing the steps Kind Regards Amin -Original Message- From: Michael Phillips [mailto:mphilli7...@hotmail.com] Sent: Friday, 4 April 2014 11:02 AM To: dev@cloudstack.apache.org Subject: RE: Interesting 4.2.1. Issue... So I manually downloaded tomcat 6.0.33 herehttps://cwiki.apache.org/confluence/display/CLOUDSTACK/Setting+up+CloudStack+Development+Environment+on+Linux Then did the following1. extracted 6.0.33 to /usr/share/tomcat6.0.33. 2. Changed symlink of /usr/share/cloudstack-managemet/bin to /usr/share/tomcat6.0.33/bin3. Changed symlink of /usr/share/cloudstack-management/lib to /usr/share/tomcat6.0.33/lib4. Verified cloudstack was running tomcat 6.0.33 by creating a tomcat_version.jsp file in /usr/share/cloudstack-management/webapps/client code for tomcat_version.jsp can be found herehttp://stackoverflow.com/questions/14925073/how-to-find-out-running-tomcat-version I'll definitely let you know how it goes... Subject: RE: Interesting 4.2.1. Issue... From: a...@opencloud.net.au To: dev@cloudstack.apache.org Date: Fri, 4 Apr 2014 10:43:35 +0800 I tried but I failed to do so, each time cloudstack attempts to install to go fetches the 6.0.35 from the repo, maybe you have installed it after installing the cloudstack, if you managed to have a running cloudstack version above the 6.0.33 feedback with the results. Kind Regards Amin -Original Message- From: Michael Phillips [mailto:mphilli7...@hotmail.com] Sent: Friday, 4 April 2014 10:41 AM To: dev@cloudstack.apache.org Subject: RE: Interesting 4.2.1. Issue... So did you try changing your version of tomcat? Subject: RE: Interesting 4.2.1. Issue... From: a...@opencloud.net.au To: dev@cloudstack.apache.org Date: Fri, 4 Apr 2014 10:35:42 +0800 cd /usr/share/tomcat6/bin/ ./version.sh The output should be 6.0.33 instead of 6.0.35 Using CATALINA_BASE: /usr/share/tomcat6 Using CATALINA_HOME: /usr/share/tomcat6 Using CATALINA_TMPDIR: /usr/share/tomcat6/temp Using JRE_HOME:/usr Using CLASSPATH: /usr/share/tomcat6/bin/bootstrap.jar Server version: Apache Tomcat/6.0.35 Server built: Server number: 6.0.35.0 OS Name:Linux OS Version: 3.11.0-18-generic Architecture: amd64 JVM Version:1.6.0_30-b30 JVM Vendor: Sun Microsystems Inc. Kind Regards Amin -Original Message- From: Michael Phillips [mailto:mphilli7...@hotmail.com] Sent: Friday, 4 April 2014 10:31 AM To: dev@cloudstack.apache.org Subject: RE: Interesting 4.2.1. Issue... I've put tomcat 6.0.33 on our mgmt servers. I'm going to monitor it for the next few days to see if we get the error again. Do you know any way way to verify the version of tomcat that's running? Subject: RE: Interesting 4.2.1. Issue... From: a...@opencloud.net.au To: dev@cloudstack.apache.org Date: Thu, 3 Apr 2014 10:35:29 +0800 No we didn't, it wouldn't matter because the memory would still fill up, the problem is it opens a thread and it fails to close it so whatever you will increase soon or later the memory will fill up (if I understand right) The error in catalina is as follows: SEVERE: The web application [/client] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@1bd66d2d]) and a value of type [com.cloud.api.SerializationContext] (value [com.cloud.api.SerializationContext@2f6baed9]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak. If someone could help with this error generated in the catalina log, that would be much appreicated. Kind Regards Amin -Original Message- From: Michael Phillips [mailto:mphilli7...@hotmail.com] Sent: Thursday, 3 April 2014 9:34 AM To: dev@cloudstack.apache.org Subject: RE: Interesting 4.2.1. Issue... A few other articles also mention setting the initial heap size -Xms to the same value as the heap size, to go ahead and fully commit the heap. Have you tried that? One other thing I am curious of is have you fiddled with the Perm Size XX:Permsize setting? Subject: RE: Interesting 4.2.1. Issue... From: a...@opencloud.net.au To: dev@cloudstack.apache.org Date: Thu, 3 Apr 2014 09:26:31 +0800 Believe or not!! We have tried setting them in both formats and still the catalina log produces java heap space error Kind Regards Amin -Original Message- From: Michael Phillips [mailto:mphilli7...@hotmail.com] Sent: Thursday, 3 April 2014 9:24 AM To: dev@cloudstack.apache.org Subject: RE: Interesting 4.2.1. Issue... This may be a silly question, but in all of the docs I am reading online in regards to increasing the java heap size, they are specifying
RE: Interesting 4.2.1. Issue...
It is 6.0.35 and it still produces this error, even after increasing the Xmx to 4g, we have installed tomcat 6.0.33 and each time we install the cloudstack it does not sense the installed 6.0.33 and attempts to install 6.0.35 as it is dependent on it. Silly solution is that we scheduled a daily restart @ 2PM through a cron job But first you have to killall jvsc then restart the management server. What we are considering is to migrate the management server to CentOS 6.5 it comes with tomcat 6.0.24 and mysql 5.1, we attempted to restore the dump on a pilot environment and it worked. If someone else has a better solution than this would you please share it? Kind Regards Amin -Original Message- From: Michael Phillips [mailto:mphilli7...@hotmail.com] Sent: Thursday, 3 April 2014 5:31 AM To: dev@cloudstack.apache.org Subject: RE: Interesting 4.2.1. Issue... So I just checked my tomcat version and we are running 6.0.35 which must be the default that comes with Ubuntu 12.04 out of the box. Our memory settings are as follows: JAVA_OPTS=-Djava.awt.headless=true -Dcom.sun.management.jmxremote.port=45219 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Xmx4g -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/log/cloudstack/management/ -XX:PermSize=512M -XX:MaxPermSize=800m So what version of tomcat are you running 6.0.35 or 6.0.33? Subject: RE: Interesting 4.2.1. Issue... From: a...@opencloud.net.au To: dev@cloudstack.apache.org Date: Tue, 1 Apr 2014 11:23:57 +0800 We have the same issue, after an upgrade from 2.2.14 to 4.2.1, and during this upgrade we had upgrade from Ubuntu 10 LTS to Ubuntu 12 LTS, it seems it related to tomcat 6.0.35, because it is recommended to have tomcat 6.0.33 which doesn't come by default with Ubuntu 12.04.4 Kind Regards Amin -Original Message- From: Marcus [mailto:shadow...@gmail.com] Sent: Tuesday, 1 April 2014 6:22 AM To: dev@cloudstack.apache.org Subject: Re: Interesting 4.2.1. Issue... I'm running 3 mgmt servers on 4.2.1, haven't seen any issues like that. You can send along your memory settings... here's what I'm running: JAVA_OPTS=-Djava.awt.headless=true -Dcom.sun.management.jmxremote.port=45219 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Xmx2g -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/log/cloudstack/management/ -XX:PermSize=512M -XX:MaxPermSize=800m On Mon, Mar 31, 2014 at 9:33 AM, Michael Phillips mphilli7...@hotmail.com wrote: So I have a redundant pair of management servers running on 4.2.1. At least once a day one of the management servers crashes and the log gets filled with the following messages: java.lang.OutOfMemoryError: Java heap spac0java.lang.ArrayIndexOutOfBoundsExceptioneCaused by: java.io.EOFException: SSL peer shut down incorrectlyCaused by: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake and there are a few others. When the one management server tanks, although the other management server is up and active, it won't actually process any UI commands until the crashed server is restarted. Example of won't process any UI command is create a new instance, create a new account, etc. After doing some searching I have found that others have noticed java heap errors in 4.2.1, and the suggested fix is to increase the heap size. I am planning on increasing it from 2g to 4g, however if the problem is something like a memory leak, then increasing the heap size will just delay the inevitable. Has anyone else fixed this issue by increasing the heap size? Or what is the recommended value? **updated...as expected I increased the heap size from 2g to 4g, and it just too k longer for the problem to reoccur... In my honest opinion of bigger concern is the fact that when one management server crashes the other stops functioning as well. So this begs the question of why even bother with a redundant pair of servers..Anybody else experience this issue? I would love to hear any dev guys opinion on this
RE: Interesting 4.2.1. Issue...
No we didn't, it wouldn't matter because the memory would still fill up, the problem is it opens a thread and it fails to close it so whatever you will increase soon or later the memory will fill up (if I understand right) The error in catalina is as follows: SEVERE: The web application [/client] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@1bd66d2d]) and a value of type [com.cloud.api.SerializationContext] (value [com.cloud.api.SerializationContext@2f6baed9]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak. If someone could help with this error generated in the catalina log, that would be much appreicated. Kind Regards Amin -Original Message- From: Michael Phillips [mailto:mphilli7...@hotmail.com] Sent: Thursday, 3 April 2014 9:34 AM To: dev@cloudstack.apache.org Subject: RE: Interesting 4.2.1. Issue... A few other articles also mention setting the initial heap size -Xms to the same value as the heap size, to go ahead and fully commit the heap. Have you tried that? One other thing I am curious of is have you fiddled with the Perm Size XX:Permsize setting? Subject: RE: Interesting 4.2.1. Issue... From: a...@opencloud.net.au To: dev@cloudstack.apache.org Date: Thu, 3 Apr 2014 09:26:31 +0800 Believe or not!! We have tried setting them in both formats and still the catalina log produces java heap space error Kind Regards Amin -Original Message- From: Michael Phillips [mailto:mphilli7...@hotmail.com] Sent: Thursday, 3 April 2014 9:24 AM To: dev@cloudstack.apache.org Subject: RE: Interesting 4.2.1. Issue... This may be a silly question, but in all of the docs I am reading online in regards to increasing the java heap size, they are specifying it as -XmxsizeMB example -Xmx2048m vs -Xmx2g Is it possible that it's not reading the 2g variable as 2GB? Subject: RE: Interesting 4.2.1. Issue... From: a...@opencloud.net.au To: dev@cloudstack.apache.org Date: Thu, 3 Apr 2014 09:06:17 +0800 It is 6.0.35 and it still produces this error, even after increasing the Xmx to 4g, we have installed tomcat 6.0.33 and each time we install the cloudstack it does not sense the installed 6.0.33 and attempts to install 6.0.35 as it is dependent on it. Silly solution is that we scheduled a daily restart @ 2PM through a cron job But first you have to killall jvsc then restart the management server. What we are considering is to migrate the management server to CentOS 6.5 it comes with tomcat 6.0.24 and mysql 5.1, we attempted to restore the dump on a pilot environment and it worked. If someone else has a better solution than this would you please share it? Kind Regards Amin -Original Message- From: Michael Phillips [mailto:mphilli7...@hotmail.com] Sent: Thursday, 3 April 2014 5:31 AM To: dev@cloudstack.apache.org Subject: RE: Interesting 4.2.1. Issue... So I just checked my tomcat version and we are running 6.0.35 which must be the default that comes with Ubuntu 12.04 out of the box. Our memory settings are as follows: JAVA_OPTS=-Djava.awt.headless=true -Dcom.sun.management.jmxremote.port=45219 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Xmx4g -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/log/cloudstack/management/ -XX:PermSize=512M -XX:MaxPermSize=800m So what version of tomcat are you running 6.0.35 or 6.0.33? Subject: RE: Interesting 4.2.1. Issue... From: a...@opencloud.net.au To: dev@cloudstack.apache.org Date: Tue, 1 Apr 2014 11:23:57 +0800 We have the same issue, after an upgrade from 2.2.14 to 4.2.1, and during this upgrade we had upgrade from Ubuntu 10 LTS to Ubuntu 12 LTS, it seems it related to tomcat 6.0.35, because it is recommended to have tomcat 6.0.33 which doesn't come by default with Ubuntu 12.04.4 Kind Regards Amin -Original Message- From: Marcus [mailto:shadow...@gmail.com] Sent: Tuesday, 1 April 2014 6:22 AM To: dev@cloudstack.apache.org Subject: Re: Interesting 4.2.1. Issue... I'm running 3 mgmt servers on 4.2.1, haven't seen any issues like that. You can send along your memory settings... here's what I'm running: JAVA_OPTS=-Djava.awt.headless=true -Dcom.sun.management.jmxremote.port=45219 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Xmx2g -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/log/cloudstack/management/ -XX:PermSize=512M -XX:MaxPermSize=800m On Mon, Mar 31, 2014 at 9:33 AM, Michael Phillips mphilli7...@hotmail.com wrote: So I have a redundant pair of management servers running on 4.2.1. At least once a day one of the management servers crashes and the log gets
Cloud Stack 4.2.1 issue (Unable to start instance due to DB Exception)
Hello, We are facing an issue when we spin up a new instance with a new network (router VM), it gives the following error: Resource [Host:59] is unreachable: Host 59: Unable to start instance due to DB Exception on: com.mysql.jdbc.JDBC4PreparedStatement@5126808a mailto:com.mysql.jdbc.JDBC4PreparedStatement@5126808a : INSERT INTO user_statistics (user_statistics.data_center_id, user_statistics.account_id, user_statistics.public_ip_address, user_statistics.device_id, user_statistics.device_type, user_statistics.network_id, user_statistics.net_bytes_received, user_statistics.net_bytes_sent, user_statistics.current_bytes_received, user_statistics.current_bytes_sent, user_statistics.agg_bytes_received, user_statistics.agg_bytes_sent) VALUES (1, 28, null, 360, _binary'DomainRouter', 237, 0, 0, 0, 0, 0, 0) Also we are facing this error every now and then: (DirectAgent-93:null) GetStorageStats Exception:SR_BACKEND_FAILURE_40The SR scan failed [opterr=uuid=870ed96e-d62b-428a-88dc-41afc75d311a]host:13944281-3f7e-4065-8580-1dc5b64cfdb1storageid: 098d4f7a-a040-3462-add1-d618d6ec53c1 However when spinning a new instance with an existing network (router VM) it works fine!! Has anyone faced this issue, your help would be much appreciated. Thanks Kind Regards Amin
RE: Interesting 4.2.1. Issue...
We have the same issue, after an upgrade from 2.2.14 to 4.2.1, and during this upgrade we had upgrade from Ubuntu 10 LTS to Ubuntu 12 LTS, it seems it related to tomcat 6.0.35, because it is recommended to have tomcat 6.0.33 which doesn't come by default with Ubuntu 12.04.4 Kind Regards Amin -Original Message- From: Marcus [mailto:shadow...@gmail.com] Sent: Tuesday, 1 April 2014 6:22 AM To: dev@cloudstack.apache.org Subject: Re: Interesting 4.2.1. Issue... I'm running 3 mgmt servers on 4.2.1, haven't seen any issues like that. You can send along your memory settings... here's what I'm running: JAVA_OPTS=-Djava.awt.headless=true -Dcom.sun.management.jmxremote.port=45219 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Xmx2g -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/log/cloudstack/management/ -XX:PermSize=512M -XX:MaxPermSize=800m On Mon, Mar 31, 2014 at 9:33 AM, Michael Phillips mphilli7...@hotmail.com wrote: So I have a redundant pair of management servers running on 4.2.1. At least once a day one of the management servers crashes and the log gets filled with the following messages: java.lang.OutOfMemoryError: Java heap spac0java.lang.ArrayIndexOutOfBoundsExceptioneCaused by: java.io.EOFException: SSL peer shut down incorrectlyCaused by: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake and there are a few others. When the one management server tanks, although the other management server is up and active, it won't actually process any UI commands until the crashed server is restarted. Example of won't process any UI command is create a new instance, create a new account, etc. After doing some searching I have found that others have noticed java heap errors in 4.2.1, and the suggested fix is to increase the heap size. I am planning on increasing it from 2g to 4g, however if the problem is something like a memory leak, then increasing the heap size will just delay the inevitable. Has anyone else fixed this issue by increasing the heap size? Or what is the recommended value? **updated...as expected I increased the heap size from 2g to 4g, and it just too k longer for the problem to reoccur... In my honest opinion of bigger concern is the fact that when one management server crashes the other stops functioning as well. So this begs the question of why even bother with a redundant pair of servers..Anybody else experience this issue? I would love to hear any dev guys opinion on this
RE: Upgrade problems 3.0.7 - 4.2.1
Hello, We have faced the same issue before, it worked with the below steps probably you missed step no. 6 * log in to UI, select Templates, My Templates, Add Template and enter the following: Name: systemvm-xenserver-4.2 Description:systemvm-xenserver-4.2 URL: http://download.cloud.com/templates/4.2/systemvmtemplate-2013-07-12-master-xen.vhd.bz2 Zone: Choose the zone where this hypervisor is used Hypervisor: XenServer Format: VHD OS Type:Debian GNU/Linux 7.0 (32-bit) (or the highest Debian release number available in the dropdown) Extractable:no Password Enabled: no Public: no Featured: no 1) service cloud-usage stop 2) service cloud-management stop 3) mysqldump -u root -pyourpassword cloud cloud-backup.sql 4) mysqldump -u root -pyourpassword cloud_usage cloud-usage-backup.sql 5) mysql -u root -pyourpassword 6) delete from cloud.resource_count;exit 7) vi /etc/apt/sources.list 8) paste the following deb http://cloudstack.apt-get.eu/ubuntu precise 4.2 9) Esc, :wq, Enter 10) wget -O - http://cloudstack.apt-get.eu/release.asc|apt-key add - 11) apt-get update 12) apt-get install cloudstack-management ** cd /usr/share/cloudstack-common/scripts/vm/hypervisor/xenserver ** wget http://download.cloud.com.s3.amazonaws.com/tools/vhd-util ** cd /usr/share/cloudstack-management/webapps/client/WEB-INF/classes/scripts/vm/hypervisor/xenserver/ ** wget http://download.cloud.com.s3.amazonaws.com/tools/vhd-util On each Hypervisor cd /opt/cloud/bin wget http://download.cloud.com.s3.amazonaws.com/tools/vhd-util chmod 777 vhd-util 13) apt-get install cloudstack-usage 14) cloudstack-setup-encryption -m yourkey -k yourkey 15) service cloudstack-management start 16) When the UI becomes accessible log in, Click Domains, then click the ROOT domain. Click the Update Resource Count button. This will refresh the resource count and generate the table deleted in step 6. 17) service cloudstack-usage start - Make Sure all hosts are UP - 18) nohup cloudstack-sysvmadm -d localhost -u cloud -pyourpassword -a sysvm.log 21 (or stop manually all sys vm's) --- tail -f sysvm.log Kind Regards Amin -Original Message- From: Paul Angus [mailto:paul.an...@shapeblue.com] Sent: Wednesday, 26 March 2014 11:56 PM To: dev@cloudstack.apache.org Subject: RE: Upgrade problems 3.0.7 - 4.2.1 Can you look in the database - is there more than one entry for 3.0.7 in the version table? A successful upgrade from 3.0.7 to 4.2.1. looks like this... # id, version, updated, step '1', '3.0.7.20140103104106', '2014-03-24 17:53:44', 'Complete' '2', '4.1.0', '2014-03-26 12:03:33', 'Complete' '3', '4.2.0', '2014-03-26 12:03:33', 'Complete' '4', '4.2.1', '2014-03-26 12:03:33', 'Complete' Regards, Paul Angus Cloud Architect S: +44 20 3603 0540 | M: +447711418784 | T: @CloudyAngus paul.an...@shapeblue.com -Original Message- From: Alex Hitchins [mailto:alex.hitch...@shapeblue.com] Sent: 26 March 2014 15:25 To: dev@cloudstack.apache.org Subject: RE: Upgrade problems 3.0.7 - 4.2.1 Ignore my last comment - I now see the -- prefx! Regards Alex Hitchins D: +44 1892 523 587 | S: +44 2036 030 540tel:+442036030540 | M: +44tel:+447968161581 7788 423 969 alex.hitch...@shapeblue.commailto:alex.hitch...@shapeblue.com From: Geoff Higginbottom [mailto:geoff.higginbot...@shapeblue.com] Sent: 26 March 2014 15:10 To: dev@cloudstack.apache.org Subject: Upgrade problems 3.0.7 - 4.2.1 Hi, I am trying to upgrade a CloudPlatfom 3.0.7 build to Apache CloudStack 4.2.1 The DB upgrade keeps failing with the logs highlighting a 'system integrity check failure' Any help would be greatly appreciated 2014-03-26 14:53:31,883 DEBUG [utils.db.ScriptRunner] (Timer-2:null) -- http://www.apache.org/licenses/LICENSE-2.0 2014-03-26 14:53:31,883 DEBUG [utils.db.ScriptRunner] (Timer-2:null) -- 2014-03-26 14:53:31,883 DEBUG [utils.db.ScriptRunner] (Timer-2:null) -- Unless required by applicable law or agreed to in writing, 2014-03-26 14:53:31,884 DEBUG [utils.db.ScriptRunner] (Timer-2:null) -- software distributed under the License is distributed on an 2014-03-26 14:53:31,884 DEBUG [utils.db.ScriptRunner] (Timer-2:null) -- AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY 2014-03-26 14:53:31,884 DEBUG [utils.db.ScriptRunner] (Timer-2:null) -- KIND, either express or implied. See the License for the 2014-03-26 14:53:31,884 DEBUG [utils.db.ScriptRunner] (Timer-2:null) -- specific language governing permissions and limitations 2014-03-26 14:53:31,884 DEBUG [utils.db.ScriptRunner] (Timer-2:null) -- under the License. 2014-03-26 14:53:31,884 DEBUG [utils.db.ScriptRunner] (Timer-2:null) INSERT IGNORE INTO `cloud
Java heap space error in cloud stack 4.2.1
Hello, I was advised to seek help in the dev community, we have upgraded our cloud stack management server from 2.2.14 to 4.2.1 and after the upgrade we face the following: 1) The management server crashes on daily basis and only a reboot fixes it. 2) The management server log output heaps of this error : a. 2014-03-24 15:11:39,509 WARN [xen.resource.CitrixResourceBase] (DirectAgent-445:null) Error while collecting disk stats from : You gave an invalid object reference. The object may have recently been deleted. The class parameter gives the type of reference given, and the handle parameter echoes the bad value given. ? ERROR [cloud.server.StatsCollector] (StatsCollector-1:null) Error trying to retrieve storage stats java.lang.OutOfMemoryError: Java heap space 3) The catalina log output the following: a. SEVERE: The web application [/client] appears to have started a thread named [Timer-0] but has failed to stop it. This is very likely to create a memory leak. I would much appreciate if anyone knows how to address this issue. Thanks in advance Kind Regards Amin
RE: Java heap space error in cloud stack 4.2.1
Thanks Chiradeep, will try and feedback with the results Kind Regards Amin -Original Message- From: Chiradeep Vittal [mailto:chiradeep.vit...@citrix.com] Sent: Tuesday, 25 March 2014 1:43 AM To: dev@cloudstack.apache.org Subject: Re: Java heap space error in cloud stack 4.2.1 Try increasing heap size in the JVM in /etc/cloudstack/management/tomcat6.conf E.g., -Xmx4g From: Amin Samir a...@opencloud.net.aumailto:a...@opencloud.net.au Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org Date: Monday, March 24, 2014 at 1:40 AM To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org Subject: Java heap space error in cloud stack 4.2.1 Hello, I was advised to seek help in the dev community, we have upgraded our cloud stack management server from 2.2.14 to 4.2.1 and after the upgrade we face the following: 1) The management server crashes on daily basis and only a reboot fixes it. 2) The management server log output heaps of this error : a. 2014-03-24 15:11:39,509 WARN [xen.resource.CitrixResourceBase] (DirectAgent-445:null) Error while collecting disk stats from : You gave an invalid object reference. The object may have recently been deleted. The class parameter gives the type of reference given, and the handle parameter echoes the bad value given. ? ERROR [cloud.server.StatsCollector] (StatsCollector-1:null) Error trying to retrieve storage stats java.lang.OutOfMemoryError: Java heap space 3) The catalina log output the following: a. SEVERE: The web application [/client] appears to have started a thread named [Timer-0] but has failed to stop it. This is very likely to create a memory leak. I would much appreciate if anyone knows how to address this issue. Thanks in advance Kind Regards Amin
