Hi
The story start with I try to setup direct attached network to the
guest network, in which I create a VM with the shared network I setup,
but the VM does not start successfully, so I tried to delete the guest
network I have created, in which I failed because of the vm was not
destroy, then I try to destroy the VM, but the VM stuck in the
explunging state:
The network setup was not resolved, I asked in this mailling list what
should I do if I have public network in the access point with advanced
network setup, they said I have to create a shared network offering
with Vlan, in which the public IP has no vlan setup, so I just typed
tagged in the vlan section,seems not working.
Below is the full story log start with I added a direct attached network:
2012-06-13 08:56:25,401 WARN [xen.resource.CitrixResourceBase]
(DirectAgent-45:null) Catch Exception: class
java.lang.NumberFormatException due to
java.lang.NumberFormatException: For input string: "tagged1"
java.lang.NumberFormatException: For input string: "tagged1"
at
java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
2012-06-13 08:56:25,402 WARN [xen.resource.CitrixResourceBase]
(DirectAgent-45:null) Unable to start i-2-415-VM due to
java.lang.NumberFormatException: For input string: "tagged1"
at
java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
2012-06-13 08:56:25,455 WARN [xen.resource.CitrixResourceBase]
(DirectAgent-45:null) Unable to clean up VBD due to
You gave an invalid object reference. The object may have recently
been deleted. The class parameter gives the type of reference given,
and the handle parameter echoes the bad value given.
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:930)
2012-06-13 08:56:25,466 WARN [xen.resource.CitrixResourceBase]
(DirectAgent-45:null) Unable to clean up VBD due to
You gave an invalid object reference. The object may have recently
been deleted. The class parameter gives the type of reference given,
and the handle parameter echoes the bad value given.
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:930)
2012-06-13 08:56:25,478 WARN [xen.resource.CitrixResourceBase]
(DirectAgent-45:null) Unable to clean up VBD due to
You gave an invalid object reference. The object may have recently
been deleted. The class parameter gives the type of reference given,
and the handle parameter echoes the bad value given.
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:930)
2012-06-13 08:56:25,489 WARN [xen.resource.CitrixResourceBase]
(DirectAgent-45:null) Unable to cleanup VIF
You gave an invalid object reference. The object may have recently
been deleted. The class parameter gives the type of reference given,
and the handle parameter echoes the bad value given.
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:938)
2012-06-13 08:56:25,515 INFO [xen.resource.XenServer56Resource]
(DirectAgent-45:null) Catch com.xensource.xenapi.Types$VifInUse:
failed to destory VLAN eth2 on host
b9f6241d-cbd6-49a8-916e-dff524569dd4 due to Network has active VIFs
2012-06-13 08:56:25,517 DEBUG [agent.transport.Request]
(DirectAgent-45:null) Seq 6-920389063: Processing: { Ans: , MgmtId:
229144658033003, via: 6, Ver: v1, Flags: 110,
[{"StartAnswer":{"vm":{"id":415,"name":"i-2-415-VM","bootloader":"PyGrub","type":"User","cpus":1,"speed":500,"minRam":536870912,"maxRam":536870912,"arch":"x86_64","os":"CentOS
5.3
(64-bit)","bootArgs":"","rebootOnCrash":false,"enableHA":false,"limitCpuUse":false,"vncPassword":"7bad48a63e4b14e7","params":{},"disks":[{"id":419,"name":"ROOT-415","mountPoint":"/iqn.2006-01.com.openfiler:tsn.2912cb92f82d/0","path":"ff3b7226-fd0b-4d0b-8af4-393496d7ee24","size":21474836480,"type":"ROOT","storagePoolType":"IscsiLUN","storagePoolUuid":"f7d6960c-84a6-332f-9068-27ef0f216b31","deviceId":0},{"id":420,"name":"DATA-415","mountPoint":"/iqn.2006-01.com.openfiler:tsn.2912cb92f82d/0","path":"202feab0-b724-4b70-b757-32f2b8dfddee","size":5368709120,"type":"DATADISK","storagePoolType":"IscsiLUN","storagePoolUuid":"f7d6960c-84a6-332f-9068-27ef0f216b31","deviceId":1},{"id":415,"name":"CentOS
5.6(64-bit) no GUI
(XenServer)","size":0,"type":"ISO","storagePoolType":"ISO","deviceId":3}],"nics":[{"deviceId":0,"networkRateMbps":200,"defaultNic":true,"ip":"10.1.1.99","netmask":"255.255.255.0","gateway":"10.1.1.1","mac":"02:00:2e:ae:00:04","dns1":"204.13.152.2","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://292","isolationUri":"vlan://292","isSecurityGroupEnabled":false,"name":"Pool-wide
network associated with
eth2"},{"deviceId":1,"networkRateMbps":200,"defaultNic":false,"ip":"46.136.134.86","netmask":"255.255.255.0","gateway":"46.136.134.1","mac":"06:4a:b4:00:08:3d","dns1":"204.13.152.2","broadcastType":"Vlan","type":"Guest","broadcastUri":"vlan://tagged1","isolationUri":"vlan://tagged1","isSecurityGroupEnabled":false,"name":"Pool-wide
network associated with eth2"}]},"result":false,"details":"Unable to
start i-2-415-VM due to ","wait":0}}] }
2012-06-13 08:56:25,517 WARN [cloud.vm.VirtualMachineManagerImpl]
(DirectAgent-45:null) Cleanup failed due to Unable to start i-2-415-VM
due to
2012-06-13 08:56:25,517 WARN [cloud.vm.VirtualMachineManagerImpl]
(Job-Executor-47:job-92) Cleanup failed due to Unable to start
i-2-415-VM due to
2012-06-13 08:56:25,590 INFO [cloud.vm.VirtualMachineManagerImpl]
(Job-Executor-47:job-92) Unable to start VM on Host[-6-Routing] due to
Unable to start i-2-415-VM due to
2012-06-13 08:56:26,159 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-47:job-92) VM state transitted from :Starting to Stopped
with event: OperationFailedvm's original host id: null new host id:
null host id before state transition: 6
2012-06-13 08:56:26,440 DEBUG [cloud.capacity.CapacityManagerImpl]
(Job-Executor-47:job-92) VM state transitted from :Stopped to Error
with event: OperationFailedToErrorvm's original host id: null new host
id: null host id before state transition: null
2012-06-13 08:56:27,209 INFO [api.commands.DeployVMCmd]
(Job-Executor-47:job-92)
com.cloud.exception.InsufficientServerCapacityException: Unable to
create a deployment for VM[User|i-2-415-VM]Scope=interface
com.cloud.dc.DataCenter; id=1
2012-06-13 08:56:27,210 WARN [cloud.api.ApiDispatcher]
(Job-Executor-47:job-92) class com.cloud.api.ServerApiException :
Unable to create a deployment for VM[User|i-2-415-VM]
2012-06-13 08:56:27,210 DEBUG [cloud.async.AsyncJobManagerImpl]
(Job-Executor-47:job-92) Complete async job-92, jobStatus: 2,
resultCode: 530, result:
com.cloud.api.response.ExceptionResponse@1660d637
2012-06-13 09:11:08,903 WARN [cloud.network.NetworkManagerImpl]
(Job-Executor-50:job-94) Can't delete the network, not all user vms
are expunged. Vm VM[User|i-2-413-VM] is in Error state
2012-06-13 09:11:08,993 WARN [cloud.api.ApiDispatcher]
(Job-Executor-50:job-94) class com.cloud.api.ServerApiException :
Failed to delete network
2012-06-13 09:11:08,993 DEBUG [cloud.async.AsyncJobManagerImpl]
(Job-Executor-50:job-94) Complete async job-94, jobStatus: 2,
resultCode: 530, result:
com.cloud.api.response.ExceptionResponse@2102e538
2012-06-13 09:11:09,144 WARN [cloud.async.AsyncJobManagerImpl]
(Job-Executor-50:job-94) Unable to unregister active job 94 from JMX
monitoring
2012-06-13 11:26:14,160 WARN [cloud.network.NetworkManagerImpl]
(Job-Executor-52:job-95) Can't delete the network, not all user vms
are expunged. Vm VM[User|i-2-413-VM] is in Error state
2012-06-13 11:26:14,243 WARN [cloud.api.ApiDispatcher]
(Job-Executor-52:job-95) class com.cloud.api.ServerApiException :
Failed to delete network
On Wed, Jun 13, 2012 at 10:46 PM, Alena Prokharchyk
<[email protected]> wrote:
> On 6/13/12 1:38 PM, "Lu Heng" <[email protected]> wrote:
>
>>Hi
>>
>>And btw, I didn't see the maintainace botton for the 3 shared storage,
>> I only see take snapshoot, snapshoot setting, that's it.
>>
>>On Wed, Jun 13, 2012 at 10:35 PM, Lu Heng <[email protected]> wrote:
>>> Hi
>>>
>>> See my reply
>>>
>>> On Wed, Jun 13, 2012 at 10:30 PM, Alena Prokharchyk
>>> <[email protected]> wrote:
>>>> On 6/13/12 1:24 PM, "Lu Heng" <[email protected]> wrote:
>>>>
>>>>>Hi
>>>>>
>>>>>I find the reason, three vm are in state of "Expunging" for 3 hours
>>>>>now, no process is being made, anyone has idea what should I do about
>>>>>these VMs？
>>>>
>>>>
>>>> What is expunge.interval in the setup? If the vms stuck in Expunging
>>>>state
>>>> with null Removed field, it indicates some problem happening during the
>>>> expunge. I suspect it's related with the fact you removed the storage
>>>>from
>>>> the xen.
>>>>
>>> Vise verse, the situation is, the VM in the expunging state for 3
>>> hours, and I find no way to delete them, so I try to delete the
>>> storage.
>
>
> So you deleted the VHDs per each volume, not the storage itself, please
> confirm. And we should never do it as there is no volume sync, so if the
> volume is removed on the backend, cloudStack has no idea it has happened.
>
> The original issue is - vm got stuck in Expunging state. We have to figure
> out this one first. Please send the logs starting from the time the
> Destroy command was executed for the Vm. And mention the vm name.
>
>
> -Alena.
>
>
>
>
>
>>>
>>> What should I do to fix this now?
>>>>
>>>>
>>>>>
>>>>>Another question is, I didn't see the delete botton with shared
>>>>>storage, so I delete it from the xencenter, but after that I still see
>>>>>the deleted storage exists in the UI, what suppose to be done?
>>>>
>>>>
>>>> Delete storage pool consists of 2 steps:
>>>>
>>>> * enable maintenance for the shared storage
>>>> * after the maintenance is completed, the Delete button will be
>>>>presented
>>>> to the admin; and the storage can be removed.
>>>>
>>>> You are not supposed to delete primary storage on the backend;
>>>>cloudStack
>>>> has no idea if you do things like that, and it expects primary storage
>>>>to
>>>> be functioning.
>>>>
>>>> So when vm expunge happens, we see that the storage is in UP state, and
>>>> try to send the destroy volume command to the Xen host. But the command
>>>> fails there because there is no such primary storage exist.
>>>>
>>>> Let me know if its your case.
>>>>
>>>> -Alena.
>>>>
>>>>
>>>>>
>>>>>On Wed, Jun 13, 2012 at 10:08 PM, Alena Prokharchyk
>>>>><[email protected]> wrote:
>>>>>> On 6/13/12 12:55 PM, "Lu Heng" <[email protected]> wrote:
>>>>>>
>>>>>>>Hi
>>>>>>>
>>>>>>>Update:
>>>>>>>
>>>>>>>The zone has been stoped. all VM has been destoryed , the route vm
>>>>>>>has
>>>>>>>been stoped(no option for destory).
>>>>>>>
>>>>>>>Still get error fail to delete network.
>>>>>>
>>>>>>
>>>>>> Can you send management server log file for debugging.
>>>>>>
>>>>>> Thanks,
>>>>>> Alena.
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>On Wed, Jun 13, 2012 at 7:23 PM, Alena Prokharchyk
>>>>>>><[email protected]> wrote:
>>>>>>>> On 6/13/12 10:18 AM, "Tamas Monos" <[email protected]> wrote:
>>>>>>>>
>>>>>>>>>Hi,
>>>>>>>>>
>>>>>>>>>Also I think the problem is when you are trying to delete the
>>>>>>>>>network
>>>>>>>>>the
>>>>>>>>>router is powered off.
>>>>>>>>>Therefore when you click delete it wants to remove the firewall
>>>>>>>>>rules
>>>>>>>>>for
>>>>>>>>>that network but the router is not running so the command fails
>>>>>>>>>which
>>>>>>>>>fails the delet operation.
>>>>>>>>
>>>>>>>>
>>>>>>>> Also not a problem. The rules can be removed when the router is
>>>>>>>>stopped.
>>>>>>>> In this case the rules are removed in DB only. When the router
>>>>>>>>starts
>>>>>>>>up,
>>>>>>>> we start clean (cleanup all the rules) and re-apply all the rules
>>>>>>>>from
>>>>>>>>the
>>>>>>>> DB. So the removed rules won't exist on the router.
>>>>>>>>
>>>>>>>> -Alena.
>>>>>>>>
>>>>>>>>>
>>>>>>>>>Regards
>>>>>>>>>
>>>>>>>>>Tamas Monos DDI
>>>>>>>>>+44(0)2034687012
>>>>>>>>>Chief Technical Office
>>>>>>>>>+44(0)2034687000
>>>>>>>>>Veber: The Hosting Specialists Fax +44(0)871
>>>>>>>>>522
>>>>>>>>>7057
>>>>>>>>>http://www.veber.co.uk
>>>>>>>>>
>>>>>>>>>Follow us on Twitter: www.twitter.com/veberhost
>>>>>>>>>Follow us on Facebook: www.facebook.com/veberhost
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>-----Original Message-----
>>>>>>>>>From: Tamas Monos [mailto:[email protected]]
>>>>>>>>>Sent: 13 June 2012 18:15
>>>>>>>>>To: [email protected]
>>>>>>>>>Subject: RE: Can not delete network
>>>>>>>>>
>>>>>>>>>Hi,
>>>>>>>>>
>>>>>>>>>I don't think you can delete a network if a router-vm for that
>>>>>>>>>network
>>>>>>>>>exists.
>>>>>>>>>If you are trying to delete a network that network should not have
>>>>>>>>>any
>>>>>>>>>VMs.
>>>>>>>>>Delete the associated router from the management interface then
>>>>>>>>>delete
>>>>>>>>>the network.
>>>>>>>>>This method works for me on 3.0.2
>>>>>>>>>
>>>>>>>>>Regards
>>>>>>>>>
>>>>>>>>>Tamas Monos DDI
>>>>>>>>>+44(0)2034687012
>>>>>>>>>Chief Technical Office
>>>>>>>>>+44(0)2034687000
>>>>>>>>>Veber: The Hosting Specialists Fax +44(0)871
>>>>>>>>>522
>>>>>>>>>7057
>>>>>>>>>http://www.veber.co.uk
>>>>>>>>>
>>>>>>>>>Follow us on Twitter: www.twitter.com/veberhost Follow us on
>>>>>>>>>Facebook:
>>>>>>>>>www.facebook.com/veberhost
>>>>>>>>>
>>>>>>>>>-----Original Message-----
>>>>>>>>>From: Alena Prokharchyk [mailto:[email protected]]
>>>>>>>>>Sent: 13 June 2012 18:12
>>>>>>>>>To: [email protected]
>>>>>>>>>Subject: Re: Can not delete network
>>>>>>>>>
>>>>>>>>>Can you please check the management server log to see if there is
>>>>>>>>>more
>>>>>>>>>detailed exception in there.
>>>>>>>>>
>>>>>>>>>-Alena.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>On 6/13/12 9:50 AM, "Lu Heng" <[email protected]> wrote:
>>>>>>>>>
>>>>>>>>>>Hi
>>>>>>>>>>
>>>>>>>>>>I have problem deleting guest network, everytime I tried it always
>>>>>>>>>>shows "fail to delete network)
>>>>>>>>>>
>>>>>>>>>>2012-06-13 11:26:14,243 WARN [cloud.api.ApiDispatcher]
>>>>>>>>>>(Job-Executor-52:job-95) class com.cloud.api.ServerApiException :
>>>>>>>>>>Failed to delete network
>>>>>>>>>>
>>>>>>>>>>Any idea how it happens?
>>>>>>>>>>--
>>>>>>>>>>--
>>>>>>>>>>Kind regards.
>>>>>>>>>>Lu
>>>>>>>>>>
>>>>>>>>>>This transmission is intended solely for the addressee(s) shown
>>>>>>>>>>above.
>>>>>>>>>>It may contain information that is privileged, confidential or
>>>>>>>>>>otherwise protected from disclosure. Any review, dissemination or
>>>>>>>>>>use
>>>>>>>>>>of this transmission or its contents by persons other than the
>>>>>>>>>>intended
>>>>>>>>>>addressee(s) is strictly prohibited. If you have received this
>>>>>>>>>>transmission in error, please notify this office immediately and
>>>>>>>>>>e-mail
>>>>>>>>>>the original at the sender's address above by replying to this
>>>>>>>>>>message
>>>>>>>>>>and including the text of the transmission received.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>--
>>>>>>>--
>>>>>>>Kind regards.
>>>>>>>Lu
>>>>>>>
>>>>>>>This transmission is intended solely for the addressee(s) shown
>>>>>>>above.
>>>>>>>It may contain information that is privileged, confidential or
>>>>>>>otherwise protected from disclosure. Any review, dissemination or use
>>>>>>>of this transmission or its contents by persons other than the
>>>>>>>intended addressee(s) is strictly prohibited. If you have received
>>>>>>>this transmission in error, please notify this office immediately and
>>>>>>>e-mail the original at the sender's address above by replying to this
>>>>>>>message and including the text of the transmission received.
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>>--
>>>>>--
>>>>>Kind regards.
>>>>>Lu
>>>>>
>>>>>This transmission is intended solely for the addressee(s) shown above.
>>>>>It may contain information that is privileged, confidential or
>>>>>otherwise protected from disclosure. Any review, dissemination or use
>>>>>of this transmission or its contents by persons other than the
>>>>>intended addressee(s) is strictly prohibited. If you have received
>>>>>this transmission in error, please notify this office immediately and
>>>>>e-mail the original at the sender's address above by replying to this
>>>>>message and including the text of the transmission received.
>>>>>
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> --
>>> Kind regards.
>>> Lu
>>>
>>> This transmission is intended solely for the addressee(s) shown above.
>>> It may contain information that is privileged, confidential or
>>> otherwise protected from disclosure. Any review, dissemination or use
>>> of this transmission or its contents by persons other than the
>>> intended addressee(s) is strictly prohibited. If you have received
>>> this transmission in error, please notify this office immediately and
>>> e-mail the original at the sender's address above by replying to this
>>> message and including the text of the transmission received.
>>
>>
>>
>>--
>>--
>>Kind regards.
>>Lu
>>
>>This transmission is intended solely for the addressee(s) shown above.
>>It may contain information that is privileged, confidential or
>>otherwise protected from disclosure. Any review, dissemination or use
>>of this transmission or its contents by persons other than the
>>intended addressee(s) is strictly prohibited. If you have received
>>this transmission in error, please notify this office immediately and
>>e-mail the original at the sender's address above by replying to this
>>message and including the text of the transmission received.
>>
>
>
--
--
Kind regards.
Lu
This transmission is intended solely for the addressee(s) shown above.
It may contain information that is privileged, confidential or
otherwise protected from disclosure. Any review, dissemination or use
of this transmission or its contents by persons other than the
intended addressee(s) is strictly prohibited. If you have received
this transmission in error, please notify this office immediately and
e-mail the original at the sender's address above by replying to this
message and including the text of the transmission received.
