Hi @all, i have a problem with a router. Only this one router do not start. All other works fine! I use KVM with RH6.4 and Cloudstack 4.1.1. All other VMs, Routers and SystemVMs running normal. I try the server in maintainance mode reboot and try again to start the router. Sadly with no positive results.
Ntwk 204 = My Domainrouter Network Ntwk 202 = Is the Link Local Network Only one Host exist. KVM Host 87 is up and work. That is my short error log - tail -f -n0 /var/log/cloudstack/management/management-server.log | grep -Ei 'error|warn' 2013-09-09 12:22:16,088 WARN [network.element.VpcVirtualRouterElement] (Job-Executor-90:job-198) Network Ntwk[204|Guest|7] is not associated with any VPC 2013-09-09 12:22:16,256 WARN [network.element.VpcVirtualRouterElement] (Job-Executor-90:job-198) Network Ntwk[202|Control|3] is not associated with any VPC 2013-09-09 12:22:16,496 ERROR [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-90:job-198) Failed to start instance VM[DomainRouter|r-4658-VM] 2013-09-09 12:22:17,496 ERROR [cloud.async.AsyncJobManagerImpl] (Job-Executor-90:job-198) Unexpected exception while executing org.apache.cloudstack.api.command.admin.router.StartRouterCmd 2013-09-09 12:22:17,497 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-90:job-198) Complete async job-198, jobStatus: 2, resultCode: 530, result: Error Code: 530 Error text: Resource [Host:87] is unreachable: Host 87: Unable to start instance due to null and this is my extendet log filtered by job [root@cloud1 ~]# cat /var/log/cloudstack/management/management-server.log | grep Job-Executor-95:job-202 2013-09-09 13:12:15,246 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-95:job-202) Executing org.apache.cloudstack.api.command.admin.router.StartRouterCmd for job-202 2013-09-09 13:12:15,331 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Starting network Ntwk[204|Guest|7]... 2013-09-09 13:12:15,334 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Lock is acquired for network id 204 as a part of network implement 2013-09-09 13:12:15,334 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Network id=204 is already implemented 2013-09-09 13:12:15,334 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Lock is released for network id 204 as a part of network implement 2013-09-09 13:12:15,336 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Starting network Ntwk[202|Control|3]... 2013-09-09 13:12:15,338 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Lock is acquired for network id 202 as a part of network implement 2013-09-09 13:12:15,338 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Network id=202 is already implemented 2013-09-09 13:12:15,338 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Lock is released for network id 202 as a part of network implement 2013-09-09 13:12:15,340 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Starting network Ntwk[200|Public|1]... 2013-09-09 13:12:15,342 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Lock is acquired for network id 200 as a part of network implement 2013-09-09 13:12:15,342 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Network id=200 is already implemented 2013-09-09 13:12:15,342 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Lock is released for network id 200 as a part of network implement 2013-09-09 13:12:15,344 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (Job-Executor-95:job-202) Starting router VM[DomainRouter|r-4658-VM] 2013-09-09 13:12:15,349 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-95:job-202) VM state transitted from :Stopped to Starting with event: StartRequestedvm's original host id: null new host id: null host id before state transition: null 2013-09-09 13:12:15,349 DEBUG [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-95:job-202) Successfully transitioned to start state for VM[DomainRouter|r-4658-VM] reservation id = 2459a261-a451-42c1-bd6b-0a71285d23f8 2013-09-09 13:12:15,405 DEBUG [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-95:job-202) Trying to deploy VM, vm has dcId: 1 and podId: 1 2013-09-09 13:12:15,405 DEBUG [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-95:job-202) Deploy avoids pods: null, clusters: null, hosts: null 2013-09-09 13:12:15,408 DEBUG [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-95:job-202) Root volume is ready, need to place VM in volume's cluster 2013-09-09 13:12:15,408 DEBUG [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-95:job-202) Vol[4711|vm=4658|ROOT] is READY, changing deployment plan to use this pool's dcId: 1 , podId: 1 , and clusterId: 1 2013-09-09 13:12:15,409 DEBUG [cloud.deploy.FirstFitPlanner] (Job-Executor-95:job-202) DeploymentPlanner allocation algorithm: random 2013-09-09 13:12:15,409 DEBUG [cloud.deploy.FirstFitPlanner] (Job-Executor-95:job-202) Trying to allocate a host and storage pools from dc:1, pod:1,cluster:1, requested cpu: 500, requested ram: 134217728 2013-09-09 13:12:15,410 DEBUG [cloud.deploy.FirstFitPlanner] (Job-Executor-95:job-202) Is ROOT volume READY (pool already allocated)?: Yes 2013-09-09 13:12:15,410 DEBUG [cloud.deploy.FirstFitPlanner] (Job-Executor-95:job-202) Searching resources only under specified Cluster: 1 2013-09-09 13:12:15,417 DEBUG [cloud.deploy.FirstFitPlanner] (Job-Executor-95:job-202) Checking resources in Cluster: 1 under Pod: 1 2013-09-09 13:12:15,421 DEBUG [allocator.impl.FirstFitAllocator] (Job-Executor-95:job-202 FirstFitRoutingAllocator) Looking for hosts in dc: 1 pod:1 cluster:1 2013-09-09 13:12:15,423 DEBUG [allocator.impl.FirstFitAllocator] (Job-Executor-95:job-202 FirstFitRoutingAllocator) FirstFitAllocator has 1 hosts to check for allocation: [Host[-87-Routing]] 2013-09-09 13:12:15,426 DEBUG [allocator.impl.FirstFitAllocator] (Job-Executor-95:job-202 FirstFitRoutingAllocator) Found 1 hosts for allocation after prioritization: [Host[-87-Routing]] 2013-09-09 13:12:15,426 DEBUG [allocator.impl.FirstFitAllocator] (Job-Executor-95:job-202 FirstFitRoutingAllocator) Looking for speed=500Mhz, Ram=128 2013-09-09 13:12:15,429 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-95:job-202 FirstFitRoutingAllocator) Checking if host: 87 has enough capacity for requested CPU: 500 and requested RAM: 134217728 , cpuOverprovisioningFactor: 9.0 2013-09-09 13:12:15,432 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-95:job-202 FirstFitRoutingAllocator) Hosts's actual total CPU: 20888 and CPU after applying overprovisioning: 187992 2013-09-09 13:12:15,432 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-95:job-202 FirstFitRoutingAllocator) Free CPU: 173592 , Requested CPU: 500 2013-09-09 13:12:15,432 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-95:job-202 FirstFitRoutingAllocator) Free RAM: 21922451456 , Requested RAM: 134217728 2013-09-09 13:12:15,432 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-95:job-202 FirstFitRoutingAllocator) Host has enough CPU and RAM available 2013-09-09 13:12:15,432 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-95:job-202 FirstFitRoutingAllocator) STATS: Can alloc CPU from host: 87, used: 8200, reserved: 6200, actual total: 20888, total with overprovisioning: 187992; requested cpu:500,alloc_from_last_host?:false ,considerReservedCapacity?: true 2013-09-09 13:12:15,432 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-95:job-202 FirstFitRoutingAllocator) STATS: Can alloc MEM from host: 87, used: 6476005376, reserved: 5268045824, total: 33666502656; requested mem: 134217728,alloc_from_last_host?:false ,considerReservedCapacity?: true 2013-09-09 13:12:15,432 DEBUG [allocator.impl.FirstFitAllocator] (Job-Executor-95:job-202 FirstFitRoutingAllocator) Found a suitable host, adding to list: 87 2013-09-09 13:12:15,432 DEBUG [allocator.impl.FirstFitAllocator] (Job-Executor-95:job-202 FirstFitRoutingAllocator) Host Allocator returning 1 suitable hosts 2013-09-09 13:12:15,434 DEBUG [cloud.deploy.FirstFitPlanner] (Job-Executor-95:job-202) Checking suitable pools for volume (Id, Type): (4711,ROOT) 2013-09-09 13:12:15,434 DEBUG [cloud.deploy.FirstFitPlanner] (Job-Executor-95:job-202) Volume has pool(204) already allocated, checking if pool can be reused, poolId: 204 2013-09-09 13:12:15,434 DEBUG [cloud.deploy.FirstFitPlanner] (Job-Executor-95:job-202) finding pool by id '204' 2013-09-09 13:12:15,435 DEBUG [cloud.deploy.FirstFitPlanner] (Job-Executor-95:job-202) Planner need not allocate a pool for this volume since its READY 2013-09-09 13:12:15,435 DEBUG [cloud.deploy.FirstFitPlanner] (Job-Executor-95:job-202) Trying to find a potenial host and associated storage pools from the suitable host/pool lists for this VM 2013-09-09 13:12:15,435 DEBUG [cloud.deploy.FirstFitPlanner] (Job-Executor-95:job-202) Checking if host: 87 can access any suitable storage pool for volume: ROOT 2013-09-09 13:12:15,437 DEBUG [cloud.deploy.FirstFitPlanner] (Job-Executor-95:job-202) Host: 87 can access pool: 204 2013-09-09 13:12:15,445 DEBUG [cloud.deploy.FirstFitPlanner] (Job-Executor-95:job-202) Found a potential host id: 87 name: sv01sfogaa.lincloud.de and associated storage pools for this VM 2013-09-09 13:12:15,447 DEBUG [cloud.deploy.FirstFitPlanner] (Job-Executor-95:job-202) Returning Deployment Destination: Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))] : Dest[Zone(1)-Pod(1)-Cluster(1)-Host(87)-Storage()] 2013-09-09 13:12:15,447 DEBUG [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-95:job-202) Deployment found - P0=VM[DomainRouter|r-4658-VM], P0=Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))] : Dest[Zone(1)-Pod(1)-Cluster(1)-Host(87)-Storage()] 2013-09-09 13:12:15,530 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-95:job-202) VM state transitted from :Starting to Starting with event: OperationRetryvm's original host id: null new host id: 87 host id before state transition: null 2013-09-09 13:12:15,537 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-95:job-202) Hosts's actual total CPU: 20888 and CPU after applying overprovisioning: 187992 2013-09-09 13:12:15,537 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-95:job-202) We are allocating VM, increasing the used capacity of this host:87 2013-09-09 13:12:15,538 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-95:job-202) Current Used CPU: 8200 , Free CPU:173592 ,Requested CPU: 500 2013-09-09 13:12:15,538 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-95:job-202) Current Used RAM: 6476005376 , Free RAM:21922451456 ,Requested RAM: 134217728 2013-09-09 13:12:15,538 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-95:job-202) CPU STATS after allocation: for host: 87, old used: 8200, old reserved: 6200, actual total: 20888, total with overprovisioning: 187992; new used:8700, reserved:6200; requested cpu:500,alloc_from_last:false 2013-09-09 13:12:15,538 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-95:job-202) RAM STATS after allocation: for host: 87, old used: 6476005376, old reserved: 5268045824, total: 33666502656; new used: 6610223104, reserved: 5268045824; requested mem: 134217728,alloc_from_last:false 2013-09-09 13:12:15,590 DEBUG [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-95:job-202) VM is being created in podId: 1 2013-09-09 13:12:15,594 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Lock is acquired for network id 200 as a part of network implement 2013-09-09 13:12:15,594 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Network id=200 is already implemented 2013-09-09 13:12:15,595 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Lock is released for network id 200 as a part of network implement 2013-09-09 13:12:15,662 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking NiciraNvp to prepare for Nic[18229-4658-null-10.140.1.124] 2013-09-09 13:12:15,663 DEBUG [network.element.NiciraNvpElement] (Job-Executor-95:job-202) Checking if NiciraNvpElement can handle service Connectivity on network null 2013-09-09 13:12:15,663 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking JuniperSRX to prepare for Nic[18229-4658-null-10.140.1.124] 2013-09-09 13:12:15,663 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking Netscaler to prepare for Nic[18229-4658-null-10.140.1.124] 2013-09-09 13:12:15,663 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking F5BigIP to prepare for Nic[18229-4658-null-10.140.1.124] 2013-09-09 13:12:15,663 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking CiscoNexus1000vVSM to prepare for Nic[18229-4658-null-10.140.1.124] 2013-09-09 13:12:15,663 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking BigSwitchVnsElement to prepare for Nic[18229-4658-null-10.140.1.124] 2013-09-09 13:12:15,663 DEBUG [network.element.BigSwitchVnsElement] (Job-Executor-95:job-202) Checking if BigSwitchVnsElement can handle service Connectivity on network null 2013-09-09 13:12:15,663 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking VirtualRouter to prepare for Nic[18229-4658-null-10.140.1.124] 2013-09-09 13:12:15,663 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking Ovs to prepare for Nic[18229-4658-null-10.140.1.124] 2013-09-09 13:12:15,663 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking SecurityGroupProvider to prepare for Nic[18229-4658-null-10.140.1.124] 2013-09-09 13:12:15,663 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking VpcVirtualRouter to prepare for Nic[18229-4658-null-10.140.1.124] 2013-09-09 13:12:15,663 WARN [network.element.VpcVirtualRouterElement] (Job-Executor-95:job-202) Network Ntwk[200|Public|1] is not associated with any VPC 2013-09-09 13:12:15,665 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Lock is acquired for network id 204 as a part of network implement 2013-09-09 13:12:15,666 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Network id=204 is already implemented 2013-09-09 13:12:15,666 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Lock is released for network id 204 as a part of network implement 2013-09-09 13:12:15,742 DEBUG [cloud.network.NetworkModelImpl] (Job-Executor-95:job-202) Service SecurityGroup is not supported in the network id=204 2013-09-09 13:12:15,801 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking NiciraNvp to prepare for Nic[18227-4658-2459a261-a451-42c1-bd6b-0a71285d23f8-10.140.10.1] 2013-09-09 13:12:15,801 DEBUG [network.element.NiciraNvpElement] (Job-Executor-95:job-202) Checking if NiciraNvpElement can handle service Connectivity on network 7Five Intern 2013-09-09 13:12:15,801 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking JuniperSRX to prepare for Nic[18227-4658-2459a261-a451-42c1-bd6b-0a71285d23f8-10.140.10.1] 2013-09-09 13:12:15,801 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking Netscaler to prepare for Nic[18227-4658-2459a261-a451-42c1-bd6b-0a71285d23f8-10.140.10.1] 2013-09-09 13:12:15,801 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking F5BigIP to prepare for Nic[18227-4658-2459a261-a451-42c1-bd6b-0a71285d23f8-10.140.10.1] 2013-09-09 13:12:15,801 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking CiscoNexus1000vVSM to prepare for Nic[18227-4658-2459a261-a451-42c1-bd6b-0a71285d23f8-10.140.10.1] 2013-09-09 13:12:15,801 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking BigSwitchVnsElement to prepare for Nic[18227-4658-2459a261-a451-42c1-bd6b-0a71285d23f8-10.140.10.1] 2013-09-09 13:12:15,801 DEBUG [network.element.BigSwitchVnsElement] (Job-Executor-95:job-202) Checking if BigSwitchVnsElement can handle service Connectivity on network 7Five Intern 2013-09-09 13:12:15,801 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking VirtualRouter to prepare for Nic[18227-4658-2459a261-a451-42c1-bd6b-0a71285d23f8-10.140.10.1] 2013-09-09 13:12:15,801 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking Ovs to prepare for Nic[18227-4658-2459a261-a451-42c1-bd6b-0a71285d23f8-10.140.10.1] 2013-09-09 13:12:15,801 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking SecurityGroupProvider to prepare for Nic[18227-4658-2459a261-a451-42c1-bd6b-0a71285d23f8-10.140.10.1] 2013-09-09 13:12:15,801 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking VpcVirtualRouter to prepare for Nic[18227-4658-2459a261-a451-42c1-bd6b-0a71285d23f8-10.140.10.1] 2013-09-09 13:12:15,801 WARN [network.element.VpcVirtualRouterElement] (Job-Executor-95:job-202) Network Ntwk[204|Guest|7] is not associated with any VPC 2013-09-09 13:12:15,803 DEBUG [cloud.network.NetworkModelImpl] (Job-Executor-95:job-202) Service SecurityGroup is not supported in the network id=204 2013-09-09 13:12:15,805 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Lock is acquired for network id 202 as a part of network implement 2013-09-09 13:12:15,805 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Network id=202 is already implemented 2013-09-09 13:12:15,806 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Lock is released for network id 202 as a part of network implement 2013-09-09 13:12:15,985 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking NiciraNvp to prepare for Nic[18228-4658-2459a261-a451-42c1-bd6b-0a71285d23f8-169.254.1.80] 2013-09-09 13:12:15,985 DEBUG [network.element.NiciraNvpElement] (Job-Executor-95:job-202) Checking if NiciraNvpElement can handle service Connectivity on network null 2013-09-09 13:12:15,985 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking JuniperSRX to prepare for Nic[18228-4658-2459a261-a451-42c1-bd6b-0a71285d23f8-169.254.1.80] 2013-09-09 13:12:15,985 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking Netscaler to prepare for Nic[18228-4658-2459a261-a451-42c1-bd6b-0a71285d23f8-169.254.1.80] 2013-09-09 13:12:15,985 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking F5BigIP to prepare for Nic[18228-4658-2459a261-a451-42c1-bd6b-0a71285d23f8-169.254.1.80] 2013-09-09 13:12:15,985 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking CiscoNexus1000vVSM to prepare for Nic[18228-4658-2459a261-a451-42c1-bd6b-0a71285d23f8-169.254.1.80] 2013-09-09 13:12:15,985 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking BigSwitchVnsElement to prepare for Nic[18228-4658-2459a261-a451-42c1-bd6b-0a71285d23f8-169.254.1.80] 2013-09-09 13:12:15,985 DEBUG [network.element.BigSwitchVnsElement] (Job-Executor-95:job-202) Checking if BigSwitchVnsElement can handle service Connectivity on network null 2013-09-09 13:12:15,985 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking VirtualRouter to prepare for Nic[18228-4658-2459a261-a451-42c1-bd6b-0a71285d23f8-169.254.1.80] 2013-09-09 13:12:15,985 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking Ovs to prepare for Nic[18228-4658-2459a261-a451-42c1-bd6b-0a71285d23f8-169.254.1.80] 2013-09-09 13:12:15,985 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking SecurityGroupProvider to prepare for Nic[18228-4658-2459a261-a451-42c1-bd6b-0a71285d23f8-169.254.1.80] 2013-09-09 13:12:15,985 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking VpcVirtualRouter to prepare for Nic[18228-4658-2459a261-a451-42c1-bd6b-0a71285d23f8-169.254.1.80] 2013-09-09 13:12:15,985 WARN [network.element.VpcVirtualRouterElement] (Job-Executor-95:job-202) Network Ntwk[202|Control|3] is not associated with any VPC 2013-09-09 13:12:15,987 DEBUG [cloud.storage.StorageManagerImpl] (Job-Executor-95:job-202) Checking if we need to prepare 1 volumes for VM[DomainRouter|r-4658-VM] 2013-09-09 13:12:15,987 DEBUG [cloud.storage.StorageManagerImpl] (Job-Executor-95:job-202) No need to recreate the volume: Vol[4711|vm=4658|ROOT], since it already has a pool assigned: 204, adding disk to VM 2013-09-09 13:12:15,997 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (Job-Executor-95:job-202) Boot Args for VM[DomainRouter|r-4658-VM]: template=domP name=r-4658-VM eth2ip=10.140.1.124 eth2mask=255.255.255.0 gateway=10.140.1.254 eth0ip=10.140.10.1 eth0mask=255.255.255.0 domain=Lincloud dhcprange=10.140.10.1 eth1ip=169.254.1.80 eth1mask=255.255.0.0 type=router disable_rp_filter=true dns1=192.168.10.1 dns2= 2013-09-09 13:12:16,096 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (Job-Executor-95:job-202) Found 8 ip(s) to apply as a part of domR VM[DomainRouter|r-4658-VM] start. 2013-09-09 13:12:16,134 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (Job-Executor-95:job-202) Resending ipAssoc, port forwarding, load balancing rules as a part of Virtual router start 2013-09-09 13:12:16,164 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (Job-Executor-95:job-202) Found 1 firewall Egress rule(s) to apply as a part of domR VM[DomainRouter|r-4658-VM] start. 2013-09-09 13:12:16,221 ERROR [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-95:job-202) Failed to start instance VM[DomainRouter|r-4658-VM] 2013-09-09 13:12:16,404 DEBUG [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-95:job-202) Cleaning up resources for the vm VM[DomainRouter|r-4658-VM] in Starting state 2013-09-09 13:12:16,406 DEBUG [agent.transport.Request] (Job-Executor-95:job-202) Seq 87-2102076304: Sending { Cmd , MgmtId: 87484702506, via: 87, Ver: v1, Flags: 100111, [{"StopCommand":{"isProxy":false,"vmName":"r-4658-VM","wait":0}}] } 2013-09-09 13:12:16,629 DEBUG [agent.transport.Request] (Job-Executor-95:job-202) Seq 87-2102076304: Received: { Ans: , MgmtId: 87484702506, via: 87, Ver: v1, Flags: 110, { StopAnswer } } 2013-09-09 13:12:16,635 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (Job-Executor-95:job-202) Successfully updated user statistics as a part of domR VM[DomainRouter|r-4658-VM] reboot/stop 2013-09-09 13:12:16,667 DEBUG [cloud.network.NetworkModelImpl] (Job-Executor-95:job-202) Service SecurityGroup is not supported in the network id=204 2013-09-09 13:12:16,721 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking NiciraNvp to release Nic[18227-4658-2459a261-a451-42c1-bd6b-0a71285d23f8-10.140.10.1] 2013-09-09 13:12:16,721 DEBUG [network.element.NiciraNvpElement] (Job-Executor-95:job-202) Checking if NiciraNvpElement can handle service Connectivity on network 7Five Intern 2013-09-09 13:12:16,721 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking JuniperSRX to release Nic[18227-4658-2459a261-a451-42c1-bd6b-0a71285d23f8-10.140.10.1] 2013-09-09 13:12:16,721 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking Netscaler to release Nic[18227-4658-2459a261-a451-42c1-bd6b-0a71285d23f8-10.140.10.1] 2013-09-09 13:12:16,721 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking F5BigIP to release Nic[18227-4658-2459a261-a451-42c1-bd6b-0a71285d23f8-10.140.10.1] 2013-09-09 13:12:16,721 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking CiscoNexus1000vVSM to release Nic[18227-4658-2459a261-a451-42c1-bd6b-0a71285d23f8-10.140.10.1] 2013-09-09 13:12:16,721 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking BigSwitchVnsElement to release Nic[18227-4658-2459a261-a451-42c1-bd6b-0a71285d23f8-10.140.10.1] 2013-09-09 13:12:16,721 DEBUG [network.element.BigSwitchVnsElement] (Job-Executor-95:job-202) Checking if BigSwitchVnsElement can handle service Connectivity on network 7Five Intern 2013-09-09 13:12:16,721 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking VirtualRouter to release Nic[18227-4658-2459a261-a451-42c1-bd6b-0a71285d23f8-10.140.10.1] 2013-09-09 13:12:16,721 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking Ovs to release Nic[18227-4658-2459a261-a451-42c1-bd6b-0a71285d23f8-10.140.10.1] 2013-09-09 13:12:16,721 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking SecurityGroupProvider to release Nic[18227-4658-2459a261-a451-42c1-bd6b-0a71285d23f8-10.140.10.1] 2013-09-09 13:12:16,721 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking VpcVirtualRouter to release Nic[18227-4658-2459a261-a451-42c1-bd6b-0a71285d23f8-10.140.10.1] 2013-09-09 13:12:16,725 DEBUG [network.guru.ControlNetworkGuru] (Job-Executor-95:job-202) Released nic: NicProfile[18228-4658-null-null-null 2013-09-09 13:12:16,777 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking NiciraNvp to release Nic[18228-4658-2459a261-a451-42c1-bd6b-0a71285d23f8-null] 2013-09-09 13:12:16,777 DEBUG [network.element.NiciraNvpElement] (Job-Executor-95:job-202) Checking if NiciraNvpElement can handle service Connectivity on network null 2013-09-09 13:12:16,777 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking JuniperSRX to release Nic[18228-4658-2459a261-a451-42c1-bd6b-0a71285d23f8-null] 2013-09-09 13:12:16,777 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking Netscaler to release Nic[18228-4658-2459a261-a451-42c1-bd6b-0a71285d23f8-null] 2013-09-09 13:12:16,777 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking F5BigIP to release Nic[18228-4658-2459a261-a451-42c1-bd6b-0a71285d23f8-null] 2013-09-09 13:12:16,777 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking CiscoNexus1000vVSM to release Nic[18228-4658-2459a261-a451-42c1-bd6b-0a71285d23f8-null] 2013-09-09 13:12:16,777 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking BigSwitchVnsElement to release Nic[18228-4658-2459a261-a451-42c1-bd6b-0a71285d23f8-null] 2013-09-09 13:12:16,777 DEBUG [network.element.BigSwitchVnsElement] (Job-Executor-95:job-202) Checking if BigSwitchVnsElement can handle service Connectivity on network null 2013-09-09 13:12:16,777 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking VirtualRouter to release Nic[18228-4658-2459a261-a451-42c1-bd6b-0a71285d23f8-null] 2013-09-09 13:12:16,778 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking Ovs to release Nic[18228-4658-2459a261-a451-42c1-bd6b-0a71285d23f8-null] 2013-09-09 13:12:16,778 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking SecurityGroupProvider to release Nic[18228-4658-2459a261-a451-42c1-bd6b-0a71285d23f8-null] 2013-09-09 13:12:16,778 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-95:job-202) Asking VpcVirtualRouter to release Nic[18228-4658-2459a261-a451-42c1-bd6b-0a71285d23f8-null] 2013-09-09 13:12:16,835 DEBUG [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-95:job-202) Successfully released network resources for the vm VM[DomainRouter|r-4658-VM] 2013-09-09 13:12:16,836 DEBUG [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-95:job-202) Successfully cleanued up resources for the vm VM[DomainRouter|r-4658-VM] in Starting state 2013-09-09 13:12:16,934 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-95:job-202) VM state transitted from :Starting to Stopped with event: OperationFailedvm's original host id: null new host id: null host id before state transition: 87 2013-09-09 13:12:16,938 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-95:job-202) Hosts's actual total CPU: 20888 and CPU after applying overprovisioning: 187992 2013-09-09 13:12:16,939 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-95:job-202) release cpu from host: 87, old used: 8700,reserved: 6200, actual total: 20888, total with overprovisioning: 187992; new used: 8200,reserved:6200; movedfromreserved: false,moveToReserveredfalse 2013-09-09 13:12:16,939 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-95:job-202) release mem from host: 87, old used: 6610223104,reserved: 5268045824, total: 33666502656; new used: 6476005376,reserved:5268045824; movedfromreserved: false,moveToReserveredfalse 2013-09-09 13:12:17,060 ERROR [cloud.async.AsyncJobManagerImpl] (Job-Executor-95:job-202) Unexpected exception while executing org.apache.cloudstack.api.command.admin.router.StartRouterCmd 2013-09-09 13:12:17,061 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-95:job-202) Complete async job-202, jobStatus: 2, resultCode: 530, result: Error Code: 530 Error text: Resource [Host:87] is unreachable: Host 87: Unable to start instance due to null Regards Andreas Mit freundlichen Grüßen Andreas Huser Geschäftsführer System Engineer / Consultant (Cisco Certified Solaris, Linux) --------------------------------------- Zellerstraße 28 - 77654 Offenburg Tel: +49(781) 12786890 Mobil: +49(176) 10308549 ahu...@7five-edv.de