in ACS 4.2.1 if you go to "Network" tab, click on the network you're having issues with, hit the "restart" button click on "clean network" it will remove current router and rebuilt a new one. please make sure to click on "restart" Network no on the X
hope this helps! On Thu, Aug 7, 2014 at 12:43 PM, Michael Lukzak <[email protected]> wrote: > I removed this host from CS, but still problem is. > > How I can "rebuild VR" from zero? > It's looks like CS want to start VM (witch really is a VirtualRouter) that > not exists on the host. > > INFO [xen.resource.CitrixResourceBase] (DirectAgent-148:) VM does not > exist on XenServer11681b28-5f38-4f20-aaae-0c9ed1e9ecc7 > > Or other way. How I can bind new network to VM? > Ony way is modify VM parameter in database? > > Michael > > > > > > > depending on your setup, but what I would do move all your Instances from > > host machine where the VR lives and reboot. Again, that's what I would > do. > > > you can try this link for XenServer 5 > > > http://blogs.citrix.com/2009/01/26/how-to-configure-an-nfs-based-iso-sr-using-local-storage-on-a-xenserver5-host/ > > > Thanks, > > > > On Thu, Aug 7, 2014 at 12:17 PM, Michael Lukzak <[email protected]> wrote: > > >> Hi, > > >> We are usning xenserver from Citrix. > > >> I found some clue. In console to server on this host I see that nfs has > >> error "Stale NFS file handle". > > >> How to remount NFS? Maybe this is a problem? > >> But why other VR are started and few of them not... ;( > > >> Best regards, > >> Michael > > > > >> > Hi Michael, > >> > if it helps, I would restart the agent on the host machine where VR > >> lives. > > >> > it depends on hypervisor but on KVM, following command: service > >> > cloudstack-agent restart > > > >> > Thanks, > >> > Motty > > > >> > On Thu, Aug 7, 2014 at 11:52 AM, Michael Lukzak <[email protected]> wrote: > > >> >> Hi, > > >> >> One of our host are died (after very high load). > >> >> Host has a lot of Virtual Routers running. > > >> >> We are using advanced networking. > > >> >> Only way to bring machine alive was hard reset. > > >> >> After this CloudStack decied to start all of VirtualRouterts, > >> >> when server was back online. > > >> >> Most of VR are started, but couple don't. > >> >> Cloudstack want to start this VM, but can't ;( > >> >> Here are log from this action. > > >> >> It's looks like that CS are looking for VR host on Xen host, but he > >> >> couldn't find. > > >> >> Please help how to repair this situation. > >> >> Restart network with clean up options ending with the same resoult. > > >> >> Is there any option to recreate VR? > >> >> I tried to remove VR from Infrastructure -> Virtual Router, but > there is > >> >> still problem > >> >> with create new one. > > > > >> >> Here are logs from "restart network with clean up options" > >> >> WARN [network.element.VpcVirtualRouterElement] > >> >> (Job-Executor-151:job-6526) Network Ntwk[200|Public|1] is not > associated > >> >> with any VPC > >> >> WARN [network.element.VpcVirtualRouterElement] > >> >> (Job-Executor-151:job-6526) Network Ntwk[338|Guest|8] is not > associated > >> >> with any VPC > >> >> WARN [network.element.VpcVirtualRouterElement] > >> >> (Job-Executor-151:job-6526) Network Ntwk[202|Control|3] is not > >> associated > >> >> with any VPC > >> >> ERROR [cloud.vm.VirtualMachineManagerImpl] > (Job-Executor-151:job-6526) > >> >> Failed to start instance VM[DomainRouter|r-1015-VM] > >> >> java.lang.NullPointerException > >> >> at > >> >> > >> > com.cloud.network.NetworkModelImpl.getIpInNetwork(NetworkModelImpl.java:763) > >> >> at > >> >> > >> > com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeNetworkRulesForNetwork(VirtualNetworkApplianceManagerImpl.java:2346) > >> >> at > >> >> > >> > com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.finalizeNetworkRulesForNetwork(VpcVirtualNetworkApplianceManagerImpl.java:928) > >> >> at > >> >> > >> > com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeCommandsOnStart(VirtualNetworkApplianceManagerImpl.java:2241) > >> >> at > >> >> > >> > com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.finalizeCommandsOnStart(VpcVirtualNetworkApplianceManagerImpl.java:767) > >> >> at > >> >> > >> > com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeDeployment(VirtualNetworkApplianceManagerImpl.java:2205) > >> >> at > >> >> > >> > com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:763) > >> >> at > >> >> > >> > com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:471) > >> >> at > >> >> > >> > com.cloud.network.router.VirtualNetworkApplianceManagerImpl.start(VirtualNetworkApplianceManagerImpl.java:2616) > >> >> at > >> >> > >> > com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startVirtualRouter(VirtualNetworkApplianceManagerImpl.java:1824) > >> >> at > >> >> > >> > com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRouters(VirtualNetworkApplianceManagerImpl.java:1924) > >> >> at > >> >> > >> > com.cloud.network.router.VirtualNetworkApplianceManagerImpl.deployVirtualRouterInGuestNetwork(VirtualNetworkApplianceManagerImpl.java:1902) > >> >> at > >> >> > >> > com.cloud.network.element.VirtualRouterElement.implement(VirtualRouterElement.java:175) > >> >> at > >> >> > >> > com.cloud.network.NetworkManagerImpl.implementNetworkElementsAndResources(NetworkManagerImpl.java:1518) > >> >> at > >> >> > >> > com.cloud.network.NetworkManagerImpl.restartNetwork(NetworkManagerImpl.java:2473) > >> >> at > >> >> > >> > com.cloud.network.NetworkServiceImpl.restartNetwork(NetworkServiceImpl.java:1381) > >> >> at > >> >> > >> > com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) > >> >> at > >> >> > >> > org.apache.cloudstack.api.command.user.network.RestartNetworkCmd.execute(RestartNetworkCmd.java:92) > >> >> at > com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:162) > >> >> at > >> >> > com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:437) > >> >> at > >> >> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > >> >> at > >> >> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > >> >> at java.util.concurrent.FutureTask.run(FutureTask.java:166) > >> >> at > >> >> > >> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > >> >> at > >> >> > >> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > >> >> at java.lang.Thread.run(Thread.java:722) > >> >> INFO [xen.resource.CitrixResourceBase] (DirectAgent-16:) VM does not > >> >> exist on XenServer0d1d5015-4061-4554-96de-406155449296 > >> >> WARN [cloud.network.NetworkManagerImpl] (Job-Executor-151:job-6526) > >> >> Failed to implement network Ntwk[338|Guest|8] elements and resources > as > >> a > >> >> part of network restart due to > >> >> com.cloud.exception.AgentUnavailableException: Resource [Host:9] is > >> >> unreachable: Host 9: Unable to start instance due to null > >> >> at > >> >> > >> > com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:841) > >> >> at > >> >> > >> > com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:471) > >> >> at > >> >> > >> > com.cloud.network.router.VirtualNetworkApplianceManagerImpl.start(VirtualNetworkApplianceManagerImpl.java:2616) > >> >> at > >> >> > >> > com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startVirtualRouter(VirtualNetworkApplianceManagerImpl.java:1824) > >> >> at > >> >> > >> > com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRouters(VirtualNetworkApplianceManagerImpl.java:1924) > >> >> at > >> >> > >> > com.cloud.network.router.VirtualNetworkApplianceManagerImpl.deployVirtualRouterInGuestNetwork(VirtualNetworkApplianceManagerImpl.java:1902) > >> >> at > >> >> > >> > com.cloud.network.element.VirtualRouterElement.implement(VirtualRouterElement.java:175) > >> >> at > >> >> > >> > com.cloud.network.NetworkManagerImpl.implementNetworkElementsAndResources(NetworkManagerImpl.java:1518) > >> >> at > >> >> > >> > com.cloud.network.NetworkManagerImpl.restartNetwork(NetworkManagerImpl.java:2473) > >> >> at > >> >> > >> > com.cloud.network.NetworkServiceImpl.restartNetwork(NetworkServiceImpl.java:1381) > >> >> at > >> >> > >> > com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) > >> >> at > >> >> > >> > org.apache.cloudstack.api.command.user.network.RestartNetworkCmd.execute(RestartNetworkCmd.java:92) > >> >> at > com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:162) > >> >> at > >> >> > com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:437) > >> >> at > >> >> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > >> >> at > >> >> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > >> >> at java.util.concurrent.FutureTask.run(FutureTask.java:166) > >> >> at > >> >> > >> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > >> >> at > >> >> > >> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > >> >> at java.lang.Thread.run(Thread.java:722) > >> >> Caused by: java.lang.NullPointerException > >> >> at > >> >> > >> > com.cloud.network.NetworkModelImpl.getIpInNetwork(NetworkModelImpl.java:763) > >> >> at > >> >> > >> > com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeNetworkRulesForNetwork(VirtualNetworkApplianceManagerImpl.java:2346) > >> >> at > >> >> > >> > com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.finalizeNetworkRulesForNetwork(VpcVirtualNetworkApplianceManagerImpl.java:928) > >> >> at > >> >> > >> > com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeCommandsOnStart(VirtualNetworkApplianceManagerImpl.java:2241) > >> >> at > >> >> > >> > com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.finalizeCommandsOnStart(VpcVirtualNetworkApplianceManagerImpl.java:767) > >> >> at > >> >> > >> > com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeDeployment(VirtualNetworkApplianceManagerImpl.java:2205) > >> >> at > >> >> > >> > com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:763) > >> >> ... 23 more > >> >> WARN [cloud.network.NetworkServiceImpl] (Job-Executor-151:job-6526) > >> >> Network id=338 failed to restart. > >> >> WARN [cloud.async.AsyncJobManagerImpl] (Job-Executor-151:job-6526) > >> Unable > >> >> to unregister active job 6526 from JMX monitoring > > > >> >> Here are log when I want to start VR (his state is stopeed, I want to > >> >> start and got this) > > >> >> WARN [network.element.VpcVirtualRouterElement] > >> >> (Job-Executor-153:job-6528) Network Ntwk[200|Public|1] is not > associated > >> >> with any VPC > >> >> WARN [network.element.VpcVirtualRouterElement] > >> >> (Job-Executor-153:job-6528) Network Ntwk[338|Guest|8] is not > associated > >> >> with any VPC > >> >> WARN [network.element.VpcVirtualRouterElement] > >> >> (Job-Executor-153:job-6528) Network Ntwk[202|Control|3] is not > >> associated > >> >> with any VPC > >> >> ERROR [cloud.vm.VirtualMachineManagerImpl] > (Job-Executor-153:job-6528) > >> >> Failed to start instance VM[DomainRouter|r-1015-VM] > >> >> java.lang.NullPointerException > >> >> at > >> >> > >> > com.cloud.network.NetworkModelImpl.getIpInNetwork(NetworkModelImpl.java:763) > >> >> at > >> >> > >> > com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeNetworkRulesForNetwork(VirtualNetworkApplianceManagerImpl.java:2346) > >> >> at > >> >> > >> > com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.finalizeNetworkRulesForNetwork(VpcVirtualNetworkApplianceManagerImpl.java:928) > >> >> at > >> >> > >> > com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeCommandsOnStart(VirtualNetworkApplianceManagerImpl.java:2241) > >> >> at > >> >> > >> > com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.finalizeCommandsOnStart(VpcVirtualNetworkApplianceManagerImpl.java:767) > >> >> at > >> >> > >> > com.cloud.network.router.VirtualNetworkApplianceManagerImpl.finalizeDeployment(VirtualNetworkApplianceManagerImpl.java:2205) > >> >> at > >> >> > >> > com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:763) > >> >> at > >> >> > >> > com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:471) > >> >> at > >> >> > >> > com.cloud.network.router.VirtualNetworkApplianceManagerImpl.start(VirtualNetworkApplianceManagerImpl.java:2616) > >> >> at > >> >> > >> > com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startVirtualRouter(VirtualNetworkApplianceManagerImpl.java:1824) > >> >> at > >> >> > >> > com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRouter(VirtualNetworkApplianceManagerImpl.java:2874) > >> >> at > >> >> > >> > com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRouter(VirtualNetworkApplianceManagerImpl.java:2824) > >> >> at > >> >> > >> > com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) > >> >> at > >> >> > >> > org.apache.cloudstack.api.command.admin.router.StartRouterCmd.execute(StartRouterCmd.java:103) > >> >> at > com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:162) > >> >> at > >> >> > com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:437) > >> >> at > >> >> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > >> >> at > >> >> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > >> >> at java.util.concurrent.FutureTask.run(FutureTask.java:166) > >> >> at > >> >> > >> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > >> >> at > >> >> > >> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > >> >> at java.lang.Thread.run(Thread.java:722) > > > >> >> Best Regards, > >> >> Michael > > > > >> -- > >> Pozdrowienia, > >> Michael Lukzak > > > > -- > Pozdrowienia, > Michael Lukzak
