I found the problem on my set up and got it working now. I used devCloud to run the management server. The default network device uses cloudbr0. However onDevcloud, I need to replace it with vm0. After I use vm0 for the private network and management network, it works.
Thanks, -Fang -----Original Message----- From: Marcus Sorensen [mailto:shadow...@gmail.com] Sent: Wednesday, January 30, 2013 9:20 AM To: cloudstack-dev@incubator.apache.org Subject: Re: Not able to start System Vms on KVM host Well, there are two things at play here. One is that the code in LibvirtComputingResource sets private.network.device to cloudbr1 if none is found in agent.properties, and the other is what is in your agent.properties. It looks like private.network.device was changed to cloudbr1 in agent.properties file in August (5238a836880951d4cf3024a433358bad8c0fa9e4), but it's always been commented out, that's just an example. I've gone all the way back to the 2.x code and LibvirtComputingResource has always set the default to cloubr1 if none was found in agent.properties. One question, how are you guys installing? Without RPM packaging working, it seems like maybe you're installing a 4.0 build and then compiling master, manually copying things, and running it? The reason I ask is because as mentioned, by default the agent.properties is commented out, and I believe it gets written to what it should be on agent joining the cluster. So if you're upgrading an existing, working cluster, the agent.properties should already be configured. If you accidentally install a new agent.properties then this problem will occur. This is one of the issues with having had the packaging system broken all these months. If one can't easily do a fresh install of the code (or a scripted upgrade that everyone will be using in the case of a package upgrade), it's hard to get a baseline to work from. On Tue, Jan 29, 2013 at 7:14 PM, Rayees Namathponnan <rayees.namathpon...@citrix.com> wrote: > I was facing same issue with master build. > > Then re-created same environment with network-refactor build (HEAD is at > 93a89b1 CLOUDSTACK-995: fix failed to detect centos 6.2), system vms are > coming up without updating /etc/cloud/agent/agent.properties > > private.network.device=cloudbr0 was in /etc/cloud/agent/agent.properties by > default. > > Looks like, it's a regression. > > Regards, > Rayees > > -----Original Message----- > From: Marcus Sorensen [mailto:shadow...@gmail.com] > Sent: Tuesday, January 29, 2013 5:46 PM > To: cloudstack-dev@incubator.apache.org > Subject: RE: Not able to start System Vms on KVM host > > OK, I will look and see when that changed to cloudbr1 as default. > On Jan 29, 2013 6:08 PM, "Sangeetha Hariharan" < > sangeetha.hariha...@citrix.com> wrote: > >> Thanks Marcus. >> >> When I added private.network.device=cloubr0 in >> /etc/cloud/agent/agent.properties and restarted the cloud-agent , I >> see the system Vms being launched successfully. >> We did not have to do these changes in the past. I don't think we did >> any of these changes when we tested RHEL 6.3 host with a build from >> "networkrefactor" branch sometime back. >> >> -Thanks >> Sangeetha >> >> -----Original Message----- >> From: Marcus Sorensen [mailto:shadow...@gmail.com] >> Sent: Tuesday, January 29, 2013 4:45 PM >> To: cloudstack-dev@incubator.apache.org >> Subject: Re: Not able to start System Vms on KVM host >> >> This is because if there is no private.network.device config in your >> /etc/cloud/agent/agent.properties file, LibvirtComputingResource.java >> defaults to 'cloudbr1'. I would suggest setting all of your traffic >> labels to cloudbr0, or setting private.network.device to cloubr0. >> I'm assuming it was set up this way because of the recommendation >> that management and public be on different physical networks, but I >> can't really say for sure what the intention was. I think the default >> agent.properties file has it defined as cloudbr1 as well. >> >> I suppose this is broken since the default in the ui says 'use >> default gateway', which it's not doing, but I don't think it's a >> regression or new bug, I think it's been that way since I started using >> cloudstack. >> >> On Tue, Jan 29, 2013 at 4:57 PM, Sangeetha Hariharan < >> sangeetha.hariha...@citrix.com> wrote: >> > Marcus, >> > >> > I tried with the patch you have provided on IPV6 branch. >> > >> > Now I see the the public interface being programed correctly. But >> > there >> are issues with 'cloudbr1' and system Vms are still not coming up. >> > The same issue is also seen when testing with master branch. >> > >> > >> > [root@Rack3Host4 agent]# brctl show >> > bridge name bridge id STP enabled interfaces >> > brem1-1363 8000.bc305bd41f86 no em1.1363 >> > cloud0 8000.000000000000 no >> > cloudbr0 8000.bc305bd41f86 no em1 >> > virbr0 8000.525400463d1f yes virbr0-nic >> > >> > >> > [root@Rack3Host4 agent]# ls /sys/devices/virtual/net >> > brem1-1363 cloud0 cloudbr0 em1.1363 lo virbr0 virbr0-nic >> > >> > Following exception seen in agent.log: >> > >> > 2013-01-29 18:40:55,451 WARN >> > [kvm.resource.LibvirtComputingResource] >> > (agentRequest-Handler-2:null) Failed to start domain: v-2-VM: >> > Cannot get interface MTU on 'cloudbr1': No such device >> > 2013-01-29 18:40:55,452 WARN >> > [kvm.resource.LibvirtComputingResource] >> > (agentRequest-Handler-2:null) Exception >> > org.libvirt.LibvirtException: Cannot get interface MTU on >> > 'cloudbr1': No >> such device >> > at org.libvirt.ErrorHandler.processError(Unknown Source) >> > at org.libvirt.Connect.processError(Unknown Source) >> > at org.libvirt.Domain.processError(Unknown Source) >> > at org.libvirt.Domain.create(Unknown Source) >> > at >> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.startDomai >> n >> (LibvirtComputingResource.java:1049) >> > at >> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(Li >> b >> virtComputingResource.java:3096) >> > at >> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeReq >> u >> est(LibvirtComputingResource.java:1174) >> > at com.cloud.agent.Agent.processRequest(Agent.java:525) >> > at >> com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:852) >> > at com.cloud.utils.nio.Task.run(Task.java:83) >> > at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor. >> j >> ava:1110) >> > at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor. >> java:603) >> > at java.lang.Thread.run(Thread.java:679) >> > 2013-01-29 18:40:55,452 WARN [cloud.agent.Agent] >> (agentRequest-Handler-2:null) Caught: >> > java.lang.NullPointerException >> > at >> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.cleanupVMN >> e >> tworks(LibvirtComputingResource.java:4223) >> > at >> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.handleVmSt >> a >> rtFailure(LibvirtComputingResource.java:2992) >> > at >> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(Li >> b >> virtComputingResource.java:3116) >> > at >> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeReq >> u >> est(LibvirtComputingResource.java:1174) >> > at com.cloud.agent.Agent.processRequest(Agent.java:525) >> > at >> com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:852) >> > at com.cloud.utils.nio.Task.run(Task.java:83) >> > at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor. >> j >> ava:1110) >> > at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor. >> java:603) >> > at java.lang.Thread.run(Thread.java:679) >> > >> > Thanks >> > Sangeetha >> > >> > >>
