[ https://issues.apache.org/jira/browse/CLOUDSTACK-1430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13591785#comment-13591785 ]
Marcus Sorensen commented on CLOUDSTACK-1430: --------------------------------------------- Thanks, as you may have noticed, the issue on the agent is this: com.cloud.utils.exception.CloudRuntimeException: org.libvirt.LibvirtException: cannot create path '/mnt/ab5c439e-8105-3e49-abec-8b1406ecc348/s-4-VM-patchdisk': Invalid argument This correlates to where CloudStack is creating the Libvirt description of the disk. The Libvirt logs may be of help here (any time there's a LibvirtException). Is that particular location writable, and does s-4-VM-patchdisk exist? Last I tested the artifacts was on local storage. I'll give NFS a shot to see if it's related to that. Until then, if you could look for those clues and see where the 'invalid argument' might be returned from libvirt that would be a good next step. The libvirt disk definition you're passing is: <volume> <name>s-4-VM-patchdisk</name> <capacity >10485760</capacity> <target> <format type='raw'/> <permissions><mode>0744</mode></permissions></target> </volume> Which looks the same as the one my working system passes. > KVM: systemVMs failed to come up on 4.1 > --------------------------------------- > > Key: CLOUDSTACK-1430 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-1430 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the > default.) > Components: KVM, Management Server > Affects Versions: 4.1.0 > Reporter: Prasanna Santhanam > Assignee: Marcus Sorensen > Priority: Blocker > Attachments: debugagentlogs.tar.bz2, mslogs.tar.bz2 > > > This is from the latest packages from jenkins.cs.o. > http://jenkins.cloudstack.org/job/package-rhel63-4.1/101/ > When the management server tries to bring up system vms, following exception > is noted. > Agent is up and running and cloudbr0 is the device for public, private traffic > [root@apache-81-3 ~]# cat /etc/cloudstack/agent/agent.properties | grep > cloudbr0 > guest.network.device=cloudbr0 > private.network.device=cloudbr0 > public.network.device=cloudbr0 > 2013-02-27 15:28:49,828 WARN [storage.secondary.SecondaryStorageManagerImpl] > (secstorage-1:null) Exception while trying to start secondary storage vm > com.cloud.exception.AgentUnavailableException: Resource [Host:3] is > unreachable: Host 3: Unable to start instance due to Unable to get answer > that is of class com.cloud.agent.api.StartAnswer > at > com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:831) > at > com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:461) > at > com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:454) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:616) > at > org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:319) > at > org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183) > at > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150) > at > org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:80) > at > com.cloud.utils.db.TransactionContextBuilder.AroundAnyMethod(TransactionContextBuilder.java:37) > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:616) > at > org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621) > at > org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610) > at > org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:65) > at > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) > at > org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:90) > at > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) > at > org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202) > at sun.proxy.$Proxy193.start(Unknown Source) > at > com.cloud.storage.secondary.SecondaryStorageManagerImpl.startSecStorageVm(SecondaryStorageManagerImpl.java:264) > at > com.cloud.storage.secondary.SecondaryStorageManagerImpl.allocCapacity(SecondaryStorageManagerImpl.java:689) > at > com.cloud.storage.secondary.SecondaryStorageManagerImpl.expandPool(SecondaryStorageManagerImpl.java:1300) > at > com.cloud.secstorage.PremiumSecondaryStorageManagerImpl.scanPool(PremiumSecondaryStorageManagerImpl.java:121) > at > com.cloud.secstorage.PremiumSecondaryStorageManagerImpl.scanPool(PremiumSecondaryStorageManagerImpl.java:52) > at > com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:104) > at > com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner.java:33) > at > com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.java:81) > at com.cloud.vm.SystemVmLoadScanner$1.run(SystemVmLoadScanner.java:72) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at > java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:679) > Caused by: com.cloud.utils.exception.CloudRuntimeException: Unable to get > answer that is of class com.cloud.agent.api.StartAnswer > at com.cloud.agent.manager.Commands.getAnswer(Commands.java:80) > at > com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:767) > ... 39 more -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira