[
https://issues.apache.org/jira/browse/CLOUDSTACK-1430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanna Santhanam updated CLOUDSTACK-1430:
-------------------------------------------
Attachment: debugagentlogs.tar.bz2
The mounts on the kvm host at the time of this failure are as follows:
/dev/mapper/vg_apache813-lv_root on / type ext4 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
tmpfs on /dev/shm type tmpfs (rw,rootcontext="system_u:object_r:tmpfs_t:s0")
/dev/sda1 on /boot type ext4 (rw)
/dev/mapper/vg_apache813-lv_home on /home type ext4 (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
nfsd on /proc/fs/nfsd type nfsd (rw)
nfs.fmt.vmops.com:/export/automation/acs/primary on
/mnt/ab5c439e-8105-3e49-abec-8b1406ecc348 type nfs
(rw,noac,vers=4,addr=10.208.13.194,clientaddr=10.208.8.69)
nfs.fmt.vmops.com:/export/automation/acs/primary1 on
/mnt/d48b8a0e-9ca6-32c4-a7de-946bf9ad2fd2 type nfs
(rw,noac,vers=4,addr=10.208.13.194,clientaddr=10.208.8.69)
nfs.fmt.vmops.com:/export/automation/acs/secondary/template/tmpl/1/3 on
/mnt/1792742a-1455-39dc-8405-32ff6ebafb42 type nfs
(rw,sync,soft,proto=tcp,acregmin=0,acregmax=0,acdirmin=0,acdirmax=0,noac,timeo=133,retrans=10,noac,vers=4,addr=10.208.13.194,clientaddr=10.208.8.69)
[root@apache-81-3 cloudstack]#
> KVM: systemVMs failed to come up on 4.1
> ---------------------------------------
>
> Key: CLOUDSTACK-1430
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-1430
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the
> default.)
> Components: KVM, Management Server
> Affects Versions: 4.1.0
> Reporter: Prasanna Santhanam
> Assignee: Marcus Sorensen
> Priority: Blocker
> Attachments: agentlogs.tar.bz2, debugagentlogs.tar.bz2, mslogs.tar.bz2
>
>
> This is from the latest packages from jenkins.cs.o.
> http://jenkins.cloudstack.org/job/package-rhel63-4.1/101/
> When the management server tries to bring up system vms, following exception
> is noted.
> Agent is up and running and cloudbr0 is the device for public, private traffic
> [root@apache-81-3 ~]# cat /etc/cloudstack/agent/agent.properties | grep
> cloudbr0
> guest.network.device=cloudbr0
> private.network.device=cloudbr0
> public.network.device=cloudbr0
> 2013-02-27 15:28:49,828 WARN [storage.secondary.SecondaryStorageManagerImpl]
> (secstorage-1:null) Exception while trying to start secondary storage vm
> com.cloud.exception.AgentUnavailableException: Resource [Host:3] is
> unreachable: Host 3: Unable to start instance due to Unable to get answer
> that is of class com.cloud.agent.api.StartAnswer
> at
> com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:831)
> at
> com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:461)
> at
> com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:454)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:616)
> at
> org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:319)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
> at
> org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:80)
> at
> com.cloud.utils.db.TransactionContextBuilder.AroundAnyMethod(TransactionContextBuilder.java:37)
> at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:616)
> at
> org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621)
> at
> org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610)
> at
> org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:65)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
> at
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:90)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
> at
> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
> at sun.proxy.$Proxy193.start(Unknown Source)
> at
> com.cloud.storage.secondary.SecondaryStorageManagerImpl.startSecStorageVm(SecondaryStorageManagerImpl.java:264)
> at
> com.cloud.storage.secondary.SecondaryStorageManagerImpl.allocCapacity(SecondaryStorageManagerImpl.java:689)
> at
> com.cloud.storage.secondary.SecondaryStorageManagerImpl.expandPool(SecondaryStorageManagerImpl.java:1300)
> at
> com.cloud.secstorage.PremiumSecondaryStorageManagerImpl.scanPool(PremiumSecondaryStorageManagerImpl.java:121)
> at
> com.cloud.secstorage.PremiumSecondaryStorageManagerImpl.scanPool(PremiumSecondaryStorageManagerImpl.java:52)
> at
> com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:104)
> at
> com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner.java:33)
> at
> com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.java:81)
> at com.cloud.vm.SystemVmLoadScanner$1.run(SystemVmLoadScanner.java:72)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at
> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:679)
> Caused by: com.cloud.utils.exception.CloudRuntimeException: Unable to get
> answer that is of class com.cloud.agent.api.StartAnswer
> at com.cloud.agent.manager.Commands.getAnswer(Commands.java:80)
> at
> com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:767)
> ... 39 more
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
