[
https://issues.apache.org/jira/browse/CLOUDSTACK-1228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13586980#comment-13586980
]
prashant kumar mishra commented on CLOUDSTACK-1228:
---------------------------------------------------
i changed permission and manually mounted .. then ran into another exception;
Checked in vcenter server around 12 system vms got created(all are in power
off state) within 30 minute , and CS keep deploying new system vms as time
passes.CS UI don't show them but you can see them in vCenter server.
snippet of log
-------------------
------------------
013-02-26 07:50:25,006 INFO [vmware.mo.DatastoreMO]
(DirectAgent-36:10.147.40.8) Search file ROOT-15-15.vmdk on
[d65fe2b3054b37289a07f4af32167d63]
2013-02-26 07:50:26,304 INFO [vmware.mo.DatastoreMO]
(DirectAgent-36:10.147.40.8) File [d65fe2b3054b37289a07f4af32167d63]
ROOT-15-15.vmdk does not exist on datastore
2013-02-26 07:50:26,519 INFO [vmware.mo.DatastoreMO]
(DirectAgent-36:10.147.40.8) Search file ROOT-15-15-flat.vmdk on
[d65fe2b3054b37289a07f4af32167d63]
2013-02-26 07:50:27,832 INFO [vmware.mo.DatastoreMO]
(DirectAgent-36:10.147.40.8) File [d65fe2b3054b37289a07f4af32167d63]
ROOT-15-15-flat.vmdk does not exist on datastore
2013-02-26 07:50:28,044 INFO [vmware.mo.DatastoreMO]
(DirectAgent-36:10.147.40.8) Search file ROOT-15-15-delta.vmdk on
[d65fe2b3054b37289a07f4af32167d63]
2013-02-26 07:50:29,358 INFO [vmware.mo.DatastoreMO]
(DirectAgent-36:10.147.40.8) File [d65fe2b3054b37289a07f4af32167d63]
ROOT-15-15-delta.vmdk does not exist on datastore
2013-02-26 07:50:32,115 INFO [vmware.resource.VmwareResource]
(DirectAgent-36:10.147.40.8) Destroy dummy VM after volume creation
2013-02-26 07:50:34,015 ERROR [vmware.resource.VmwareResource]
(DirectAgent-36:10.147.40.8) CreateCommand failed due to Exception:
java.lang.Exception
Message: Unable to create disk [d65fe2b3054b37289a07f4af32167d63]
ROOT-15-15.vmdk due to Invalid configuration for device '0'.
com.vmware.vim25.InvalidDeviceSpec
java.lang.Exception: Unable to create disk [d65fe2b3054b37289a07f4af32167d63]
ROOT-15-15.vmdk due to Invalid configuration for device '0'.
com.vmware.vim25.InvalidDeviceSpec
at
com.cloud.hypervisor.vmware.mo.VirtualMachineMO.createDisk(VirtualMachineMO.java:937)
at
com.cloud.hypervisor.vmware.mo.VirtualMachineMO.createDisk(VirtualMachineMO.java:855)
at
com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResource.java:3915)
at
com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(VmwareResource.java:342)
at
com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
2013-02-26 07:50:34,018 DEBUG [agent.manager.DirectAgentAttache]
(DirectAgent-36:null) Seq 1-745013312: Response Received:
==========
2
==========
2013-02-26 10:12:31,486 DEBUG [utils.script.Script]
(DirectAgent-15:10.147.40.7) Looking for vms/systemvm.iso
2013-02-26 10:12:31,486 DEBUG [utils.script.Script]
(DirectAgent-15:10.147.40.7) Current binaries reside at
/usr/share/cloudstack-management/webapps/client/WEB-INF/lib
2013-02-26 10:12:31,486 DEBUG [utils.script.Script]
(DirectAgent-15:10.147.40.7) Looking for vms/systemvm.iso in
/usr/share/cloudstack-management/webapps/client/WEB-INF/lib/vms/systemvm.iso
2013-02-26 10:12:31,486 DEBUG [utils.script.Script]
(DirectAgent-15:10.147.40.7) Looking for vms/systemvm.iso in
/usr/share/cloudstack-management/webapps/client/WEB-INF/lib/vms/systemvm.iso
2013-02-26 10:12:31,487 DEBUG [utils.script.Script]
(DirectAgent-15:10.147.40.7) Looking for vms/systemvm.iso in
/usr/share/cloudstack-management/webapps/client/WEB-INF/vms/systemvm.iso
2013-02-26 10:12:31,487 DEBUG [utils.script.Script]
(DirectAgent-15:10.147.40.7) Looking for vms/systemvm.iso in
/usr/share/cloudstack-management/webapps/client/vms/systemvm.iso
2013-02-26 10:12:31,487 DEBUG [utils.script.Script]
(DirectAgent-15:10.147.40.7) Looking for vms/systemvm.iso in
/usr/share/cloudstack-management/webapps/vms/systemvm.iso
2013-02-26 10:12:31,487 DEBUG [utils.script.Script]
(DirectAgent-15:10.147.40.7) Looking for vms/systemvm.iso in
/usr/share/cloudstack-management/vms/systemvm.iso
2013-02-26 10:12:31,487 DEBUG [utils.script.Script]
(DirectAgent-15:10.147.40.7) Looking for vms/systemvm.iso in
/usr/share/vms/systemvm.iso
2013-02-26 10:12:31,488 DEBUG [utils.script.Script]
(DirectAgent-15:10.147.40.7) Looking for vms/systemvm.iso in
/usr/vms/systemvm.iso
2013-02-26 10:12:31,488 DEBUG [utils.script.Script]
(DirectAgent-15:10.147.40.7) Looking for vms/systemvm.iso in /vms/systemvm.iso
2013-02-26 10:12:31,488 DEBUG [utils.script.Script]
(DirectAgent-15:10.147.40.7) Searching in environment.properties
2013-02-26 10:12:31,488 DEBUG [utils.script.Script]
(DirectAgent-15:10.147.40.7) environment.properties says scripts should be in
/usr/share/cloudstack-common
2013-02-26 10:12:31,489 DEBUG [utils.script.Script]
(DirectAgent-15:10.147.40.7) Looking for vms/systemvm.iso in
/usr/share/cloudstack-common/vms/systemvm.iso
2013-02-26 10:12:31,489 DEBUG [cloud.server.ConfigurationServerImpl]
(DirectAgent-15:10.147.40.7) Executing:
/usr/share/cloudstack-common/scripts/vm/systemvm/injectkeys.sh
/var/cloudstack/management/.ssh/id_rsa.pub
/var/cloudstack/management/.ssh/id_rsa
/usr/share/cloudstack-common/vms/systemvm.iso
2013-02-26 10:12:31,599 DEBUG [cloud.server.ConfigurationServerImpl]
(DirectAgent-15:10.147.40.7) Execution is successful.
2013-02-26 10:12:31,600 ERROR [vmware.manager.VmwareManagerImpl]
(DirectAgent-15:10.147.40.7) Unexpected exception
java.io.FileNotFoundException: /usr/lib/cloud/common/vms/systemvm.iso (No such
file or directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:137)
at com.cloud.utils.FileUtil.copyfile(FileUtil.java:68)
at
com.cloud.hypervisor.vmware.manager.VmwareManagerImpl.prepareSecondaryStorageStore(VmwareManagerImpl.java:555)
at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at
org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint(Cglib2AopProxy.java:689)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at
org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:80)
at
com.cloud.utils.db.TransactionContextBuilder.AroundAnyMethod(TransactionContextBuilder.java:43)
at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at
org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621)
at
org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610)
at
org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:65)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:90)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at
org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:622)
at
com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResource.java:2145)
at
com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(VmwareResource.java:426)
at
com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
2013-02-26 10:12:31,607 ERROR [vmware.manager.VmwareManagerImpl]
(DirectAgent-15:10.147.40.7) Unable to copy systemvm ISO on secondary storage.
src location: /usr/lib/cloud/common/vms/systemvm.iso, dest location:
/var/cloudstack/mnt/VM/6903220404277.6defdeab/systemvm/systemvm-4.2.0-SNAPSHOT.iso
2013-02-26 10:12:31,609 WARN [vmware.resource.VmwareResource]
(DirectAgent-15:10.147.40.7) StartCommand failed due to Exception:
com.cloud.utils.exception.CloudRuntimeException
Message: Unable to copy systemvm ISO on secondary storage. src location:
/usr/lib/cloud/common/vms/systemvm.iso, dest location:
/var/cloudstack/mnt/VM/6903220404277.6defdeab/systemvm/systemvm-4.2.0-SNAPSHOT.iso
com.cloud.utils.exception.CloudRuntimeException: Unable to copy systemvm ISO on
secondary storage. src location: /usr/lib/cloud/common/vms/systemvm.iso, dest
location:
/var/cloudstack/mnt/VM/6903220404277.6defdeab/systemvm/systemvm-4.2.0-SNAPSHOT.iso
at
com.cloud.hypervisor.vmware.manager.VmwareManagerImpl.prepareSecondaryStorageStore(VmwareManagerImpl.java:561)
at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at
org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint(Cglib2AopProxy.java:689)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at
org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:80)
at
com.cloud.utils.db.TransactionContextBuilder.AroundAnyMethod(TransactionContextBuilder.java:43)
at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at
org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621)
at
org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610)
at
org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:65)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:90)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at
org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:622)
at
com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResource.java:2145)
at
com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(VmwareResource.java:426)
at
com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
2013-02-26 10:12:31,616 DEBUG [agent.manager.DirectAgentAttache]
(DirectAgent-15:null) Seq 1-845021211: Cancelling because one of the answers is
false and it is stop on error.
> Unable to Create System Vm's in the VMware Hypervisor setup.
> ------------------------------------------------------------
>
> Key: CLOUDSTACK-1228
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-1228
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the
> default.)
> Components: Install and Setup
> Affects Versions: 4.1.0
> Reporter: Kiran Koneti
> Assignee: Venkata Siva Vijayendra Bhamidipati
> Priority: Blocker
> Fix For: 4.1.0
>
> Attachments: catalina.2013-02-11.zip, catalina.zip,
> management-server-18-2-2013.zip, management-server.zip
>
>
> Created a Advanced Zone setup using the Latest 4.1 build
> (http://repo-ccp.citrix.com/releases/ASF/rhel/6.3/4.1/CloudStack-non-OSS-11-rhel6.3.tar.gz)
> While trying to create the system VM's error messages are obserevd.
> Attaching the Management server logs.
> Error message Observed:
> Executing copyTemplateFromSecondaryToPrimary. secondaryStorage:
> nfs://10.147.28.7/export/home/kiran/s, templatePathAtSecondaryStorage:
> template/tmpl/1/8/, templateName: routing-8
> 2013-02-11 23:03:41,530 ERROR [vmware.manager.VmwareManagerImpl]
> (DirectAgent-2:10.147.40.18) Unable to create mount:
> /var/lib/cloud/mnt/VM/7313725325399.47bfef59
> 2013-02-11 23:03:41,531 ERROR [vmware.manager.VmwareManagerImpl]
> (DirectAgent-2:10.147.40.18) Unable to create mount:
> /var/lib/cloud/mnt/VM/7313725325399.648a617f
> 2013-02-11 23:03:41,531 ERROR [vmware.manager.VmwareManagerImpl]
> (DirectAgent-2:10.147.40.18) Unable to create mount:
> /var/lib/cloud/mnt/VM/7313725325399.2257f5b0
> 2013-02-11 23:03:41,532 ERROR [vmware.manager.VmwareManagerImpl]
> (DirectAgent-2:10.147.40.18) Unable to create mount:
> /var/lib/cloud/mnt/VM/7313725325399.3e22e9af
> 2013-02-11 23:03:41,533 ERROR [vmware.manager.VmwareManagerImpl]
> (DirectAgent-2:10.147.40.18) Unable to create mount:
> /var/lib/cloud/mnt/VM/7313725325399.1e46e9bc
> 2013-02-11 23:03:41,534 ERROR [vmware.manager.VmwareManagerImpl]
> (DirectAgent-2:10.147.40.18) Unable to create mount:
> /var/lib/cloud/mnt/VM/7313725325399.49fee4a0
> 2013-02-11 23:03:41,535 ERROR [vmware.manager.VmwareManagerImpl]
> (DirectAgent-2:10.147.40.18) Unable to create mount:
> /var/lib/cloud/mnt/VM/7313725325399.c1b6a94
> 2013-02-11 23:03:41,535 ERROR [vmware.manager.VmwareManagerImpl]
> (DirectAgent-2:10.147.40.18) Unable to create mount:
> /var/lib/cloud/mnt/VM/7313725325399.5604ba42
> 2013-02-11 23:03:41,536 ERROR [vmware.manager.VmwareManagerImpl]
> (DirectAgent-2:10.147.40.18) Unable to create mount:
> /var/lib/cloud/mnt/VM/7313725325399.3632e0de
> 2013-02-11 23:03:41,537 ERROR [vmware.manager.VmwareManagerImpl]
> (DirectAgent-2:10.147.40.18) Unable to create mount:
> /var/lib/cloud/mnt/VM/7313725325399.22475368
> 2013-02-11 23:03:41,537 WARN [vmware.manager.VmwareManagerImpl]
> (DirectAgent-2:10.147.40.18) Unable to create a mount point
> 2013-02-11 23:03:41,538 ERROR [vmware.manager.VmwareManagerImpl]
> (DirectAgent-2:10.147.40.18) Unable to create mount point for
> nfs://10.147.28.7/export/home/kiran/s
> 2013-02-11 23:03:41,538 INFO [vmware.manager.VmwareStorageManagerImpl]
> (DirectAgent-2:10.147.40.18) Secondary storage mount point: /mnt/sec
> 2013-02-11 23:03:41,538 INFO [vmware.manager.VmwareStorageManagerImpl]
> (DirectAgent-2:10.147.40.18) Executing command: tar --no-same-owner -xf
> /mnt/sec/template/tmpl/1/8/routing-8.ova
> 2013-02-11 23:03:41,539 DEBUG [vmware.manager.VmwareStorageManagerImpl]
> (DirectAgent-2:10.147.40.18) Executing: tar --no-same-owner -xf
> /mnt/sec/template/tmpl/1/8/routing-8.ova
> 2013-02-11 23:03:41,603 WARN [vmware.manager.VmwareStorageManagerImpl]
> (DirectAgent-2:10.147.40.18) Exception: tar --no-same-owner -xf
> /mnt/sec/template/tmpl/1/8/routing-8.ova
> java.io.IOException: Cannot run program "tar" (in directory
> "/mnt/sec/template/tmpl/1/8"): java.io.IOException: error=2, No such file or
> directory
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:475)
> at com.cloud.utils.script.Script.execute(Script.java:183)
> at com.cloud.utils.script.Script.execute(Script.java:161)
> at
> com.cloud.hypervisor.vmware.manager.VmwareStorageManagerImpl.copyTemplateFromSecondaryToPrimary(VmwareStorageManagerImpl.java:443)
> at
> com.cloud.hypervisor.vmware.manager.VmwareStorageManagerImpl.execute(VmwareStorageManagerImpl.java:138)
> at
> com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResource.java:3587)
> at
> com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(VmwareResource.java:401)
> at
> com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> Caused by: java.io.IOException: java.io.IOException: error=2, No such file or
> directory
> at java.lang.UNIXProcess.<init>(UNIXProcess.java:164)
> at java.lang.ProcessImpl.start(ProcessImpl.java:81)
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:468)
> ... 15 more
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
