[
https://issues.apache.org/jira/browse/CLOUDSTACK-3774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13730661#comment-13730661
]
ASF subversion and git services commented on CLOUDSTACK-3774:
-------------------------------------------------------------
Commit 1741fdfb12fd37ce73708b4d563ac22217098c03 in branch refs/heads/master
from [~devdeep]
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=1741fdf ]
CLOUDSTACK-3774: NPE while creating template from snapshot on a upgraded setup.
After upgrade the SSVM was staying in connecting/alert state. This was because
while handling the processConnect command for ssvm the management server was
trying to template sync. For resource limit calculation it was trying to get the
remote size of the template. If the template was no longer available a number
format exception was thrown. The process connect wasn't getting completed and
the ssvm used to stay in connecting/alert state. While creating template from
snapshot cloudstack looks for up and enabled ssvms and because there wasn't
any available (as the ssvm was in coonecting/alert state) it used to pick the
wrong resource (LocalNfs*Resource) instead of the NfsSecondaryStorageResource.
Fixed the issue by making sure number format exceptions are avoided so that
SSVM moves to the right state.
> [XenServer]NPE while creating template from snapshot on a upgraded setup
> ------------------------------------------------------------------------
>
> Key: CLOUDSTACK-3774
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-3774
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the
> default.)
> Components: Storage Controller, Upgrade, XenServer
> Affects Versions: 4.2.0
> Reporter: Sailaja Mada
> Assignee: Devdeep Singh
> Priority: Blocker
> Fix For: 4.2.0
>
> Attachments: afterupgrademysqldumpusage1.dmp, apilog.log, apilog.log,
> beforeupgrade_mysqldump.dmp, cloud.log, management-server.log,
> management-server.log, postupgradecloud-backup.dmp, snap.png, sysvm.log,
> templatefromsnapshot.png
>
>
> Setup: Xen 6.1 hosts , Two zone ( Zone1 with local storage ,Zone 2 - NFS )
> Upgraded from 3.0.6 to 4.2
> Steps:
> 1) Create Account X and deploy VM on a NFS storage Zone 3
> 2) Create snapshot from ROOT volume of this instance
> 3) Tried to create template from this snapshot.
> Observations :
> 1. It failed with NPE.
> 2. UI results NPE as a failure message
> 2013-07-24 19:07:50,164 DEBUG [storage.resource.NfsSecondaryStorageResource]
> (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) snapshots
> already exists on secondary storage, and is mounted at
> /var/lib/cloud/mnt/secStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1
> 2013-07-24 19:07:50,164 INFO [storage.resource.NfsSecondaryStorageResource]
> (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) snapshots
> directory created/exists on Secondary Storage.
> 2013-07-24 19:07:50,165 DEBUG [storage.resource.NfsSecondaryStorageResource]
> (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) volumes
> already exists on secondary storage, and is mounted at
> /var/lib/cloud/mnt/secStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1
> 2013-07-24 19:07:50,166 INFO [storage.resource.NfsSecondaryStorageResource]
> (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) volumes
> directory created/exists on Secondary Storage.
> 2013-07-24 19:07:50,166 DEBUG
> [storage.resource.LocalNfsSecondaryStorageResource] (Job-Executor-25:job-85 =
> [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) Executing: sudo mount
> 2013-07-24 19:07:50,224 DEBUG
> [storage.resource.LocalNfsSecondaryStorageResource] (Job-Executor-25:job-85 =
> [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) Execution is successful.
> 2013-07-24 19:07:50,224 DEBUG
> [storage.resource.LocalNfsSecondaryStorageResource] (Job-Executor-25:job-85 =
> [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) mount point
> /var/lib/cloud/mnt/secStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1 already
> exists
> 2013-07-24 19:07:50,309 DEBUG [storage.resource.NfsSecondaryStorageResource]
> (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ])
> Executing: null -p
> /var/lib/cloud/mnt/secStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1/snapshots/4/40
> -s b1988755-6c46-4c73-9d3c-2fd751d27825.vhd -n
> f29545c7-cb53-49c8-81a4-5e9ce566093a.vhd -t
> /var/lib/cloud/mnt/secStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1/template/tmpl/4/207
> 2013-07-24 19:07:50,309 WARN [storage.resource.NfsSecondaryStorageResource]
> (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ])
> Exception: null -p
> /var/lib/cloud/mnt/secStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1/snapshots/4/40
> -s b1988755-6c46-4c73-9d3c-2fd751d27825.vhd -n
> f29545c7-cb53-49c8-81a4-5e9ce566093a.vhd -t
> /var/lib/cloud/mnt/secStorage/0a639c32-c2df-36e3-883d-eebcb5093bf1/template/tmpl/4/207
> java.lang.NullPointerException
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:457)
> at com.cloud.utils.script.Script.execute(Script.java:183)
> at com.cloud.utils.script.Script.execute(Script.java:161)
> at
> org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfsXenserver(NfsSecondaryStorageResource.java:360)
> at
> org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfs(NfsSecondaryStorageResource.java:401)
> at
> org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.createTemplateFromSnapshot(NfsSecondaryStorageResource.java:419)
> at
> org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.execute(NfsSecondaryStorageResource.java:445)
> at
> org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:208)
> at
> org.apache.cloudstack.storage.resource.LocalNfsSecondaryStorageResource.executeRequest(LocalNfsSecondaryStorageResource.java:76)
> at
> org.apache.cloudstack.storage.LocalHostEndpoint.sendMessage(LocalHostEndpoint.java:93)
> at
> org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.createTemplateFromSnapshot(AncientDataMotionStrategy.java:384)
> at
> org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:344)
> at
> org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55)
> at
> org.apache.cloudstack.storage.image.TemplateServiceImpl.copyAsync(TemplateServiceImpl.java:549)
> at
> org.apache.cloudstack.storage.image.TemplateServiceImpl.createTemplateFromSnapshotAsync(TemplateServiceImpl.java:556)
> at
> com.cloud.template.TemplateManagerImpl.createPrivateTemplate(TemplateManagerImpl.java:1355)
> at
> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
> at
> org.apache.cloudstack.api.command.user.template.CreateTemplateCmd.execute(CreateTemplateCmd.java:263)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
> at
> com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2013-07-24 19:07:50,348 DEBUG [cloud.template.TemplateManagerImpl]
> (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) Failed to
> create templatejava.lang.NullPointerException
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:457)
> at com.cloud.utils.script.Script.execute(Script.java:183)
> at com.cloud.utils.script.Script.execute(Script.java:161)
> at
> org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfsXenserver(NfsSecondaryStorageResource.java:360)
> at
> org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfs(NfsSecondaryStorageResource.java:401)
> at
> org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.createTemplateFromSnapshot(NfsSecondaryStorageResource.java:419)
> at
> org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.execute(NfsSecondaryStorageResource.java:445)
> at
> org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:208)
> at
> org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:208)
> at
> org.apache.cloudstack.storage.resource.LocalNfsSecondaryStorageResource.executeRequest(LocalNfsSecondaryStorageResource.java:76)
> at
> org.apache.cloudstack.storage.LocalHostEndpoint.sendMessage(LocalHostEndpoint.java:93)
> at
> org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.createTemplateFromSnapshot(AncientDataMotionStrategy.java:384)
> at
> org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:344)
> at
> org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55)
> at
> org.apache.cloudstack.storage.image.TemplateServiceImpl.copyAsync(TemplateServiceImpl.java:549)
> at
> org.apache.cloudstack.storage.image.TemplateServiceImpl.createTemplateFromSnapshotAsync(TemplateServiceImpl.java:556)
> at
> com.cloud.template.TemplateManagerImpl.createPrivateTemplate(TemplateManagerImpl.java:1355)
> at
> com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f.CGLIB$createPrivateTemplate$32(<generated>)
> at
> com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f_FastClassByCloudStack_738e8b.invoke(<generated>)
> at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
> at
> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
> at
> com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f.createPrivateTemplate(<generated>)
> at
> org.apache.cloudstack.api.command.user.template.CreateTemplateCmd.execute(CreateTemplateCmd.java:263)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
> at
> com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2013-07-24 19:07:50,381 ERROR [cloud.async.AsyncJobManagerImpl]
> (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ])
> Unexpected exception while executing
> org.apache.cloudstack.api.command.user.template.CreateTemplateCmd
> com.cloud.utils.exception.CloudRuntimeException: Failed to create
> templatejava.lang.NullPointerException
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:457)
> at com.cloud.utils.script.Script.execute(Script.java:183)
> at com.cloud.utils.script.Script.execute(Script.java:161)
> at
> org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfsXenserver(NfsSecondaryStorageResource.java:360)
> at
> org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfs(NfsSecondaryStorageResource.java:401)
> at
> org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.createTemplateFromSnapshot(NfsSecondaryStorageResource.java:419)
> at
> org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.execute(NfsSecondaryStorageResource.java:445)
> at
> org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:208)
> at
> org.apache.cloudstack.storage.resource.LocalNfsSecondaryStorageResource.executeRequest(LocalNfsSecondaryStorageResource.java:76)
> at
> org.apache.cloudstack.storage.LocalHostEndpoint.sendMessage(LocalHostEndpoint.java:93)
> at
> org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.createTemplateFromSnapshot(AncientDataMotionStrategy.java:384)
> at
> org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:344)
> at
> org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55)
> at
> org.apache.cloudstack.storage.image.TemplateServiceImpl.copyAsync(TemplateServiceImpl.java:549)
> at
> org.apache.cloudstack.storage.image.TemplateServiceImpl.createTemplateFromSnapshotAsync(TemplateServiceImpl.java:556)
> at
> com.cloud.template.TemplateManagerImpl.createPrivateTemplate(TemplateManagerImpl.java:1355)
> at
> com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f.CGLIB$createPrivateTemplate$32(<generated>)
> at
> com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f_FastClassByCloudStack_738e8b.invoke(<generated>)
> at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
> at
> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
> at
> com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f.createPrivateTemplate(<generated>)
> at
> org.apache.cloudstack.api.command.user.template.CreateTemplateCmd.execute(CreateTemplateCmd.java:263)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
> at
> com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> at
> com.cloud.template.TemplateManagerImpl.createPrivateTemplate(TemplateManagerImpl.java:1369)
> at
> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
> at
> org.apache.cloudstack.api.command.user.template.CreateTemplateCmd.execute(CreateTemplateCmd.java:263)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
> at
> com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2013-07-24 19:07:50,390 DEBUG [cloud.async.AsyncJobManagerImpl]
> (Job-Executor-25:job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ]) Complete
> async job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56 ], jobStatus: 2,
> resultCode: 530, result: Error Code: 530 Error text: Failed to create
> templatejava.lang.NullPointerException
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:457)
> at com.cloud.utils.script.Script.execute(Script.java:183)
> at com.cloud.utils.script.Script.execute(Script.java:161)
> at
> org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfsXenserver(NfsSecondaryStorageResource.java:360)
> at
> org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.copySnapshotToTemplateFromNfsToNfs(NfsSecondaryStorageResource.java:401)
> at
> org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.createTemplateFromSnapshot(NfsSecondaryStorageResource.java:419)
> at
> org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.execute(NfsSecondaryStorageResource.java:445)
> at
> org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:208)
> at
> org.apache.cloudstack.storage.resource.LocalNfsSecondaryStorageResource.executeRequest(LocalNfsSecondaryStorageResource.java:76)
> at
> org.apache.cloudstack.storage.LocalHostEndpoint.sendMessage(LocalHostEndpoint.java:93)
> at
> org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.createTemplateFromSnapshot(AncientDataMotionStrategy.java:384)
> at
> org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:344)
> at
> org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55)
> at
> org.apache.cloudstack.storage.image.TemplateServiceImpl.copyAsync(TemplateServiceImpl.java:549)
> at
> org.apache.cloudstack.storage.image.TemplateServiceImpl.createTemplateFromSnapshotAsync(TemplateServiceImpl.java:556)
> at
> com.cloud.template.TemplateManagerImpl.createPrivateTemplate(TemplateManagerImpl.java:1355)
> at
> com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f.CGLIB$createPrivateTemplate$32(<generated>)
> at
> com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f_FastClassByCloudStack_738e8b.invoke(<generated>)
> at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
> at
> com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
> at
> com.cloud.template.TemplateManagerImpl_EnhancerByCloudStack_9244e17f.createPrivateTemplate(<generated>)
> at
> org.apache.cloudstack.api.command.user.template.CreateTemplateCmd.execute(CreateTemplateCmd.java:263)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
> at
> com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2013-07-24 19:07:52,603 DEBUG [cloud.api.ApiServlet] (catalina-exec-15:null)
> ===START=== 10.144.6.19 -- GET
> command=queryAsyncJobResult&jobId=9d0965c2-23ce-4c82-abec-f68c5e523c56&response=json&sessionkey=AFany%2FrOuQtuA2%2FzjVw%2F4ryjKsc%3D&_=1374653618676
> 2013-07-24 19:07:52,629 DEBUG [cloud.async.AsyncJobManagerImpl]
> (catalina-exec-15:null) Async job-85 = [ 9d0965c2-23ce-4c82-abec-f68c5e523c56
> ] completed
> 2013-07-24 19:07:52,640 DEBUG [cloud.api.ApiServlet] (catalina-exec-15:null)
> ===END=== 10.144.6.19 -- GET
> command=queryAsyncJobResult&jobId=9d0965c2-23ce-4c82-abec-f68c5e523c56&response=json&sessionkey=AFany%2FrOuQtuA2%2FzjVw%2F4ryjKsc%3D&_=1374653618676
> 2013-07-24 19:07:52,769 DEBUG [agent.manager.AgentManagerImpl]
> (AgentManager-Handler-11:null) SeqA 10-8916: Processing Seq 10-8916: { Cmd ,
> MgmtId: -1, via: 10, Ver: v1, Flags: 11,
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":18,"_loadInfo":"{\n
> \"connections\": []\n}","wait":0}}] }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
