Re: ACS 14.16.1::XCP-ng 8.2.1 - Secondary storage mount issue|Volume Snapshot

Mon, 04 Apr 2022 23:31:59 -0700 

Hi,
What Vivek said, also make sure there is no firewall between hypervisor and sec storage, or at least disable it temporarily to rule things out. NFSv3 and v4 are slightly different at network level, some port difference etc. 

---
Nux!
www.nux.ro

On 2022-04-05 07:25, Vivek Kumar wrote:

Hey Midhun,

This error is you see is coming from the XenServer, looks like it’s
not able to mount the NFS on the destination directory, I know you
have verified the mount point but can you try to manually mount your
secondary storage on XenServer, also try checking the global settings
- secstorage.nfs.version to check and enforce your nfs version.

Regards,
Vivek Kumar





On 05-Apr-2022, at 11:25 AM, Midhun Jose 
<[email protected]> wrote:



Hi @all,


I am using Apache cloudstack 4.16.1 integrated with XCP-ng 8.2.1. I am 
not able to take VM volume snapshot and sometimes vm creation is also 
not working. I am getting the error as attached screenshot and while 
checking the logs we are getting below error,



P.S: I have veriifed that nfs server is working fine and secondary 
storage is mounted in xen server. Can someome piont out the reason on 
this..


~~~~~~

2022-04-04 13:13:44,163 WARN  [c.c.h.x.r.CitrixResourceBase] 
(DirectAgent-17:ctx-f6381170) (logid:4835671f) Task failed! Task 
record:                 uuid: e907d695-825f-24e4-5ce3-afd74eb0ece4

           nameLabel: Async.host.call_plugin
     nameDescription:
   allowedOperations: []
   currentOperations: {}
             created: Mon Apr 04 13:13:43 IST 2022
            finished: Mon Apr 04 13:13:43 IST 2022
              status: failure
          residentOn: com.xensource.xenapi.Host@5fa0255a
            progress: 1.0
                type: <none/>
              result:

           errorInfo: [XENAPI_PLUGIN_FAILURE, 
mountNfsSecondaryStorage, SROSError, Error reporting error, unknown 
key Unexpected error while trying to mount 
10.31.9.226:/storage/cloudstack/secondary/ to 
/var/cloud_mount/92ac41e2-5210-36ff-8d7b-97407b9fcfd3]

         otherConfig: {}
           subtaskOf: com.xensource.xenapi.Task@aaf13f6f
            subtasks: []

2022-04-04 13:13:44,173 WARN  [c.c.h.x.r.CitrixResourceBase] 
(DirectAgent-17:ctx-f6381170) (logid:4835671f) callHostPlugin failed 
for cmd: mountNfsSecondaryStorage with args remoteDir: 
10.31.9.226:/storage/cloudstack/secondary/, localDir: 
/var/cloud_mount/92ac41e2-5210-36ff-8d7b-97407b9fcfd3, nfsVersion: 
null,  due to Task failed! Task record:                 uuid: 
e907d695-825f-24e4-5ce3-afd74eb0ece4

           nameLabel: Async.host.call_plugin
     nameDescription:
   allowedOperations: []
   currentOperations: {}
             created: Mon Apr 04 13:13:43 IST 2022
            finished: Mon Apr 04 13:13:43 IST 2022
              status: failure
          residentOn: com.xensource.xenapi.Host@5fa0255a
            progress: 1.0
                type: <none/>
              result:

           errorInfo: [XENAPI_PLUGIN_FAILURE, 
mountNfsSecondaryStorage, SROSError, Error reporting error, unknown 
key Unexpected error while trying to mount 
10.31.9.226:/storage/cloudstack/secondary/ to 
/var/cloud_mount/92ac41e2-5210-36ff-8d7b-97407b9fcfd3]

         otherConfig: {}
           subtaskOf: com.xensource.xenapi.Task@aaf13f6f
            subtasks: []

Task failed! Task record:                 uuid: 
e907d695-825f-24e4-5ce3-afd74eb0ece4

           nameLabel: Async.host.call_plugin
     nameDescription:
   allowedOperations: []
   currentOperations: {}
             created: Mon Apr 04 13:13:43 IST 2022
            finished: Mon Apr 04 13:13:43 IST 2022
              status: failure
          residentOn: com.xensource.xenapi.Host@5fa0255a
            progress: 1.0
                type: <none/>
              result:

           errorInfo: [XENAPI_PLUGIN_FAILURE, 
mountNfsSecondaryStorage, SROSError, Error reporting error, unknown 
key Unexpected error while trying to mount 
10.31.9.226:/storage/cloudstack/secondary/ to 
/var/cloud_mount/92ac41e2-5210-36ff-8d7b-97407b9fcfd3]

         otherConfig: {}
           subtaskOf: com.xensource.xenapi.Task@aaf13f6f
            subtasks: []

Task failed! Task record:                 uuid: 
e907d695-825f-24e4-5ce3-afd74eb0ece4

           nameLabel: Async.host.call_plugin
     nameDescription:
   allowedOperations: []
   currentOperations: {}
             created: Mon Apr 04 13:13:43 IST 2022
            finished: Mon Apr 04 13:13:43 IST 2022
              status: failure
          residentOn: com.xensource.xenapi.Host@5fa0255a
            progress: 1.0
                type: <none/>
              result:

           errorInfo: [XENAPI_PLUGIN_FAILURE, 
mountNfsSecondaryStorage, SROSError, Error reporting error, unknown 
key Unexpected error while trying to mount 
10.31.9.226:/storage/cloudstack/secondary/ to 
/var/cloud_mount/92ac41e2-5210-36ff-8d7b-97407b9fcfd3]

         otherConfig: {}
           subtaskOf: com.xensource.xenapi.Task@aaf13f6f
            subtasks: []

        at 
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.checkForSuccess(CitrixResourceBase.java:500)
        at 
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.callHostPluginAsync(CitrixResourceBase.java:422)
        at 
com.cloud.hypervisor.xenserver.resource.Xenserver625StorageProcessor.mountNfs(Xenserver625StorageProcessor.java:80)
        at 
com.cloud.hypervisor.xenserver.resource.Xenserver625StorageProcessor.createFileSr(Xenserver625StorageProcessor.java:249)
        at 
com.cloud.hypervisor.xenserver.resource.Xenserver625StorageProcessor.copyTemplateToPrimaryStorage(Xenserver625StorageProcessor.java:286)
        at 
com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.execute(StorageSubsystemCommandHandlerBase.java:95)
        at 
com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.handleStorageCommands(StorageSubsystemCommandHandlerBase.java:59)
        at 
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStorageSubSystemCommandWrapper.execute(CitrixStorageSubSystemCommandWrapper.java:36)
        at 
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStorageSubSystemCommandWrapper.execute(CitrixStorageSubSystemCommandWrapper.java:30)
        at 
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122)
        at 
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1758)
        at 
com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315)
        at 
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
        at 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
        at 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
        at 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
        at 
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
        at 
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
        at 
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)

        at java.base/java.lang.Thread.run(Thread.java:829)

2022-04-04 13:13:44,180 DEBUG [c.c.h.x.r.CitrixResourceBase] 
(DirectAgent-17:ctx-f6381170) (logid:4835671f) unable to destroy 
task(com.xensource.xenapi.Task@3cd54231) on 
host(bc164e35-7efe-4ff7-9460-1c92cdb44396) due to You gave an invalid 
object reference.  The object may have recently been deleted.  The 
class parameter gives the type of reference given, and the handle 
parameter echoes the bad value given.
2022-04-04 13:13:44,180 WARN  [c.c.h.x.r.XenServerStorageProcessor] 
(DirectAgent-17:ctx-f6381170) (logid:4835671f) Could not mount 
secondary storage 10.31.9.226:/storage/cloudstack/secondary/ on host 
/var/cloud_mount/92ac41e2-5210-36ff-8d7b-97407b9fcfd3
2022-04-04 13:13:44,181 WARN  [c.c.h.x.r.XenServerStorageProcessor] 
(DirectAgent-17:ctx-f6381170) (logid:4835671f) Catch Exception 
com.cloud.utils.exception.CloudRuntimeException for template due to 
com.cloud.utils.exception.CloudRuntimeException: Could not mount 
secondary storage 10.31.9.226:/storage/cloudstack/secondary/ on host 
/var/cloud_mount/92ac41e2-5210-36ff-8d7b-97407b9fcfd3
com.cloud.utils.exception.CloudRuntimeException: Could not mount 
secondary storage 10.31.9.226:/storage/cloudstack/secondary/ on host 
/var/cloud_mount/92ac41e2-5210-36ff-8d7b-97407b9fcfd3
        at 
com.cloud.hypervisor.xenserver.resource.Xenserver625StorageProcessor.mountNfs(Xenserver625StorageProcessor.java:84)
        at 
com.cloud.hypervisor.xenserver.resource.Xenserver625StorageProcessor.createFileSr(Xenserver625StorageProcessor.java:249)
        at 
com.cloud.hypervisor.xenserver.resource.Xenserver625StorageProcessor.copyTemplateToPrimaryStorage(Xenserver625StorageProcessor.java:286)
        at 
com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.execute(StorageSubsystemCommandHandlerBase.java:95)
        at 
com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.handleStorageCommands(StorageSubsystemCommandHandlerBase.java:59)
        at 
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStorageSubSystemCommandWrapper.execute(CitrixStorageSubSystemCommandWrapper.java:36)
        at 
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStorageSubSystemCommandWrapper.execute(CitrixStorageSubSystemCommandWrapper.java:30)
        at 
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122)
        at 
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1758)
        at 
com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttach
2022-04-04 13:13:44,163 WARN  [c.c.h.x.r.CitrixResourceBase] 
(DirectAgent-17:ctx-f6381170) (logid:4835671f) Task failed! Task 
record:                 uuid: e907d695-825f-24e4-5ce3-afd74eb0ece4

           nameLabel: Async.host.call_plugin
     nameDescription:
   allowedOperations: []
   currentOperations: {}
             created: Mon Apr 04 13:13:43 IST 2022
            finished: Mon Apr 04 13:13:43 IST 2022
              status: failure
          residentOn: com.xensource.xenapi.Host@5fa0255a
            progress: 1.0
                type: <none/>
              result:

           errorInfo: [XENAPI_PLUGIN_FAILURE, 
mountNfsSecondaryStorage, SROSError, Error reporting error, unknown 
key Unexpected error while trying to mount 
10.31.9.226:/storage/cloudstack/secondary/ to 
/var/cloud_mount/92ac41e2-5210-36ff-8d7b-97407b9fcfd3]

         otherConfig: {}
           subtaskOf: com.xensource.xenapi.Task@aaf13f6f
            subtasks: []

2022-04-04 13:13:44,173 WARN  [c.c.h.x.r.CitrixResourceBase] 
(DirectAgent-17:ctx-f6381170) (logid:4835671f) callHostPlugin failed 
for cmd: mountNfsSecondaryStorage with args remoteDir: 
10.31.9.226:/storage/cloudstack/secondary/, localDir: 
/var/cloud_mount/92ac41e2-5210-36ff-8d7b-97407b9fcfd3, nfsVersion: 
null,  due to Task failed! Task record:                 uuid: 
e907d695-825f-24e4-5ce3-afd74eb0ece4

           nameLabel: Async.host.call_plugin
     nameDescription:
   allowedOperations: []
   currentOperations: {}
             created: Mon Apr 04 13:13:43 IST 2022
            finished: Mon Apr 04 13:13:43 IST 2022
              status: failure
          residentOn: com.xensource.xenapi.Host@5fa0255a
            progress: 1.0
                type: <none/>
              result:

           errorInfo: [XENAPI_PLUGIN_FAILURE, 
mountNfsSecondaryStorage, SROSError, Error reporting error, unknown 
key Unexpected error while trying to mount 
10.31.9.226:/storage/cloudstack/secondary/ to 
/var/cloud_mount/92ac41e2-5210-36ff-8d7b-97407b9fcfd3]

         otherConfig: {}
           subtaskOf: com.xensource.xenapi.Task@aaf13f6f
            subtasks: []

Task failed! Task record:                 uuid: 
e907d695-825f-24e4-5ce3-afd74eb0ece4

           nameLabel: Async.host.call_plugin
     nameDescription:
   allowedOperations: []
   currentOperations: {}
             created: Mon Apr 04 13:13:43 IST 2022
            finished: Mon Apr 04 13:13:43 IST 2022
              status: failure
          residentOn: com.xensource.xenapi.Host@5fa0255a
            progress: 1.0
                type: <none/>
              result:

           errorInfo: [XENAPI_PLUGIN_FAILURE, 
mountNfsSecondaryStorage, SROSError, Error reporting error, unknown 
key Unexpected error while trying to mount 
10.31.9.226:/storage/cloudstack/secondary/ to 
/var/cloud_mount/92ac41e2-5210-36ff-8d7b-97407b9fcfd3]

         otherConfig: {}
           subtaskOf: com.xensource.xenapi.Task@aaf13f6f
            subtasks: []

Task failed! Task record:                 uuid: 
e907d695-825f-24e4-5ce3-afd74eb0ece4

           nameLabel: Async.host.call_plugin
     nameDescription:
   allowedOperations: []
   currentOperations: {}
             created: Mon Apr 04 13:13:43 IST 2022
            finished: Mon Apr 04 13:13:43 IST 2022
              status: failure
          residentOn: com.xensource.xenapi.Host@5fa0255a
            progress: 1.0
                type: <none/>
              result:

           errorInfo: [XENAPI_PLUGIN_FAILURE, 
mountNfsSecondaryStorage, SROSError, Error reporting error, unknown 
key Unexpected error while trying to mount 
10.31.9.226:/storage/cloudstack/secondary/ to 
/var/cloud_mount/92ac41e2-5210-36ff-8d7b-97407b9fcfd3]

         otherConfig: {}
           subtaskOf: com.xensource.xenapi.Task@aaf13f6f
            subtasks: []

        at 
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.checkForSuccess(CitrixResourceBase.java:500)
        at 
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.callHostPluginAsync(CitrixResourceBase.java:422)
        at 
com.cloud.hypervisor.xenserver.resource.Xenserver625StorageProcessor.mountNfs(Xenserver625StorageProcessor.java:80)
        at 
com.cloud.hypervisor.xenserver.resource.Xenserver625StorageProcessor.createFileSr(Xenserver625StorageProcessor.java:249)
        at 
com.cloud.hypervisor.xenserver.resource.Xenserver625StorageProcessor.copyTemplateToPrimaryStorage(Xenserver625StorageProcessor.java:286)
        at 
com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.execute(StorageSubsystemCommandHandlerBase.java:95)
        at 
com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.handleStorageCommands(StorageSubsystemCommandHandlerBase.java:59)
        at 
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStorageSubSystemCommandWrapper.execute(CitrixStorageSubSystemCommandWrapper.java:36)
        at 
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStorageSubSystemCommandWrapper.execute(CitrixStorageSubSystemCommandWrapper.java:30)
        at 
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122)
        at 
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1758)
        at 
com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315)
        at 
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
        at 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
        at 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
        at 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
        at 
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
        at 
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at 
java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at 
java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
        at 
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at 
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)

        at java.base/java.lang.Thread.run(Thread.java:829)

2022-04-04 13:13:44,180 DEBUG [c.c.h.x.r.CitrixResourceBase] 
(DirectAgent-17:ctx-f6381170) (logid:4835671f) unable to destroy 
task(com.xensource.xenapi.Task@3cd54231) on 
host(bc164e35-7efe-4ff7-9460-1c92cdb44396) due to You gave an invalid 
object reference.  The object may have recently been deleted.  The 
class parameter gives the type of reference given, and the handle 
parameter echoes the bad value given.
2022-04-04 13:13:44,180 WARN  [c.c.h.x.r.XenServerStorageProcessor] 
(DirectAgent-17:ctx-f6381170) (logid:4835671f) Could not mount 
secondary storage 10.31.9.226:/storage/cloudstack/secondary/ on host 
/var/cloud_mount/92ac41e2-5210-36ff-8d7b-97407b9fcfd3
2022-04-04 13:13:44,181 WARN  [c.c.h.x.r.XenServerStorageProcessor] 
(DirectAgent-17:ctx-f6381170) (logid:4835671f) Catch Exception 
com.cloud.utils.exception.CloudRuntimeException for template due to 
com.cloud.utils.exception.CloudRuntimeException: Could not mount 
secondary storage 10.31.9.226:/storage/cloudstack/secondary/ on host 
/var/cloud_mount/92ac41e2-5210-36ff-8d7b-97407b9fcfd3
com.cloud.utils.exception.CloudRuntimeException: Could not mount 
secondary storage 10.31.9.226:/storage/cloudstack/secondary/ on host 
/var/cloud_mount/92ac41e2-5210-36ff-8d7b-97407b9fcfd3
        at 
com.cloud.hypervisor.xenserver.resource.Xenserver625StorageProcessor.mountNfs(Xenserver625StorageProcessor.java:84)
        at 
com.cloud.hypervisor.xenserver.resource.Xenserver625StorageProcessor.createFileSr(Xenserver625StorageProcessor.java:249)
        at 
com.cloud.hypervisor.xenserver.resource.Xenserver625StorageProcessor.copyTemplateToPrimaryStorage(Xenserver625StorageProcessor.java:286)
        at 
com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.execute(StorageSubsystemCommandHandlerBase.java:95)
        at 
com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.handleStorageCommands(StorageSubsystemCommandHandlerBase.java:59)
        at 
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStorageSubSystemCommandWrapper.execute(CitrixStorageSubSystemCommandWrapper.java:36)
        at 
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStorageSubSystemCommandWrapper.execute(CitrixStorageSubSystemCommandWrapper.java:30)
        at 
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122)
        at 
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1758)
        at 
com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttach


~~~~~~


Best Regards,
Midhun























					Reply via email to