Hi Glenn/Prashant,
I am forwarding the exact log for the voloume snapshot
Please check and let us know the error
2016-05-29 01:07:10,021 WARN [xen.resource.CitrixResourceBase]
(DirectAgent-250:null) Task failed! Task record: uuid:
8f8e8e52-de8d-7310-22c5-030719540292
nameLabel: Async.host.call_plugin
nameDescription:
allowedOperations: []
currentOperations: {}
created: Sun May 29 01:04:54 PDT 2016
finished: Sun May 29 01:07:09 PDT 2016
status: failure
residentOn: com.xensource.xenapi.Host@2115a7db
progress: 1.0
type: <none/>
result:
errorInfo: [XENAPI_PLUGIN_FAILURE, backupSnapshot, CommandException,
32]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
2016-05-29 01:07:10,023 WARN [xen.resource.CitrixResourceBase]
(DirectAgent-250:null) callHostPlugin failed for cmd: backupSnapshot with args
secondaryStorageMountPath: 205.144.169.99:/export1/secondary, localMountPoint:
/var/run/cloud_mount/60ef4a9f-de15-3c55-a95e-7c471608332c, backupUuid:
6278b10c-3288-4e20-84b5-ada34b710e23, path: snapshots/3/1142, prevBackupUuid:
snapshots/3/1142/a3585a4c-c2a1-4fe6-8fff-1ec799ecb7d6, snapshotUuid:
66e1b2cf-68bb-4245-ae0d-8a1fc6df18e3, isISCSI: true, primaryStorageSRUuid:
48be6dd0-cb35-132e-2661-99842fadbaba, due to Task failed! Task record:
uuid: 8f8e8e52-de8d-7310-22c5-030719540292
nameLabel: Async.host.call_plugin
nameDescription:
allowedOperations: []
currentOperations: {}
created: Sun May 29 01:04:54 PDT 2016
finished: Sun May 29 01:07:09 PDT 2016
status: failure
residentOn: com.xensource.xenapi.Host@2115a7db
progress: 1.0
type: <none/>
result:
errorInfo: [XENAPI_PLUGIN_FAILURE, backupSnapshot, CommandException,
32]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
Task failed! Task record: uuid:
8f8e8e52-de8d-7310-22c5-030719540292
nameLabel: Async.host.call_plugin
nameDescription:
allowedOperations: []
currentOperations: {}
created: Sun May 29 01:04:54 PDT 2016
finished: Sun May 29 01:07:09 PDT 2016
status: failure
residentOn: com.xensource.xenapi.Host@2115a7db
progress: 1.0
type: <none/>
result:
errorInfo: [XENAPI_PLUGIN_FAILURE, backupSnapshot, CommandException,
32]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.checkForSuccess(CitrixResourceBase.java:3605)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.callHostPluginAsync(CitrixResourceBase.java:3919)
at
com.cloud.hypervisor.xen.resource.XenServerStorageProcessor.backupSnapshot(XenServerStorageProcessor.java:1117)
at
com.cloud.hypervisor.xen.resource.XenServerStorageProcessor.backupSnapshot(XenServerStorageProcessor.java:1307)
at
com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.execute(StorageSubsystemCommandHandlerBase.java:84)
at
com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.handleStorageCommands(StorageSubsystemCommandHandlerBase.java:49)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:621)
at
com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73)
at
com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:104)
at
com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:679)
2016-05-29 01:07:11,040 WARN [xen.resource.XenServerStorageProcessor]
(DirectAgent-250:null) Could not copy backupUuid: null from primary storage
48be6dd0-cb35-132e-2661-99842fadbaba to secondary storage
205.144.169.99:/export1/secondary due to null
2016-05-29 01:07:11,040 WARN [xen.resource.XenServerStorageProcessor]
(DirectAgent-250:null) BackupSnapshot Failed due to Could not copy backupUuid:
null from primary storage 48be6dd0-cb35-132e-2661-99842fadbaba to secondary
storage 205.144.169.99:/export1/secondary due to null
com.cloud.utils.exception.CloudRuntimeException: Could not copy backupUuid:
null from primary storage 48be6dd0-cb35-132e-2661-99842fadbaba to secondary
storage 205.144.169.99:/export1/secondary due to null
at
com.cloud.hypervisor.xen.resource.XenServerStorageProcessor.backupSnapshot(XenServerStorageProcessor.java:1146)
at
com.cloud.hypervisor.xen.resource.XenServerStorageProcessor.backupSnapshot(XenServerStorageProcessor.java:1307)
at
com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.execute(StorageSubsystemCommandHandlerBase.java:84)
at
com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.handleStorageCommands(StorageSubsystemCommandHandlerBase.java:49)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:621)
at
com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73)
at
com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73)
at
com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:104)
at
com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:679)
2016-05-29 01:07:11,041 DEBUG [agent.manager.DirectAgentAttache]
(DirectAgent-250:null) Seq 39-2062304528: Response Received:
2016-05-29 01:07:11,041 DEBUG [agent.transport.Request] (DirectAgent-250:null)
Seq 39-2062304528: Processing: { Ans: , MgmtId: 120860783173, via: 39, Ver:
v1, Flags: 110,
[{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"result":false,"details":"BackupSnapshot
Failed due to Could not copy backupUuid: null from primary storage
48be6dd0-cb35-132e-2661-99842fadbaba to secondary storage
205.144.169.99:/export1/secondary due to null","wait":0}}] }
2016-05-29 01:07:11,042 DEBUG [agent.manager.AgentAttache]
(DirectAgent-250:null) Seq 39-2062304541: Sending now. is current sequence.
2016-05-29 01:07:11,042 DEBUG [agent.transport.Request]
(Job-Executor-10:job-31466 = [ f3aca400-b871-41b9-927d-18b1bd8b0635 ]) Seq
39-2062304528: Received: { Ans: , MgmtId: 120860783173, via: 39, Ver: v1,
Flags: 110, { CopyCmdAnswer } }
2016-05-29 01:07:11,042 DEBUG [agent.transport.Request] (DirectAgent-250:null)
Seq 39-2062304541: Executing: { Cmd , MgmtId: 120860783173, via: 39, Ver: v1,
Flags: 100111,
[{"com.cloud.agent.api.CheckS2SVpnConnectionsCommand":{"vpnIps":[],"accessDetails":{"router.ip":"169.254.1.111","router.name":"r-781-VM"},"wait":30}}]
}
2016-05-29 01:07:11,042 DEBUG [agent.manager.DirectAgentAttache]
(DirectAgent-250:null) Seq 39-2062304541: Executing request
2016-05-29 01:07:11,056 DEBUG [storage.snapshot.SnapshotManagerImpl]
(Job-Executor-10:job-31466 = [ f3aca400-b871-41b9-927d-18b1bd8b0635 ]) Failed
to create snapshot
com.cloud.utils.exception.CloudRuntimeException: BackupSnapshot Failed due to
Could not copy backupUuid: null from primary storage
48be6dd0-cb35-132e-2661-99842fadbaba to secondary storage
205.144.169.99:/export1/secondary due to null
at
org.apache.cloudstack.storage.snapshot.SnapshotServiceImpl.backupSnapshot(SnapshotServiceImpl.java:280)
at
org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.backupSnapshot(XenserverSnapshotStrategy.java:138)
at
org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.takeSnapshot(XenserverSnapshotStrategy.java:264)
at
com.cloud.storage.snapshot.SnapshotManagerImpl.takeSnapshot(SnapshotManagerImpl.java:1013)
at
com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
at
org.apache.cloudstack.storage.volume.VolumeServiceImpl.takeSnapshot(VolumeServiceImpl.java:1307)
at
com.cloud.storage.VolumeManagerImpl.takeSnapshot(VolumeManagerImpl.java:2719)
at
org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd.execute(CreateSnapshotCmd.java:170)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
at
com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:679)
2016-05-29 01:07:11,063 DEBUG [storage.volume.VolumeServiceImpl]
(Job-Executor-10:job-31466 = [ f3aca400-b871-41b9-927d-18b1bd8b0635 ]) Take
snapshot: 1142 failed
com.cloud.utils.exception.CloudRuntimeException: Failed to create snapshot
at
com.cloud.storage.snapshot.SnapshotManagerImpl.takeSnapshot(SnapshotManagerImpl.java:1040)
at
com.cloud.storage.VolumeManagerImpl.takeSnapshot(VolumeManagerImpl.java:2719)
at
org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd.execute(CreateSnapshotCmd.java:170)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
at
com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:679)
Caused by: com.cloud.utils.exception.CloudRuntimeException: BackupSnapshot
Failed due to Could not copy backupUuid: null from primary storage
48be6dd0-cb35-132e-2661-99842fadbaba to secondary storage
205.144.169.99:/export1/secondary due to null
at
org.apache.cloudstack.storage.snapshot.SnapshotServiceImpl.backupSnapshot(SnapshotServiceImpl.java:280)
at
org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.backupSnapshot(XenserverSnapshotStrategy.java:138)
at
org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.takeSnapshot(XenserverSnapshotStrategy.java:264)
at
com.cloud.storage.snapshot.SnapshotManagerImpl.takeSnapshot(SnapshotManagerImpl.java:1013)
... 16 more
2016-05-29 01:07:11,070 DEBUG [cloud.async.AsyncJobManagerImpl]
(Job-Executor-10:job-31466 = [ f3aca400-b871-41b9-927d-18b1bd8b0635 ]) Complete
async job-31466 = [ f3aca400-b871-41b9-927d-18b1bd8b0635 ], jobStatus: 2,
resultCode: 530, result: Error Code: 530 Error text: Failed to create snapshot
due to an internal error creating snapshot for volume 1142
-----Original Message-----
From: Prashant Mishra [mailto:[email protected]]
Sent: Friday, May 27, 2016 10:06 AM
To: [email protected]
Subject: Re: DAILY SNAPSHOT ISSUE FOR VOLUME SNAPSHOT
As Glenn said please provide MS log for further investigation , CREATED
ON PRIMARY is the state of snapshot ?
On 5/27/16, 9:57 AM, "Venkat Boggarapu" <[email protected]> wrote:
>Hi Glenn,
>
>I tried to remove the error and taken manual snapshot for the volume,
>manual snapshot is done, after that for two days daily snapshots is
>working good for that volume.
>
>After two days the error comes again as "CREATED ON PRIMARY". This is
>happening from last three to four weeks.
>
>Thanks,
>Venkat Boggarapu
>
>-----Original Message-----
>From: Glenn Wagner [mailto:[email protected]]
>Sent: Thursday, May 26, 2016 1:04 PM
>To: [email protected]
>Subject: RE: DAILY SNAPSHOT ISSUE FOR VOLUME SNAPSHOT
>
>Hi,
>
>Can you upload (pastebin) your management server logs for us to see the
>error,
>
>Regards
>Glenn
>
>[email protected]
>www.shapeblue.com
>2nd Floor, Oudehuis Centre, 122 Main Rd, Somerset West, Cape Town
>7130South Africa @shapeblue
>
>
>
>-----Original Message-----
>From: Venkat Boggarapu [mailto:[email protected]]
>Sent: Thursday, 26 May 2016 9:29 AM
>To: [email protected]
>Subject: DAILY SNAPSHOT ISSUE FOR VOLUME SNAPSHOT
>
>Hi Guys,
>
>My environment is Cloudstack-4.2.0 and Xenserver is 6.2.0
>
>I kept daily scheduled volume snapshots for all the vms in my environment.
>
>But from last three weeks I am getting error "CREATED ON PRIMARY" for
>one volume snapshot on ROOT volume.
>
>Please help me regarding if any one faced the same issue.
>
>Thanks,
>Venkat Boggarapu
DISCLAIMER
==========
This e-mail may contain privileged and confidential information which is the
property of Accelerite, a Persistent Systems business. It is intended only for
the use of the individual or entity to which it is addressed. If you are not
the intended recipient, you are not authorized to read, retain, copy, print,
distribute or use this message. If you have received this communication in
error, please notify the sender and delete all copies of this message.
Accelerite, a Persistent Systems business does not accept any liability for
virus infected mails.
