France created CLOUDSTACK-8044:
----------------------------------
Summary: Failed to create snapshot due to an internal error
creating snapshot for volume 372 -> Failure from sparse_dd: Fatal error:
exception Invalid_argument("index out of bounds")
Key: CLOUDSTACK-8044
URL: https://issues.apache.org/jira/browse/CLOUDSTACK-8044
Project: CloudStack
Issue Type: Bug
Security Level: Public (Anyone can view this level - this is the default.)
Components: Management Server
Affects Versions: 4.3.1
Environment: XS 6.0.2+Hotfixes, ACS 4.3.1
Reporter: France
After upgrading from CS 4.1.1 to CS 4.3.1, one of the instances can not get a
snapshot. There were/are some errors for all instances, as described here:
https://issues.apache.org/jira/browse/CLOUDSTACK-8013
but this is currently the only instance, where snapshots are actually not
working. This instance has had its root disk size extended and
cloud.volumes.size field updated manually in the DB accordingly after the
upgrade.
Suresh Babu, has supposedly tried the same procedure, but failed to get the
error as we did.
Here is the management server log:
014-12-08 08:46:35,858 DEBUG [c.c.a.ApiServlet]
(http-6443-exec-121:ctx-5b80bdcb) ===START=== XX.XX.XX.XX -- GET
command=createSnapshot&volumeid=e1cf5716-4af1-47cf-8f8d-84979ca55183&quiescevm=false&response=json&sessionkey=CENSOREDo%3D&_=1418024978323
2014-12-08 08:46:35,869 DEBUG [c.c.u.AccountManagerImpl]
(http-6443-exec-121:ctx-5b80bdcb ctx-992184e6) Access to
Acct[2f00e8d9-77b7-41eb-9aa4-2bf884268a3d-leoL] granted to
Acct[2f00e8d9-77b7-41eb-9aa4-2bf884268a3d-leoL] by DomainChecker
2014-12-08 08:46:35,882 DEBUG [c.c.u.AccountManagerImpl]
(http-6443-exec-121:ctx-5b80bdcb ctx-992184e6) Access to
org.apache.cloudstack.storage.volume.VolumeObject@6329ffac granted to
Acct[2f00e8d9-77b7-41eb-9aa4-2bf884268a3d-leoL] by DomainChecker
2014-12-08 08:46:35,951 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Job-Executor-20:ctx-f5bed97c) Add job-2834 into job monitoring
2014-12-08 08:46:35,951 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Job-Executor-20:ctx-f5bed97c) Executing AsyncJobVO {id:2834, userId: 43,
accountId: 45, instanceType: Snapshot, instanceId: 1224, cmd:
org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd, cmdInfo:
{"id":"1224","response":"json","sessionkey":"CENSOREDo\u003d","cmdEventType":"SNAPSHOT.CREATE","ctxUserId":"43","httpmethod":"GET","volumeid":"e1cf5716-4af1-47cf-8f8d-84979ca55183","_":"1418024978323","quiescevm":"false","ctxAccountId":"45","ctxStartEventId":"62631"},
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result:
null, initMsid: 95545481387, completeMsid: null, lastUpdated: null, lastPolled:
null, created: null}
2014-12-08 08:46:35,951 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(http-6443-exec-121:ctx-5b80bdcb ctx-992184e6) submit async job-2834, details:
AsyncJobVO {id:2834, userId: 43, accountId: 45, instanceType: Snapshot,
instanceId: 1224, cmd:
org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd, cmdInfo:
{"id":"1224","response":"json","sessionkey":"CENSOREDo\u003d","cmdEventType":"SNAPSHOT.CREATE","ctxUserId":"43","httpmethod":"GET","volumeid":"e1cf5716-4af1-47cf-8f8d-84979ca55183","_":"1418024978323","quiescevm":"false","ctxAccountId":"45","ctxStartEventId":"62631"},
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result:
null, initMsid: 95545481387, completeMsid: null, lastUpdated: null, lastPolled:
null, created: null}
2014-12-08 08:46:35,953 DEBUG [c.c.a.ApiServlet]
(http-6443-exec-121:ctx-5b80bdcb ctx-992184e6) ===END=== XX.XX.XX.XX -- GET
command=createSnapshot&volumeid=e1cf5716-4af1-47cf-8f8d-84979ca55183&quiescevm=false&response=json&sessionkey=CENSOREDo%3D&_=1418024978323
2014-12-08 08:46:35,959 DEBUG [c.c.u.AccountManagerImpl]
(Job-Executor-20:ctx-f5bed97c ctx-992184e6) Access to
Acct[2f00e8d9-77b7-41eb-9aa4-2bf884268a3d-leoL] granted to
Acct[2f00e8d9-77b7-41eb-9aa4-2bf884268a3d-leoL] by DomainChecker
2014-12-08 08:46:35,983 INFO [o.a.c.a.c.u.s.CreateSnapshotCmd]
(Job-Executor-20:ctx-f5bed97c ctx-992184e6) VOLSS: createSnapshotCmd
starts:1418024795983
2014-12-08 08:46:36,073 DEBUG [c.c.a.t.Request] (Job-Executor-20:ctx-f5bed97c
ctx-992184e6) Seq 4-2104434319: Sending { Cmd , MgmtId: 95545481387, via:
4(x4.c.some.domain), Ver: v1, Flags: 100011,
[{"org.apache.cloudstack.storage.command.CreateObjectCommand":{"data":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"volume":{"uuid":"e1cf5716-4af1-47cf-8f8d-84979ca55183","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"90c57ad9-fd30-39a9-930b-2163adc8798d","id":209,"poolType":"IscsiLUN","host":"some.storage.fqdn.1","path":"/iqn.2010-03.c.some.domain:storage.c.some.domain.s3/1","port":3260,"url":"IscsiLUN://some.storage.fqdn.1//iqn.2010-03.c.some.domain:storage.c.some.domain.s3/1/?ROLE=Primary&STOREUUID=90c57ad9-fd30-39a9-930b-2163adc8798d"}},"name":"ROOT-324","size":107374182400,"path":"b6aafe6d-0091-4131-801d-79bf6352c61a","volumeId":372,"vmName":"i-45-324-VM","accountId":45,"format":"VHD","id":372,"deviceId":0,"hypervisorType":"XenServer"},"dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"90c57ad9-fd30-39a9-930b-2163adc8798d","id":209,"poolType":"IscsiLUN","host":"some.storage.fqdn.1","path":"/iqn.2010-03.c.some.domain:storage.c.some.domain.s3/1","port":3260,"url":"IscsiLUN://some.storage.fqdn.1//iqn.2010-03.c.some.domain:storage.c.some.domain.s3/1/?ROLE=Primary&STOREUUID=90c57ad9-fd30-39a9-930b-2163adc8798d"}},"vmName":"i-45-324-VM","name":"VMnameXY_ROOT-324_20141208074635","hypervisorType":"XenServer","id":1224,"quiescevm":false,"physicalSize":0}},"wait":0}}]
}
2014-12-08 08:46:36,074 DEBUG [c.c.a.t.Request] (Job-Executor-20:ctx-f5bed97c
ctx-992184e6) Seq 4-2104434319: Executing: { Cmd , MgmtId: 95545481387, via:
4(x4.c.some.domain), Ver: v1, Flags: 100011,
[{"org.apache.cloudstack.storage.command.CreateObjectCommand":{"data":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"volume":{"uuid":"e1cf5716-4af1-47cf-8f8d-84979ca55183","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"90c57ad9-fd30-39a9-930b-2163adc8798d","id":209,"poolType":"IscsiLUN","host":"some.storage.fqdn.1","path":"/iqn.2010-03.c.some.domain:storage.c.some.domain.s3/1","port":3260,"url":"IscsiLUN://some.storage.fqdn.1//iqn.2010-03.c.some.domain:storage.c.some.domain.s3/1/?ROLE=Primary&STOREUUID=90c57ad9-fd30-39a9-930b-2163adc8798d"}},"name":"ROOT-324","size":107374182400,"path":"b6aafe6d-0091-4131-801d-79bf6352c61a","volumeId":372,"vmName":"i-45-324-VM","accountId":45,"format":"VHD","id":372,"deviceId":0,"hypervisorType":"XenServer"},"dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"90c57ad9-fd30-39a9-930b-2163adc8798d","id":209,"poolType":"IscsiLUN","host":"some.storage.fqdn.1","path":"/iqn.2010-03.c.some.domain:storage.c.some.domain.s3/1","port":3260,"url":"IscsiLUN://some.storage.fqdn.1//iqn.2010-03.c.some.domain:storage.c.some.domain.s3/1/?ROLE=Primary&STOREUUID=90c57ad9-fd30-39a9-930b-2163adc8798d"}},"vmName":"i-45-324-VM","name":"VMnameXY_ROOT-324_20141208074635","hypervisorType":"XenServer","id":1224,"quiescevm":false,"physicalSize":0}},"wait":0}}]
}
2014-12-08 08:46:36,074 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-55:ctx-c9141c38) Seq 4-2104434319: Executing request
2014-12-08 08:46:36,105 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-26:ctx-2dbaf7f0) Seq 4-2104434318: Response Received:
2014-12-08 08:46:36,106 DEBUG [c.c.a.t.Request] (StatsCollector-3:ctx-fdd617c4)
Seq 4-2104434318: Received: { Ans: , MgmtId: 95545481387, via: 4, Ver: v1,
Flags: 10, { GetHostStatsAnswer } }
2014-12-08 08:46:36,115 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-490:ctx-3bb693d5) Seq 18-40520662: Executing request
2014-12-08 08:46:36,323 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-490:ctx-3bb693d5) Seq 18-40520662: Response Received:
2014-12-08 08:46:36,324 DEBUG [c.c.a.t.Request] (StatsCollector-3:ctx-fdd617c4)
Seq 18-40520662: Received: { Ans: , MgmtId: 95545481387, via: 18, Ver: v1,
Flags: 10, { GetHostStatsAnswer } }
2014-12-08 08:46:36,331 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-264:ctx-e3747a2c) Seq 20-58412855: Executing request
2014-12-08 08:46:36,657 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-264:ctx-e3747a2c) Seq 20-58412855: Response Received:
2014-12-08 08:46:36,657 DEBUG [c.c.a.t.Request] (StatsCollector-3:ctx-fdd617c4)
Seq 20-58412855: Received: { Ans: , MgmtId: 95545481387, via: 20, Ver: v1,
Flags: 10, { GetHostStatsAnswer } }
2014-12-08 08:46:37,471 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-8:null) SeqA 27-899693: Processing Seq 27-899693: { Cmd
, MgmtId: -1, via: 27, Ver: v1, Flags: 11,
[{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":473,"_loadInfo":"{\n
\"connections\": []\n}","wait":0}}] }
2014-12-08 08:46:37,503 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-8:null) SeqA 27-899693: Sending Seq 27-899693: { Ans: ,
MgmtId: 95545481387, via: 27, Ver: v1, Flags: 100010,
[{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
2014-12-08 08:46:39,022 DEBUG [c.c.a.ApiServlet]
(http-6443-exec-119:ctx-a36727f4) ===START=== XX.XX.XX.XX -- GET
command=queryAsyncJobResult&jobId=762562cc-1049-4c5d-a947-672ba6bd8b96&response=json&sessionkey=CENSOREDo%3D&_=1418024981486
2014-12-08 08:46:39,051 DEBUG [c.c.a.ApiServlet]
(http-6443-exec-119:ctx-a36727f4 ctx-64cd7dcd) ===END=== XX.XX.XX.XX -- GET
command=queryAsyncJobResult&jobId=762562cc-1049-4c5d-a947-672ba6bd8b96&response=json&sessionkey=CENSOREDo%3D&_=1418024981486
2014-12-08 08:46:40,081 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-55:ctx-c9141c38) Seq 4-2104434319: Response Received:
2014-12-08 08:46:40,082 DEBUG [c.c.a.t.Request] (DirectAgent-55:ctx-c9141c38)
Seq 4-2104434319: Processing: { Ans: , MgmtId: 95545481387, via: 4, Ver: v1,
Flags: 10,
[{"org.apache.cloudstack.storage.command.CreateObjectAnswer":{"data":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"9d829e99-bc1d-41ee-a652-02df9125c1c9","id":0,"quiescevm":false,"physicalSize":0}},"result":true,"wait":0}}]
}
2014-12-08 08:46:40,082 DEBUG [c.c.a.t.Request] (Job-Executor-20:ctx-f5bed97c
ctx-992184e6) Seq 4-2104434319: Received: { Ans: , MgmtId: 95545481387, via:
4, Ver: v1, Flags: 10, { CreateObjectAnswer } }
2014-12-08 08:46:40,140 DEBUG [o.a.c.s.m.AncientDataMotionStrategy]
(Job-Executor-20:ctx-f5bed97c ctx-992184e6) copyAsync inspecting src type
SNAPSHOT copyAsync inspecting dest type SNAPSHOT
2014-12-08 08:46:40,184 DEBUG [c.c.a.t.Request] (Job-Executor-20:ctx-f5bed97c
ctx-992184e6) Seq 2-902104676: Sending { Cmd , MgmtId: 95545481387, via:
2(x2.c.some.domain), Ver: v1, Flags: 100011,
[{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"9d829e99-bc1d-41ee-a652-02df9125c1c9","volume":{"uuid":"e1cf5716-4af1-47cf-8f8d-84979ca55183","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"90c57ad9-fd30-39a9-930b-2163adc8798d","id":209,"poolType":"IscsiLUN","host":"some.storage.fqdn.1","path":"/iqn.2010-03.c.some.domain:storage.c.some.domain.s3/1","port":3260,"url":"IscsiLUN://some.storage.fqdn.1//iqn.2010-03.c.some.domain:storage.c.some.domain.s3/1/?ROLE=Primary&STOREUUID=90c57ad9-fd30-39a9-930b-2163adc8798d"}},"name":"ROOT-324","size":107374182400,"path":"b6aafe6d-0091-4131-801d-79bf6352c61a","volumeId":372,"vmName":"i-45-324-VM","accountId":45,"format":"VHD","id":372,"deviceId":0,"hypervisorType":"XenServer"},"dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"90c57ad9-fd30-39a9-930b-2163adc8798d","id":209,"poolType":"IscsiLUN","host":"some.storage.fqdn.1","path":"/iqn.2010-03.c.some.domain:storage.c.some.domain.s3/1","port":3260,"url":"IscsiLUN://some.storage.fqdn.1//iqn.2010-03.c.some.domain:storage.c.some.domain.s3/1/?ROLE=Primary&STOREUUID=90c57ad9-fd30-39a9-930b-2163adc8798d"}},"vmName":"i-45-324-VM","name":"VMnameXY_ROOT-324_20141208074635","hypervisorType":"XenServer","id":1224,"quiescevm":false,"physicalSize":0}},"destTO":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"snapshots/45/372","volume":{"uuid":"e1cf5716-4af1-47cf-8f8d-84979ca55183","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"90c57ad9-fd30-39a9-930b-2163adc8798d","id":209,"poolType":"IscsiLUN","host":"some.storage.fqdn.1","path":"/iqn.2010-03.c.some.domain:storage.c.some.domain.s3/1","port":3260,"url":"IscsiLUN://some.storage.fqdn.1//iqn.2010-03.c.some.domain:storage.c.some.domain.s3/1/?ROLE=Primary&STOREUUID=90c57ad9-fd30-39a9-930b-2163adc8798d"}},"name":"ROOT-324","size":107374182400,"path":"b6aafe6d-0091-4131-801d-79bf6352c61a","volumeId":372,"vmName":"i-45-324-VM","accountId":45,"format":"VHD","id":372,"deviceId":0,"hypervisorType":"XenServer"},"dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://s6.c.some.domain/mnt/rootPool/nfsSecondary","_role":"Image"}},"vmName":"i-45-324-VM","name":"VMnameXY_ROOT-324_20141208074635","hypervisorType":"XenServer","id":1224,"quiescevm":false,"physicalSize":0}},"executeInSequence":false,"options":{"fullSnapshot":"true"},"wait":21600}}]
}
2014-12-08 08:46:40,185 DEBUG [c.c.a.t.Request] (Job-Executor-20:ctx-f5bed97c
ctx-992184e6) Seq 2-902104676: Executing: { Cmd , MgmtId: 95545481387, via:
2(x2.c.some.domain), Ver: v1, Flags: 100011,
[{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"9d829e99-bc1d-41ee-a652-02df9125c1c9","volume":{"uuid":"e1cf5716-4af1-47cf-8f8d-84979ca55183","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"90c57ad9-fd30-39a9-930b-2163adc8798d","id":209,"poolType":"IscsiLUN","host":"some.storage.fqdn.1","path":"/iqn.2010-03.c.some.domain:storage.c.some.domain.s3/1","port":3260,"url":"IscsiLUN://some.storage.fqdn.1//iqn.2010-03.c.some.domain:storage.c.some.domain.s3/1/?ROLE=Primary&STOREUUID=90c57ad9-fd30-39a9-930b-2163adc8798d"}},"name":"ROOT-324","size":107374182400,"path":"b6aafe6d-0091-4131-801d-79bf6352c61a","volumeId":372,"vmName":"i-45-324-VM","accountId":45,"format":"VHD","id":372,"deviceId":0,"hypervisorType":"XenServer"},"dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"90c57ad9-fd30-39a9-930b-2163adc8798d","id":209,"poolType":"IscsiLUN","host":"some.storage.fqdn.1","path":"/iqn.2010-03.c.some.domain:storage.c.some.domain.s3/1","port":3260,"url":"IscsiLUN://some.storage.fqdn.1//iqn.2010-03.c.some.domain:storage.c.some.domain.s3/1/?ROLE=Primary&STOREUUID=90c57ad9-fd30-39a9-930b-2163adc8798d"}},"vmName":"i-45-324-VM","name":"VMnameXY_ROOT-324_20141208074635","hypervisorType":"XenServer","id":1224,"quiescevm":false,"physicalSize":0}},"destTO":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"snapshots/45/372","volume":{"uuid":"e1cf5716-4af1-47cf-8f8d-84979ca55183","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"90c57ad9-fd30-39a9-930b-2163adc8798d","id":209,"poolType":"IscsiLUN","host":"some.storage.fqdn.1","path":"/iqn.2010-03.c.some.domain:storage.c.some.domain.s3/1","port":3260,"url":"IscsiLUN://some.storage.fqdn.1//iqn.2010-03.c.some.domain:storage.c.some.domain.s3/1/?ROLE=Primary&STOREUUID=90c57ad9-fd30-39a9-930b-2163adc8798d"}},"name":"ROOT-324","size":107374182400,"path":"b6aafe6d-0091-4131-801d-79bf6352c61a","volumeId":372,"vmName":"i-45-324-VM","accountId":45,"format":"VHD","id":372,"deviceId":0,"hypervisorType":"XenServer"},"dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://s6.c.some.domain/mnt/rootPool/nfsSecondary","_role":"Image"}},"vmName":"i-45-324-VM","name":"VMnameXY_ROOT-324_20141208074635","hypervisorType":"XenServer","id":1224,"quiescevm":false,"physicalSize":0}},"executeInSequence":false,"options":{"fullSnapshot":"true"},"wait":21600}}]
}
2014-12-08 08:46:40,186 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-205:ctx-eadb0ebb) Seq 2-902104676: Executing request
2014-12-08 08:46:40,757 DEBUG [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-205:ctx-eadb0ebb) Creating a not shared SR for
nfs://s6.c.some.domain/mnt/rootPool/nfsSecondary/snapshots/45/372
2014-12-08 08:46:41,594 DEBUG [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-205:ctx-eadb0ebb) Checking 57ffd43a-d997-395a-9be8-51d4d8922805 or
SR de7d0b77-3559-8024-1dae-ac1ccdd088e0 on
XS[99627967-f074-4312-8958-89f1745cce5b-XX.XX.XX.XX]
2014-12-08 08:46:41,606 DEBUG [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-205:ctx-eadb0ebb) Host XX.XX.XX.XX
OpaqueRef:2f9d181d-70f7-4bc0-47aa-ab7cacc90323: Created a SR; UUID is
de7d0b77-3559-8024-1dae-ac1ccdd088e0 device config is
{serverpath=/mnt/rootPool/nfsSecondary/snapshots/45/372,
server=s6.c.some.domain}
2014-12-08 08:46:42,021 DEBUG [c.c.a.ApiServlet]
(http-6443-exec-110:ctx-f9a8ba10) ===START=== XX.XX.XX.XX -- GET
command=queryAsyncJobResult&jobId=762562cc-1049-4c5d-a947-672ba6bd8b96&response=json&sessionkey=CENSOREDo%3D&_=1418024984486
2014-12-08 08:46:42,043 DEBUG [c.c.a.ApiServlet]
(http-6443-exec-110:ctx-f9a8ba10 ctx-efef7156) ===END=== XX.XX.XX.XX -- GET
command=queryAsyncJobResult&jobId=762562cc-1049-4c5d-a947-672ba6bd8b96&response=json&sessionkey=CENSOREDo%3D&_=1418024984486
2014-12-08 08:46:42,493 DEBUG [c.c.c.ConsoleProxyManagerImpl]
(consoleproxy-1:ctx-1e3cd74d) Zone 1 is ready to launch console proxy
2014-12-08 08:46:42,581 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
(secstorage-1:ctx-29320eaf) Zone 1 is ready to launch secondary storage VM
2014-12-08 08:46:45,023 DEBUG [c.c.a.ApiServlet]
(http-6443-exec-116:ctx-b1a42d14) ===START=== XX.XX.XX.XX -- GET
command=queryAsyncJobResult&jobId=762562cc-1049-4c5d-a947-672ba6bd8b96&response=json&sessionkey=CENSOREDo%3D&_=1418024987487
2014-12-08 08:46:45,053 DEBUG [c.c.a.ApiServlet]
(http-6443-exec-116:ctx-b1a42d14 ctx-ca1e53d1) ===END=== XX.XX.XX.XX -- GET
command=queryAsyncJobResult&jobId=762562cc-1049-4c5d-a947-672ba6bd8b96&response=json&sessionkey=CENSOREDo%3D&_=1418024987487
2014-12-08 08:46:47,471 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-1:null) SeqA 27-899694: Processing Seq 27-899694: { Cmd
, MgmtId: -1, via: 27, Ver: v1, Flags: 11,
[{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":473,"_loadInfo":"{\n
\"connections\": []\n}","wait":0}}] }
2014-12-08 08:46:48,023 DEBUG [c.c.a.ApiServlet]
(http-6443-exec-118:ctx-4fda25ac) ===START=== XX.XX.XX.XX -- GET
command=queryAsyncJobResult&jobId=762562cc-1049-4c5d-a947-672ba6bd8b96&response=json&sessionkey=CENSOREDo%3D&_=1418024990488
2014-12-08 08:46:49,820 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-1:null) SeqA 27-899694: Sending Seq 27-899694: { Ans: ,
MgmtId: 95545481387, via: 27, Ver: v1, Flags: 100010,
[{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
2014-12-08 08:46:49,820 DEBUG [c.c.c.ClusterManagerImpl]
(Cluster-Heartbeat-1:ctx-05ac6324) Management server heartbeat takes too long
to finish. profiler: Done. Duration: 2388ms, profilerHeartbeatUpdate: Done.
Duration: 2385ms, profilerPeerScan: Done. Duration: 3ms
2014-12-08 08:46:49,828 DEBUG [c.c.a.ApiServlet]
(http-6443-exec-118:ctx-4fda25ac ctx-42973d6d) ===END=== XX.XX.XX.XX -- GET
command=queryAsyncJobResult&jobId=762562cc-1049-4c5d-a947-672ba6bd8b96&response=json&sessionkey=CENSOREDo%3D&_=1418024990488
2014-12-08 08:46:51,023 DEBUG [c.c.a.ApiServlet]
(http-6443-exec-120:ctx-ad2b5b3c) ===START=== XX.XX.XX.XX -- GET
command=queryAsyncJobResult&jobId=762562cc-1049-4c5d-a947-672ba6bd8b96&response=json&sessionkey=CENSOREDo%3D&_=1418024993488
2014-12-08 08:46:51,085 DEBUG [c.c.a.ApiServlet]
(http-6443-exec-120:ctx-ad2b5b3c ctx-ef4952d2) ===END=== XX.XX.XX.XX -- GET
command=queryAsyncJobResult&jobId=762562cc-1049-4c5d-a947-672ba6bd8b96&response=json&sessionkey=CENSOREDo%3D&_=1418024993488
2014-12-08 08:46:54,024 DEBUG [c.c.a.ApiServlet]
(http-6443-exec-121:ctx-d365a166) ===START=== XX.XX.XX.XX -- GET
command=queryAsyncJobResult&jobId=762562cc-1049-4c5d-a947-672ba6bd8b96&response=json&sessionkey=CENSOREDo%3D&_=1418024996489
2014-12-08 08:46:54,043 DEBUG [c.c.a.ApiServlet]
(http-6443-exec-121:ctx-d365a166 ctx-3aa2a5d6) ===END=== XX.XX.XX.XX -- GET
command=queryAsyncJobResult&jobId=762562cc-1049-4c5d-a947-672ba6bd8b96&response=json&sessionkey=CENSOREDo%3D&_=1418024996489
2014-12-08 08:46:57,026 DEBUG [c.c.a.ApiServlet]
(http-6443-exec-119:ctx-7cb8bab2) ===START=== XX.XX.XX.XX -- GET
command=queryAsyncJobResult&jobId=762562cc-1049-4c5d-a947-672ba6bd8b96&response=json&sessionkey=CENSOREDo%3D&_=1418024999490
2014-12-08 08:46:57,045 DEBUG [c.c.a.ApiServlet]
(http-6443-exec-119:ctx-7cb8bab2 ctx-81dcf0d5) ===END=== XX.XX.XX.XX -- GET
command=queryAsyncJobResult&jobId=762562cc-1049-4c5d-a947-672ba6bd8b96&response=json&sessionkey=CENSOREDo%3D&_=1418024999490
2014-12-08 08:46:57,433 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-2:null) SeqA 27-899695: Processing Seq 27-899695: { Cmd
, MgmtId: -1, via: 27, Ver: v1, Flags: 11,
[{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":473,"_loadInfo":"{\n
\"connections\": []\n}","wait":0}}] }
2014-12-08 08:46:57,437 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-2:null) SeqA 27-899695: Sending Seq 27-899695: { Ans: ,
MgmtId: 95545481387, via: 27, Ver: v1, Flags: 100010,
[{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
2014-12-08 08:46:57,922 WARN [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-205:ctx-eadb0ebb) Task failed! Task record: uuid:
88d03ed7-d996-aad2-974d-6ab74c3656cc
nameLabel: Async.VDI.copy
nameDescription:
allowedOperations: []
currentOperations: {}
created: Mon Dec 08 08:46:43 CET 2014
finished: Mon Dec 08 08:46:57 CET 2014
status: failure
residentOn: com.xensource.xenapi.Host@c612622b
progress: 1.0
type: <none/>
result:
errorInfo: [INTERNAL_ERROR, Failure("sparse_dd")]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
2014-12-08 08:46:57,937 DEBUG [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-205:ctx-eadb0ebb) Host XX.XX.XX.XX
OpaqueRef:2f9d181d-70f7-4bc0-47aa-ab7cacc90323: Removing SR
2014-12-08 08:46:57,948 DEBUG [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-205:ctx-eadb0ebb) Host XX.XX.XX.XX
OpaqueRef:56da730b-b5dc-f972-6a8e-74ee643ac7a8: Unplugging pbd
2014-12-08 08:47:00,026 DEBUG [c.c.a.ApiServlet]
(http-6443-exec-110:ctx-54e9aca1) ===START=== XX.XX.XX.XX -- GET
command=queryAsyncJobResult&jobId=762562cc-1049-4c5d-a947-672ba6bd8b96&response=json&sessionkey=CENSOREDo%3D&_=1418025002491
2014-12-08 08:47:00,047 DEBUG [c.c.a.ApiServlet]
(http-6443-exec-110:ctx-54e9aca1 ctx-619a48e5) ===END=== XX.XX.XX.XX -- GET
command=queryAsyncJobResult&jobId=762562cc-1049-4c5d-a947-672ba6bd8b96&response=json&sessionkey=CENSOREDo%3D&_=1418025002491
2014-12-08 08:47:01,013 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-ce02975d) Found 15 routers to update status.
2014-12-08 08:47:01,016 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-ce02975d) Found 0 networks to update RvR status.
2014-12-08 08:47:01,586 DEBUG [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-205:ctx-eadb0ebb) Host XX.XX.XX.XX
OpaqueRef:2f9d181d-70f7-4bc0-47aa-ab7cacc90323: Forgetting
2014-12-08 08:47:01,604 WARN [c.c.h.x.r.XenServerStorageProcessor]
(DirectAgent-205:ctx-eadb0ebb) BackupSnapshot Failed due to Task failed! Task
record: uuid: 88d03ed7-d996-aad2-974d-6ab74c3656cc
nameLabel: Async.VDI.copy
nameDescription:
allowedOperations: []
currentOperations: {}
created: Mon Dec 08 08:46:43 CET 2014
finished: Mon Dec 08 08:46:57 CET 2014
status: failure
residentOn: com.xensource.xenapi.Host@c612622b
progress: 1.0
type: <none/>
result:
errorInfo: [INTERNAL_ERROR, Failure("sparse_dd")]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
Task failed! Task record: uuid:
88d03ed7-d996-aad2-974d-6ab74c3656cc
nameLabel: Async.VDI.copy
nameDescription:
allowedOperations: []
currentOperations: {}
created: Mon Dec 08 08:46:43 CET 2014
finished: Mon Dec 08 08:46:57 CET 2014
status: failure
residentOn: com.xensource.xenapi.Host@c612622b
progress: 1.0
type: <none/>
result:
errorInfo: [INTERNAL_ERROR, Failure("sparse_dd")]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.checkForSuccess(CitrixResourceBase.java:3764)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.cloudVDIcopy(CitrixResourceBase.java:3942)
at
com.cloud.hypervisor.xen.resource.XenServerStorageProcessor.backupSnapshot(XenServerStorageProcessor.java:1345)
at
com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.execute(StorageSubsystemCommandHandlerBase.java:90)
at
com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.handleStorageCommands(StorageSubsystemCommandHandlerBase.java:50)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:609)
at
com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:59)
at
com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:216)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:701)
2014-12-08 08:47:02,694 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-205:ctx-eadb0ebb) Seq 2-902104676: Response Received:
2014-12-08 08:47:02,694 DEBUG [c.c.a.t.Request] (DirectAgent-205:ctx-eadb0ebb)
Seq 2-902104676: Processing: { Ans: , MgmtId: 95545481387, via: 2, Ver: v1,
Flags: 10,
[{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"result":false,"details":"BackupSnapshot
Failed due to Task failed! Task record: uuid:
88d03ed7-d996-aad2-974d-6ab74c3656cc\n nameLabel: Async.VDI.copy\n
nameDescription: \n allowedOperations: []\n currentOperations: {}\n
created: Mon Dec 08 08:46:43 CET 2014\n finished: Mon Dec 08
08:46:57 CET 2014\n status: failure\n residentOn:
com.xensource.xenapi.Host@c612622b\n progress: 1.0\n
type: <none/>\n result: \n errorInfo: [INTERNAL_ERROR,
Failure(\"sparse_dd\")]\n otherConfig: {}\n subtaskOf:
com.xensource.xenapi.Task@aaf13f6f\n subtasks: []\n","wait":0}}] }
2014-12-08 08:47:02,694 DEBUG [c.c.a.t.Request] (Job-Executor-20:ctx-f5bed97c
ctx-992184e6) Seq 2-902104676: Received: { Ans: , MgmtId: 95545481387, via: 2,
Ver: v1, Flags: 10, { CopyCmdAnswer } }
2014-12-08 08:47:02,724 DEBUG [c.c.s.s.SnapshotManagerImpl]
(Job-Executor-20:ctx-f5bed97c ctx-992184e6) Failed to create snapshot
com.cloud.utils.exception.CloudRuntimeException: BackupSnapshot Failed due to
Task failed! Task record: uuid:
88d03ed7-d996-aad2-974d-6ab74c3656cc
nameLabel: Async.VDI.copy
nameDescription:
allowedOperations: []
currentOperations: {}
created: Mon Dec 08 08:46:43 CET 2014
finished: Mon Dec 08 08:46:57 CET 2014
status: failure
residentOn: com.xensource.xenapi.Host@c612622b
progress: 1.0
type: <none/>
result:
errorInfo: [INTERNAL_ERROR, Failure("sparse_dd")]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
at
org.apache.cloudstack.storage.snapshot.SnapshotServiceImpl.backupSnapshot(SnapshotServiceImpl.java:292)
at
org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.backupSnapshot(XenserverSnapshotStrategy.java:142)
at
org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.takeSnapshot(XenserverSnapshotStrategy.java:301)
at
com.cloud.storage.snapshot.SnapshotManagerImpl.takeSnapshot(SnapshotManagerImpl.java:945)
at sun.reflect.GeneratedMethodAccessor357.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:622)
at
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at com.sun.proxy.$Proxy160.takeSnapshot(Unknown Source)
at
org.apache.cloudstack.storage.volume.VolumeServiceImpl.takeSnapshot(VolumeServiceImpl.java:1381)
at
com.cloud.storage.VolumeApiServiceImpl.orchestrateTakeVolumeSnapshot(VolumeApiServiceImpl.java:1783)
at
com.cloud.storage.VolumeApiServiceImpl.takeSnapshot(VolumeApiServiceImpl.java:1724)
at sun.reflect.GeneratedMethodAccessor356.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:622)
at
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at com.sun.proxy.$Proxy196.takeSnapshot(Unknown Source)
at
org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd.execute(CreateSnapshotCmd.java:181)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:161)
at
com.cloud.api.ApiAsyncJobDispatcher.runJobInContext(ApiAsyncJobDispatcher.java:109)
at
com.cloud.api.ApiAsyncJobDispatcher$1.run(ApiAsyncJobDispatcher.java:66)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:63)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:509)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:701)
2014-12-08 08:47:02,739 DEBUG [o.a.c.s.v.VolumeServiceImpl]
(Job-Executor-20:ctx-f5bed97c ctx-992184e6) Take snapshot: 372 failed
com.cloud.utils.exception.CloudRuntimeException: Failed to create snapshot
at
com.cloud.storage.snapshot.SnapshotManagerImpl.takeSnapshot(SnapshotManagerImpl.java:963)
at sun.reflect.GeneratedMethodAccessor357.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:622)
at
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at com.sun.proxy.$Proxy160.takeSnapshot(Unknown Source)
at
org.apache.cloudstack.storage.volume.VolumeServiceImpl.takeSnapshot(VolumeServiceImpl.java:1381)
at
com.cloud.storage.VolumeApiServiceImpl.orchestrateTakeVolumeSnapshot(VolumeApiServiceImpl.java:1783)
at
com.cloud.storage.VolumeApiServiceImpl.takeSnapshot(VolumeApiServiceImpl.java:1724)
at sun.reflect.GeneratedMethodAccessor356.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:622)
at
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at com.sun.proxy.$Proxy196.takeSnapshot(Unknown Source)
at
org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd.execute(CreateSnapshotCmd.java:181)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:161)
at
com.cloud.api.ApiAsyncJobDispatcher.runJobInContext(ApiAsyncJobDispatcher.java:109)
at
com.cloud.api.ApiAsyncJobDispatcher$1.run(ApiAsyncJobDispatcher.java:66)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:63)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:509)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:701)
Caused by: com.cloud.utils.exception.CloudRuntimeException: BackupSnapshot
Failed due to Task failed! Task record: uuid:
88d03ed7-d996-aad2-974d-6ab74c3656cc
nameLabel: Async.VDI.copy
nameDescription:
allowedOperations: []
currentOperations: {}
created: Mon Dec 08 08:46:43 CET 2014
finished: Mon Dec 08 08:46:57 CET 2014
status: failure
residentOn: com.xensource.xenapi.Host@c612622b
progress: 1.0
type: <none/>
result:
errorInfo: [INTERNAL_ERROR, Failure("sparse_dd")]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
at
org.apache.cloudstack.storage.snapshot.SnapshotServiceImpl.backupSnapshot(SnapshotServiceImpl.java:292)
at
org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.backupSnapshot(XenserverSnapshotStrategy.java:142)
at
org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.takeSnapshot(XenserverSnapshotStrategy.java:301)
at
com.cloud.storage.snapshot.SnapshotManagerImpl.takeSnapshot(SnapshotManagerImpl.java:945)
... 43 more
2014-12-08 08:47:02,741 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Job-Executor-20:ctx-f5bed97c) Complete async job-2834, jobStatus: FAILED,
resultCode: 530, result:
org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":530,"errortext":"Failed
to create snapshot due to an internal error creating snapshot for volume 372"}
2014-12-08 08:47:02,746 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Job-Executor-20:ctx-f5bed97c) Done executing
org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd for job-2834
2014-12-08 08:47:02,750 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Job-Executor-20:ctx-f5bed97c) Remove job-2834 from job monitoring
Here is the log from X2 hypervisor, where VM resides:
Dec 8 08:46:36 x2 tapdisk[11369]: gaps written/skipped: 925/0
Dec 8 08:46:36 x2 tapdisk[11369]:
/dev/VG_XenStorage-12b1f1c1-b6fc-9177-4f6c-a8cfb054867b/VHD-b6aafe6d-0091-4131-801d-79bf6352c61a:
b: 51200, a: 6839, f: 981, n: 28075704
Dec 8 08:46:36 x2 tapdisk[11369]: closed image
/dev/VG_XenStorage-12b1f1c1-b6fc-9177-4f6c-a8cfb054867b/VHD-b6aafe6d-0091-4131-801d-79bf6352c61a
(0 users, state: 0x00000000, type: 4)
Dec 8 08:46:36 x2 tapdisk[11369]: gaps written/skipped: 0/0
Dec 8 08:46:36 x2 tapdisk[11369]:
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--c1464f17--a4a7--4412--937c--6461c2b40d69:
b: 51200, a: 7758, f: 1809, n: 0
Dec 8 08:46:36 x2 tapdisk[11369]: closed image
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--c1464f17--a4a7--4412--937c--6461c2b40d69
(0 users, state: 0x00000002, type: 4)
Dec 8 08:46:36 x2 tapdisk[11369]: gaps written/skipped: 0/0
Dec 8 08:46:36 x2 tapdisk[11369]:
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--f14b82ae--456b--4d26--b9d5--4df0d4d054bc:
b: 30720, a: 27801, f: 26067, n: 0
Dec 8 08:46:36 x2 tapdisk[11369]: closed image
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--f14b82ae--456b--4d26--b9d5--4df0d4d054bc
(0 users, state: 0x00000002, type: 4)
Dec 8 08:46:36 x2 tapdisk[11369]: gaps written/skipped: 0/0
Dec 8 08:46:36 x2 tapdisk[11369]:
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--a5b5f6a8--1f87--437c--837a--536126654e8e:
b: 30720, a: 5845, f: 4996, n: 0
Dec 8 08:46:36 x2 tapdisk[11369]: closed image
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--a5b5f6a8--1f87--437c--837a--536126654e8e
(0 users, state: 0x00000002, type: 4)
Dec 8 08:46:37 x2 multipathd: dm-49: remove map (uevent)
Dec 8 08:46:37 x2 multipathd: dm-49: devmap not registered, can't remove
Dec 8 08:46:37 x2 multipathd: dm-49: remove map (uevent)
Dec 8 08:46:37 x2 multipathd: dm-49: devmap not registered, can't remove
Dec 8 08:46:38 x2 multipathd: dm-4: add map (uevent)
Dec 8 08:46:38 x2 multipathd: dm-4: add map (uevent)
Dec 8 08:46:38 x2 multipathd: dm-49: add map (uevent)
Dec 8 08:46:38 x2 multipathd: dm-49: add map (uevent)
Dec 8 08:46:38 x2 tapdisk[11369]:
/dev/VG_XenStorage-12b1f1c1-b6fc-9177-4f6c-a8cfb054867b/VHD-b6aafe6d-0091-4131-801d-79bf6352c61a
version: tap 0x00010003, b: 51200, a: 0, f: 0, n: 8455
Dec 8 08:46:38 x2 tapdisk[11369]: opened image
/dev/VG_XenStorage-12b1f1c1-b6fc-9177-4f6c-a8cfb054867b/VHD-b6aafe6d-0091-4131-801d-79bf6352c61a
(1 users, state: 0x00000001, type: 4, rw)
Dec 8 08:46:38 x2 tapdisk[11369]:
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--2169bd3d--77ef--4f09--b7c3--5e20878e2959
version: tap 0x00010003, b: 51200, a: 6839, f: 981, n: 0
Dec 8 08:46:38 x2 tapdisk[11369]: opened image
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--2169bd3d--77ef--4f09--b7c3--5e20878e2959
(1 users, state: 0x00000003, type: 4, ro)
Dec 8 08:46:38 x2 tapdisk[11369]:
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--c1464f17--a4a7--4412--937c--6461c2b40d69
version: tap 0x00010003, b: 51200, a: 7758, f: 1809, n: 0
Dec 8 08:46:38 x2 tapdisk[11369]: opened image
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--c1464f17--a4a7--4412--937c--6461c2b40d69
(1 users, state: 0x00000003, type: 4, ro)
Dec 8 08:46:38 x2 tapdisk[11369]:
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--f14b82ae--456b--4d26--b9d5--4df0d4d054bc
version: tap 0x00010003, b: 30720, a: 27801, f: 26067, n: 0
Dec 8 08:46:38 x2 tapdisk[11369]: opened image
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--f14b82ae--456b--4d26--b9d5--4df0d4d054bc
(1 users, state: 0x00000003, type: 4, ro)
Dec 8 08:46:38 x2 tapdisk[11369]:
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--a5b5f6a8--1f87--437c--837a--536126654e8e
version: tap 0x00010003, b: 30720, a: 5845, f: 4996, n: 0
Dec 8 08:46:38 x2 tapdisk[11369]: opened image
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--a5b5f6a8--1f87--437c--837a--536126654e8e
(1 users, state: 0x00000003, type: 4, ro)
Dec 8 08:46:38 x2 tapdisk[11369]: VBD CHAIN:
Dec 8 08:46:38 x2 tapdisk[11369]:
/dev/VG_XenStorage-12b1f1c1-b6fc-9177-4f6c-a8cfb054867b/VHD-b6aafe6d-0091-4131-801d-79bf6352c61a:
type:vhd(4) storage:lvm(3)
Dec 8 08:46:38 x2 tapdisk[11369]:
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--2169bd3d--77ef--4f09--b7c3--5e20878e2959:
type:vhd(4) storage:lvm(3)
Dec 8 08:46:38 x2 tapdisk[11369]:
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--c1464f17--a4a7--4412--937c--6461c2b40d69:
type:vhd(4) storage:lvm(3)
Dec 8 08:46:38 x2 tapdisk[11369]:
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--f14b82ae--456b--4d26--b9d5--4df0d4d054bc:
type:vhd(4) storage:lvm(3)
Dec 8 08:46:38 x2 tapdisk[11369]:
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--a5b5f6a8--1f87--437c--837a--536126654e8e:
type:vhd(4) storage:lvm(3)
Dec 8 08:46:47 x2 multipathd: dm-59: add map (uevent)
Dec 8 08:46:47 x2 tapdisk[2030]: tapdisk-control: init, 10 x 4k buffers
Dec 8 08:46:47 x2 tapdisk[2030]: I/O queue driver: lio
Dec 8 08:46:47 x2 tapdisk[2030]: tapdisk-log: started, level 0
Dec 8 08:46:47 x2 tapdisk[2030]:
/dev/VG_XenStorage-12b1f1c1-b6fc-9177-4f6c-a8cfb054867b/VHD-9d829e99-bc1d-41ee-a652-02df9125c1c9
version: tap 0x00010003, b: 51200, a: 0, f: 0, n: 0
Dec 8 08:46:47 x2 tapdisk[2030]: opened image
/dev/VG_XenStorage-12b1f1c1-b6fc-9177-4f6c-a8cfb054867b/VHD-9d829e99-bc1d-41ee-a652-02df9125c1c9
(1 users, state: 0x00000003, type: 4, ro)
Dec 8 08:46:47 x2 tapdisk[2030]:
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--2169bd3d--77ef--4f09--b7c3--5e20878e2959
version: tap 0x00010003, b: 51200, a: 6839, f: 981, n: 0
Dec 8 08:46:47 x2 tapdisk[2030]: opened image
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--2169bd3d--77ef--4f09--b7c3--5e20878e2959
(1 users, state: 0x00000003, type: 4, ro)
Dec 8 08:46:47 x2 tapdisk[2030]:
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--c1464f17--a4a7--4412--937c--6461c2b40d69
version: tap 0x00010003, b: 51200, a: 7758, f: 1809, n: 0
Dec 8 08:46:47 x2 tapdisk[2030]: opened image
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--c1464f17--a4a7--4412--937c--6461c2b40d69
(1 users, state: 0x00000003, type: 4, ro)
Dec 8 08:46:47 x2 tapdisk[2030]:
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--f14b82ae--456b--4d26--b9d5--4df0d4d054bc
version: tap 0x00010003, b: 30720, a: 27801, f: 26067, n: 0
Dec 8 08:46:47 x2 tapdisk[2030]: opened image
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--f14b82ae--456b--4d26--b9d5--4df0d4d054bc
(1 users, state: 0x00000003, type: 4, ro)
Dec 8 08:46:47 x2 tapdisk[2030]:
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--a5b5f6a8--1f87--437c--837a--536126654e8e
version: tap 0x00010003, b: 30720, a: 5845, f: 4996, n: 0
Dec 8 08:46:47 x2 tapdisk[2030]: opened image
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--a5b5f6a8--1f87--437c--837a--536126654e8e
(1 users, state: 0x00000003, type: 4, ro)
Dec 8 08:46:47 x2 tapdisk[2030]: VBD CHAIN:
Dec 8 08:46:47 x2 tapdisk[2030]:
/dev/VG_XenStorage-12b1f1c1-b6fc-9177-4f6c-a8cfb054867b/VHD-9d829e99-bc1d-41ee-a652-02df9125c1c9:
type:vhd(4) storage:lvm(3)
Dec 8 08:46:47 x2 tapdisk[2030]:
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--2169bd3d--77ef--4f09--b7c3--5e20878e2959:
type:vhd(4) storage:lvm(3)
Dec 8 08:46:47 x2 tapdisk[2030]:
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--c1464f17--a4a7--4412--937c--6461c2b40d69:
type:vhd(4) storage:lvm(3)
Dec 8 08:46:47 x2 tapdisk[2030]:
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--f14b82ae--456b--4d26--b9d5--4df0d4d054bc:
type:vhd(4) storage:lvm(3)
Dec 8 08:46:47 x2 tapdisk[2030]:
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--a5b5f6a8--1f87--437c--837a--536126654e8e:
type:vhd(4) storage:lvm(3)
Dec 8 08:46:47 x2 tapdisk[2030]: bdev: capacity=209715200 sector_size=512/512
flags=0
Dec 8 08:46:47 x2 multipathd: tdi: add path (uevent)
Dec 8 08:46:47 x2 multipathd: tdi: failed to store path info
Dec 8 08:46:47 x2 multipathd: uevent trigger error
Dec 8 08:46:47 x2 kernel: block tdi: sector-size: 512/512 capacity: 209715200
Dec 8 08:46:47 x2 kernel: blkback: event-channel 79
Dec 8 08:46:47 x2 kernel: blkback: ring-ref 251
Dec 8 08:46:47 x2 kernel: blkback: protocol 1 (x86_32-abi)
Dec 8 08:46:47 x2 kernel: blkfront: xvda: barriers enabled
Dec 8 08:46:47 x2 multipathd: xvda: add path (uevent)
Dec 8 08:46:47 x2 multipathd: xvda: failed to store path info
Dec 8 08:46:47 x2 multipathd: uevent trigger error
Dec 8 08:46:47 x2 kernel: xvda: xvda1 xvda2
Dec 8 08:46:48 x2 vbd.uevent[add](backend/vbd/0/51712): wrote
/xapi/0/hotplug/vbd/51712/hotplug = 'online'
Dec 8 08:46:48 x2 vbd.uevent[add](backend/vbd/0/51712): wrote
backend/vbd/0/51712/hotplug-status = 'connected'
Dec 8 08:46:50 x2 tapdisk[2373]: tapdisk-control: init, 10 x 4k buffers
Dec 8 08:46:50 x2 tapdisk[2373]: I/O queue driver: lio
Dec 8 08:46:50 x2 tapdisk[2373]: tapdisk-log: started, level 0
Dec 8 08:46:50 x2 tapdisk[2373]:
/var/run/sr-mount/de7d0b77-3559-8024-1dae-ac1ccdd088e0/9daa65c6-3a9c-4d25-8a76-23e5b608945d.vhd
version: tap 0x00010003, b: 51200, a: 0, f: 0, n: 417
Dec 8 08:46:50 x2 tapdisk[2373]: opened image
/var/run/sr-mount/de7d0b77-3559-8024-1dae-ac1ccdd088e0/9daa65c6-3a9c-4d25-8a76-23e5b608945d.vhd
(1 users, state: 0x00000001, type: 4, rw)
Dec 8 08:46:50 x2 tapdisk[2373]: VBD CHAIN:
Dec 8 08:46:50 x2 tapdisk[2373]:
/var/run/sr-mount/de7d0b77-3559-8024-1dae-ac1ccdd088e0/9daa65c6-3a9c-4d25-8a76-23e5b608945d.vhd:
type:vhd(4) storage:nfs(1)
Dec 8 08:46:50 x2 tapdisk[2373]: bdev: capacity=209715200 sector_size=512/512
flags=0
Dec 8 08:46:50 x2 multipathd: tdo: add path (uevent)
Dec 8 08:46:50 x2 multipathd: tdo: failed to store path info
Dec 8 08:46:50 x2 multipathd: uevent trigger error
Dec 8 08:46:50 x2 kernel: block tdo: sector-size: 512/512 capacity: 209715200
Dec 8 08:46:50 x2 kernel: blkback: event-channel 81
Dec 8 08:46:50 x2 kernel: blkback: ring-ref 130
Dec 8 08:46:50 x2 kernel: blkback: protocol 1 (x86_32-abi)
Dec 8 08:46:50 x2 multipathd: xvdb: add path (uevent)
Dec 8 08:46:50 x2 multipathd: xvdb: failed to store path info
Dec 8 08:46:50 x2 multipathd: uevent trigger error
Dec 8 08:46:50 x2 kernel: blkfront: xvdb: barriers enabled
Dec 8 08:46:50 x2 kernel: xvdb: unknown partition table
Dec 8 08:46:50 x2 vbd.uevent[add](backend/vbd/0/51728): wrote
/xapi/0/hotplug/vbd/51728/hotplug = 'online'
Dec 8 08:46:50 x2 vbd.uevent[add](backend/vbd/0/51728): wrote
backend/vbd/0/51728/hotplug-status = 'connected'
Dec 8 08:46:51 x2 sparse_dd: [ info|x2.c.some.domain|0||sparse_dd] streaming
from raw /dev/xvda using BAT from
/dev/VG_XenStorage-12b1f1c1-b6fc-9177-4f6c-a8cfb054867b/VHD-9d829e99-bc1d-41ee-a652-02df9125c1c9
(relative to None) to raw file:///dev/xvdb
Dec 8 08:46:51 x2 fe: 2421 (/opt/xensource/libexec/sparse_dd -machine -src
/dev/xvda -dest /dev/xvdb -siz...) exitted with code 2
Dec 8 08:46:51 x2 xapi: [error|x2.c.some.domain|1285367
inet-RPC|Async.VDI.copy R:dd071aa37b78|xapi] sparse_dd exit: 2
Dec 8 08:46:51 x2 xapi: [error|x2.c.some.domain|1285367
inet-RPC|Async.VDI.copy R:dd071aa37b78|xapi] Failure from sparse_dd: Fatal
error: exception Invalid_argument("index out of bounds")
Dec 8 08:46:51 x2 multipathd: xvdb: remove path (uevent)
Dec 8 08:46:51 x2 multipathd: xvdb: spurious uevent, path not in pathvec
Dec 8 08:46:51 x2 multipathd: uevent trigger error
Dec 8 08:46:51 x2 vbd.uevent[remove](backend/vbd/0/51728): removed
/xapi/0/hotplug/vbd/51728/hotplug
Dec 8 08:46:51 x2 vbd.uevent[remove](backend/vbd/0/51728): removed
backend/vbd/0/51728/hotplug-status
Dec 8 08:46:52 x2 multipathd: tdo: remove path (uevent)
Dec 8 08:46:52 x2 multipathd: tdo: spurious uevent, path not in pathvec
Dec 8 08:46:52 x2 multipathd: uevent trigger error
Dec 8 08:46:52 x2 tapdisk[2373]: gaps written/skipped: 0/0
Dec 8 08:46:52 x2 tapdisk[2373]:
/var/run/sr-mount/de7d0b77-3559-8024-1dae-ac1ccdd088e0/9daa65c6-3a9c-4d25-8a76-23e5b608945d.vhd:
b: 51200, a: 0, f: 0, n: 417
Dec 8 08:46:52 x2 tapdisk[2373]: closed image
/var/run/sr-mount/de7d0b77-3559-8024-1dae-ac1ccdd088e0/9daa65c6-3a9c-4d25-8a76-23e5b608945d.vhd
(0 users, state: 0x00000000, type: 4)
Dec 8 08:46:52 x2 tapdisk[2373]: tapdisk-log: closing after 0 errors
Dec 8 08:46:52 x2 tapdisk[2373]: tapdisk-syslog: 11 messages, 902 bytes,
xmits: 12, failed: 0, dropped: 0
Dec 8 08:46:52 x2 tapdisk[2373]: tapdisk-control: draining 1 connections
Dec 8 08:46:52 x2 tapdisk[2373]: tapdisk-control: done
Dec 8 08:46:53 x2 multipathd: xvda: remove path (uevent)
Dec 8 08:46:53 x2 multipathd: xvda: spurious uevent, path not in pathvec
Dec 8 08:46:53 x2 multipathd: uevent trigger error
Dec 8 08:46:53 x2 vbd.uevent[remove](backend/vbd/0/51712): removed
/xapi/0/hotplug/vbd/51712/hotplug
Dec 8 08:46:53 x2 vbd.uevent[remove](backend/vbd/0/51712): removed
backend/vbd/0/51712/hotplug-status
Dec 8 08:46:54 x2 tapdisk[2030]: gaps written/skipped: 0/0
Dec 8 08:46:54 x2 multipathd: tdi: remove path (uevent)
Dec 8 08:46:54 x2 multipathd: tdi: spurious uevent, path not in pathvec
Dec 8 08:46:54 x2 multipathd: uevent trigger error
Dec 8 08:46:54 x2 tapdisk[2030]:
/dev/VG_XenStorage-12b1f1c1-b6fc-9177-4f6c-a8cfb054867b/VHD-9d829e99-bc1d-41ee-a652-02df9125c1c9:
b: 51200, a: 0, f: 0, n: 0
Dec 8 08:46:54 x2 tapdisk[2030]: closed image
/dev/VG_XenStorage-12b1f1c1-b6fc-9177-4f6c-a8cfb054867b/VHD-9d829e99-bc1d-41ee-a652-02df9125c1c9
(0 users, state: 0x00000002, type: 4)
Dec 8 08:46:54 x2 tapdisk[2030]: gaps written/skipped: 0/0
Dec 8 08:46:54 x2 tapdisk[2030]:
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--2169bd3d--77ef--4f09--b7c3--5e20878e2959:
b: 51200, a: 6839, f: 981, n: 0
Dec 8 08:46:54 x2 tapdisk[2030]: closed image
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--2169bd3d--77ef--4f09--b7c3--5e20878e2959
(0 users, state: 0x00000002, type: 4)
Dec 8 08:46:54 x2 tapdisk[2030]: gaps written/skipped: 0/0
Dec 8 08:46:54 x2 tapdisk[2030]:
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--c1464f17--a4a7--4412--937c--6461c2b40d69:
b: 51200, a: 7758, f: 1809, n: 0
Dec 8 08:46:54 x2 tapdisk[2030]: closed image
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--c1464f17--a4a7--4412--937c--6461c2b40d69
(0 users, state: 0x00000002, type: 4)
Dec 8 08:46:54 x2 tapdisk[2030]: gaps written/skipped: 0/0
Dec 8 08:46:54 x2 tapdisk[2030]:
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--f14b82ae--456b--4d26--b9d5--4df0d4d054bc:
b: 30720, a: 27801, f: 26067, n: 0
Dec 8 08:46:54 x2 tapdisk[2030]: closed image
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--f14b82ae--456b--4d26--b9d5--4df0d4d054bc
(0 users, state: 0x00000002, type: 4)
Dec 8 08:46:54 x2 tapdisk[2030]: gaps written/skipped: 0/0
Dec 8 08:46:54 x2 tapdisk[2030]:
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--a5b5f6a8--1f87--437c--837a--536126654e8e:
b: 30720, a: 5845, f: 4996, n: 0
Dec 8 08:46:54 x2 tapdisk[2030]: closed image
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--a5b5f6a8--1f87--437c--837a--536126654e8e
(0 users, state: 0x00000002, type: 4)
Dec 8 08:46:54 x2 tapdisk[2030]: tapdisk-log: closing after 0 errors
Dec 8 08:46:54 x2 tapdisk[2030]: tapdisk-syslog: 15 messages, 1603 bytes,
xmits: 17, failed: 0, dropped: 0
Dec 8 08:46:54 x2 tapdisk[2030]: tapdisk-control: draining 1 connections
Dec 8 08:46:54 x2 tapdisk[2030]: tapdisk-control: done
Dec 8 08:46:55 x2 multipathd: dm-59: remove map (uevent)
Dec 8 08:46:55 x2 multipathd: dm-59: devmap not registered, can't remove
Dec 8 08:46:55 x2 multipathd: dm-59: remove map (uevent)
Dec 8 08:46:55 x2 multipathd: dm-59: devmap not registered, can't remove
Dec 8 08:47:07 x2 tapdisk[11369]: gaps written/skipped: 0/0
Dec 8 08:47:07 x2 tapdisk[11369]:
/dev/VG_XenStorage-12b1f1c1-b6fc-9177-4f6c-a8cfb054867b/VHD-b6aafe6d-0091-4131-801d-79bf6352c61a:
b: 51200, a: 54, f: 0, n: 230064
Dec 8 08:47:07 x2 tapdisk[11369]: closed image
/dev/VG_XenStorage-12b1f1c1-b6fc-9177-4f6c-a8cfb054867b/VHD-b6aafe6d-0091-4131-801d-79bf6352c61a
(0 users, state: 0x00000000, type: 4)
Dec 8 08:47:07 x2 tapdisk[11369]: gaps written/skipped: 0/0
Dec 8 08:47:07 x2 tapdisk[11369]:
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--2169bd3d--77ef--4f09--b7c3--5e20878e2959:
b: 51200, a: 6839, f: 981, n: 0
Dec 8 08:47:07 x2 tapdisk[11369]: closed image
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--2169bd3d--77ef--4f09--b7c3--5e20878e2959
(0 users, state: 0x00000002, type: 4)
Dec 8 08:47:07 x2 tapdisk[11369]: gaps written/skipped: 0/0
Dec 8 08:47:07 x2 tapdisk[11369]:
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--c1464f17--a4a7--4412--937c--6461c2b40d69:
b: 51200, a: 7758, f: 1809, n: 0
Dec 8 08:47:07 x2 tapdisk[11369]: closed image
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--c1464f17--a4a7--4412--937c--6461c2b40d69
(0 users, state: 0x00000002, type: 4)
Dec 8 08:47:07 x2 tapdisk[11369]: gaps written/skipped: 0/0
Dec 8 08:47:07 x2 tapdisk[11369]:
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--f14b82ae--456b--4d26--b9d5--4df0d4d054bc:
b: 30720, a: 27801, f: 26067, n: 0
Dec 8 08:47:07 x2 tapdisk[11369]: closed image
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--f14b82ae--456b--4d26--b9d5--4df0d4d054bc
(0 users, state: 0x00000002, type: 4)
Dec 8 08:47:07 x2 tapdisk[11369]: gaps written/skipped: 0/0
Dec 8 08:47:07 x2 tapdisk[11369]:
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--a5b5f6a8--1f87--437c--837a--536126654e8e:
b: 30720, a: 5845, f: 4996, n: 0
Dec 8 08:47:07 x2 tapdisk[11369]: closed image
/dev/mapper/VG_XenStorage--12b1f1c1--b6fc--9177--4f6c--a8cfb054867b-VHD--a5b5f6a8--1f87--437c--837a--536126654e8e
(0 users, state: 0x00000002, type: 4)
Dec 8 08:47:09 x2 multipathd: dm-4: remove map (uevent)
Dec 8 08:47:09 x2 multipathd: dm-4: devmap not registered, can't remove
Dec 8 08:47:09 x2 multipathd: dm-4: remove map (uevent)
Dec 8 08:47:09 x2 multipathd: dm-4: devmap not registered, can't remove
Dec 8 08:47:09 x2 multipathd: dm-4: add map (uevent)
Dec 8 08:47:09 x2 multipathd: dm-4: add map (uevent)
Dec 8 08:47:09 x2 multipathd: dm-59: add map (uevent)
Dec 8 08:47:09 x2 multipathd: dm-59: add map (uevent)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
