top-secrett commented on issue #9734:
URL: https://github.com/apache/cloudstack/issues/9734#issuecomment-2377168314
Example when first volume snapshot stores on the second secstore and I
create the next volume snapshot
~~~
2024-09-26 16:58:32,135 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-58:ctx-c9d433a7 job-72454) (logid:f365dd96) Executing
AsyncJobVO: {id:72454, userId: 298, accountId: 225, instanceType: Snapshot,
instanceId: 867, cmd:
org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd, cmdInfo:
{"asyncBackup":"false","quiescevm":"false","response":"json","ctxUserId":"298","volumeId":"da420806-0ff3-43b8-9ee0-4aa2adfd9b99","httpmethod":"GET","ctxStartEventId":"1811467","id":"867","ctxDetails":"{\"interface
com.cloud.storage.Volume\":\"da420806-0ff3-43b8-9ee0-4aa2adfd9b99\",\"interface
com.cloud.storage.Snapshot\":\"c62b0aa0-5a1c-475c-b7f6-6c9fab17e325\"}","ctxAccountId":"225","uuid":"c62b0aa0-5a1c-475c-b7f6-6c9fab17e325","cmdEventType":"SNAPSHOT.CREATE"},
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result:
null, initMsid: 108597816265214, completeMsid: null, lastUpdated: null,
lastPolled: null, created: null, removed: null}
2024-09-26 16:58:32,147 DEBUG [c.c.u.AccountManagerImpl]
(API-Job-Executor-58:ctx-c9d433a7 job-72454 ctx-829ed64e) (logid:f365dd96)
Access to Account
[{"accountName":"karasev-cluster5","id":225,"uuid":"fd32d818-5d3b-4d3c-9084-34704beefaf5"}]
granted to Account
[{"accountName":"karasev-cluster5","id":225,"uuid":"fd32d818-5d3b-4d3c-9084-34704beefaf5"}]
by DomainChecker
2024-09-26 16:58:32,167 DEBUG [c.c.a.ApiServlet]
(qtp501107890-4830812:ctx-4bacce10) (logid:11a55db0) ===START=== 172.16.11.61
-- GET
jobId=f365dd96-c018-454a-874c-37dce9396d96&command=queryAsyncJobResult&response=json
2024-09-26 16:58:32,183 DEBUG [c.c.u.AccountManagerImpl]
(API-Job-Executor-58:ctx-c9d433a7 job-72454 ctx-829ed64e) (logid:f365dd96)
Access to org.apache.cloudstack.storage.volume.VolumeObject@f1e18b7 granted to
Account
[{"accountName":"karasev-cluster5","id":225,"uuid":"fd32d818-5d3b-4d3c-9084-34704beefaf5"}]
by DomainChecker
2024-09-26 16:58:32,190 DEBUG [c.c.a.ApiServlet]
(qtp501107890-4830812:ctx-4bacce10 ctx-aa1d9072) (logid:11a55db0) ===END===
172.16.11.61 -- GET
jobId=f365dd96-c018-454a-874c-37dce9396d96&command=queryAsyncJobResult&response=json
2024-09-26 16:58:32,192 DEBUG [c.c.u.AccountManagerImpl]
(API-Job-Executor-58:ctx-c9d433a7 job-72454 ctx-829ed64e) (logid:f365dd96)
Access to VM instance
{"id":5468,"instanceName":"i-225-5468-VM","type":"User","uuid":"d1e26871-d01f-44cc-ac61-f19f46ed741a"}
granted to Account
[{"accountName":"karasev-cluster5","id":225,"uuid":"fd32d818-5d3b-4d3c-9084-34704beefaf5"}]
by DomainChecker
2024-09-26 16:58:32,206 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-58:ctx-c9d433a7 job-72454 ctx-829ed64e) (logid:f365dd96) Sync
job-72455 execution on object VmWorkJobQueue.5468
2024-09-26 16:58:32,575 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-148:ctx-ef42787d job-72454/job-72455) (logid:f365dd96)
Executing AsyncJobVO: {id:72455, userId: 298, accountId: 225, instanceType:
null, instanceId: null, cmd: com.cloud.vm.VmWorkTakeVolumeSnapshot, cmdInfo:
rO0ABXNyACVjb20uY2xvdWQudm0uVm1Xb3JrVGFrZVZvbHVtZVNuYXBzaG90BL5gG4Li1c8CAAdaAAthc3luY0JhY2t1cFoACXF1aWVzY2VWbUwADGxvY2F0aW9uVHlwZXQAKUxjb20vY2xvdWQvc3RvcmFnZS9TbmFwc2hvdCRMb2NhdGlvblR5cGU7TAAIcG9saWN5SWR0ABBMamF2YS9sYW5nL0xvbmc7TAAKc25hcHNob3RJZHEAfgACTAAIdm9sdW1lSWRxAH4AAkwAB3pvbmVJZHN0ABBMamF2YS91dGlsL0xpc3Q7eHIAE2NvbS5jbG91ZC52bS5WbVdvcmufmbZW8CVnawIABEoACWFjY291bnRJZEoABnVzZXJJZEoABHZtSWRMAAtoYW5kbGVyTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO3hwAAAAAAAAAOEAAAAAAAABKgAAAAAAABVcdAAUVm9sdW1lQXBpU2VydmljZUltcGwAAHBzcgAOamF2YS5sYW5nLkxvbmc7i-SQzI8j3wIAAUoABXZhbHVleHIAEGphdmEubGFuZy5OdW1iZXKGrJUdC5TgiwIAAHhwAAAAAAAAAABzcQB-AAgAAAAAAAADY3NxAH4ACAAAAAAAABgMcA,
cmdVersion: 0, status: IN_PROGRESS, processStatu
s: 0, resultCode: 0, result: null, initMsid: 108597816265214, completeMsid:
null, lastUpdated: null, lastPolled: null, created: Thu Sep 26 16:58:32 MSK
2024, removed: null}
2024-09-26 16:58:32,575 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-148:ctx-ef42787d job-72454/job-72455) (logid:f365dd96) Run
VM work job: com.cloud.vm.VmWorkTakeVolumeSnapshot for VM 5468, job origin:
72454
2024-09-26 16:58:32,576 DEBUG [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-148:ctx-ef42787d job-72454/job-72455 ctx-d1890501)
(logid:f365dd96) Execute VM work job:
com.cloud.vm.VmWorkTakeVolumeSnapshot{"volumeId":6156,"policyId":0,"snapshotId":867,"quiesceVm":false,"asyncBackup":false,"userId":298,"accountId":225,"vmId":5468,"handlerName":"VolumeApiServiceImpl"}
2024-09-26 16:58:32,588 DEBUG [o.a.c.s.s.StorPoolSnapshotStrategy]
(Work-Job-Executor-148:ctx-ef42787d job-72454/job-72455 ctx-d1890501)
(logid:f365dd96) StorpoolSnapshotStrategy.canHandle:
snapshot=backup-test-karasev-3_ROOT-5468_20240926135832,
uuid=c62b0aa0-5a1c-475c-b7f6-6c9fab17e325, op=TAKE
2024-09-26 16:58:32,622 DEBUG
[o.a.c.s.d.d.CloudStackPrimaryDataStoreDriverImpl]
(Work-Job-Executor-148:ctx-ef42787d job-72454/job-72455 ctx-d1890501)
(logid:f365dd96) Taking snapshot of
org.apache.cloudstack.storage.snapshot.SnapshotObject@18f701c2
2024-09-26 16:58:32,653 DEBUG
[o.a.c.s.d.d.CloudStackPrimaryDataStoreDriverImpl]
(Work-Job-Executor-148:ctx-ef42787d job-72454/job-72455 ctx-d1890501)
(logid:f365dd96) Taking snapshot of
org.apache.cloudstack.storage.snapshot.SnapshotObject@18f701c2 and encryption
required is false
2024-09-26 16:58:32,653 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
(Work-Job-Executor-148:ctx-ef42787d job-72454/job-72455 ctx-d1890501)
(logid:f365dd96) getCommandHostDelegation: class
org.apache.cloudstack.storage.command.CreateObjectCommand
2024-09-26 16:58:32,655 DEBUG [c.c.h.XenServerGuru]
(Work-Job-Executor-148:ctx-ef42787d job-72454/job-72455 ctx-d1890501)
(logid:f365dd96) We are returning the default host to execute commands because
the command is not of Copy type.
2024-09-26 16:58:32,657 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-148:ctx-ef42787d job-72454/job-72455 ctx-d1890501)
(logid:f365dd96) Seq 77353-4517673376206130891: Sending { Cmd , MgmtId:
108597816265214, via: 77353(xcp-sr246-u1314.aisrp.local), Ver: v1, Flags:
100011,
[{"org.apache.cloudstack.storage.command.CreateObjectCommand":{"data":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"volume":{"uuid":"da420806-0ff3-43b8-9ee0-4aa2adfd9b99","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"DM7100_LUN3","name":"cluster5-dm7100_lun3","id":"53","poolType":"PreSetup","host":"localhost","path":"/DM7100_LUN3","port":"0","url":"PreSetup://localhost/DM7100_LUN3/?ROLE=Primary&STOREUUID=DM7100_LUN3","isManaged":"false"}},"name":"ROOT-5468","size":"(20.00
GB)
21474836480","path":"9e2a4d6e-3ab8-46f4-85cf-c7454fc9ea17","volumeId":"6156","vmName":"i-225-5468-VM","accountId":"225","format":"VHD","provisioningType":"THIN","poolId":"53","id":"6
156","deviceId":"0","cacheMode":"NONE","hypervisorType":"XenServer","directDownload":"false","deployAsIs":"false","followRedirects":"true"},"parentSnapshotPath":"70424963-5933-4e73-b185-cfdc402175cd","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"DM7100_LUN3","name":"cluster5-dm7100_lun3","id":"53","poolType":"PreSetup","host":"localhost","path":"/DM7100_LUN3","port":"0","url":"PreSetup://localhost/DM7100_LUN3/?ROLE=Primary&STOREUUID=DM7100_LUN3","isManaged":"false"}},"vmName":"i-225-5468-VM","name":"backup-test-karasev-3_ROOT-5468_20240926135832","hypervisorType":"XenServer","id":"867","quiescevm":"false","parents":["70424963-5933-4e73-b185-cfdc402175cd"],"physicalSize":"0","accountId":"225","followRedirects":"false"}},"wait":"0","bypassHostMaintenance":"false"}}]
}
2024-09-26 16:58:32,659 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-148:ctx-ef42787d job-72454/job-72455 ctx-d1890501)
(logid:f365dd96) Seq 77353-4517673376206130891: Executing: { Cmd , MgmtId:
108597816265214, via: 77353(xcp-sr246-u1314.aisrp.local), Ver: v1, Flags:
100011,
[{"org.apache.cloudstack.storage.command.CreateObjectCommand":{"data":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"volume":{"uuid":"da420806-0ff3-43b8-9ee0-4aa2adfd9b99","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"DM7100_LUN3","name":"cluster5-dm7100_lun3","id":"53","poolType":"PreSetup","host":"localhost","path":"/DM7100_LUN3","port":"0","url":"PreSetup://localhost/DM7100_LUN3/?ROLE=Primary&STOREUUID=DM7100_LUN3","isManaged":"false"}},"name":"ROOT-5468","size":"(20.00
GB)
21474836480","path":"9e2a4d6e-3ab8-46f4-85cf-c7454fc9ea17","volumeId":"6156","vmName":"i-225-5468-VM","accountId":"225","format":"VHD","provisioningType":"THIN","poolId":"53","id"
:"6156","deviceId":"0","cacheMode":"NONE","hypervisorType":"XenServer","directDownload":"false","deployAsIs":"false","followRedirects":"true"},"parentSnapshotPath":"70424963-5933-4e73-b185-cfdc402175cd","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"DM7100_LUN3","name":"cluster5-dm7100_lun3","id":"53","poolType":"PreSetup","host":"localhost","path":"/DM7100_LUN3","port":"0","url":"PreSetup://localhost/DM7100_LUN3/?ROLE=Primary&STOREUUID=DM7100_LUN3","isManaged":"false"}},"vmName":"i-225-5468-VM","name":"backup-test-karasev-3_ROOT-5468_20240926135832","hypervisorType":"XenServer","id":"867","quiescevm":"false","parents":["70424963-5933-4e73-b185-cfdc402175cd"],"physicalSize":"0","accountId":"225","followRedirects":"false"}},"wait":"0","bypassHostMaintenance":"false"}}]
}
2024-09-26 16:58:32,659 DEBUG [c.c.s.r.StorageSubsystemCommandHandlerBase]
(DirectAgent-128:ctx-052ce805) (logid:f365dd96) Executing command
CreateObjectCommand:
[{"data":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"volume":{"uuid":"da420806-0ff3-43b8-9ee0-4aa2adfd9b99","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"DM7100_LUN3","name":"cluster5-dm7100_lun3","id":53,"poolType":"PreSetup","host":"localhost","path":"/DM7100_LUN3","port":0,"url":"PreSetup://localhost/DM7100_LUN3/?ROLE=Primary&STOREUUID=DM7100_LUN3","isManaged":false}},"name":"ROOT-5468","size":21474836480,"path":"9e2a4d6e-3ab8-46f4-85cf-c7454fc9ea17","volumeId":6156,"vmName":"i-225-5468-VM","accountId":225,"format":"VHD","provisioningType":"THIN","poolId":53,"id":6156,"deviceId":0,"cacheMode":"NONE","hypervisorType":"XenServer","directDownload":false,"deployAsIs":false,"followRedirects":true},"parentSnapshotPath":"70424963-5933-4e73-b185-cfdc402175cd","dataSt
ore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"DM7100_LUN3","name":"cluster5-dm7100_lun3","id":53,"poolType":"PreSetup","host":"localhost","path":"/DM7100_LUN3","port":0,"url":"PreSetup://localhost/DM7100_LUN3/?ROLE=Primary&STOREUUID=DM7100_LUN3","isManaged":false}},"vmName":"i-225-5468-VM","name":"backup-test-karasev-3_ROOT-5468_20240926135832","hypervisorType":"XenServer","id":867,"quiescevm":false,"parents":["70424963-5933-4e73-b185-cfdc402175cd"],"physicalSize":0,"accountId":225,"followRedirects":false}},"wait":0,"bypassHostMaintenance":false}].
2024-09-26 16:58:35,268 DEBUG [c.c.a.ApiServlet]
(qtp501107890-4820302:ctx-f711cc5b) (logid:4e44467e) ===START=== 172.16.11.61
-- GET
jobId=f365dd96-c018-454a-874c-37dce9396d96&command=queryAsyncJobResult&response=json
2024-09-26 16:58:35,290 DEBUG [c.c.a.ApiServlet]
(qtp501107890-4820302:ctx-f711cc5b ctx-e8b6659b) (logid:4e44467e) ===END===
172.16.11.61 -- GET
jobId=f365dd96-c018-454a-874c-37dce9396d96&command=queryAsyncJobResult&response=json
2024-09-26 16:58:38,370 DEBUG [c.c.a.ApiServlet]
(qtp501107890-4830812:ctx-fd303fe2) (logid:ef8914d8) ===START=== 172.16.11.61
-- GET
jobId=f365dd96-c018-454a-874c-37dce9396d96&command=queryAsyncJobResult&response=json
2024-09-26 16:58:38,394 DEBUG [c.c.a.ApiServlet]
(qtp501107890-4830812:ctx-fd303fe2 ctx-aec37e2b) (logid:ef8914d8) ===END===
172.16.11.61 -- GET
jobId=f365dd96-c018-454a-874c-37dce9396d96&command=queryAsyncJobResult&response=json
2024-09-26 16:58:39,977 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-128:ctx-052ce805) (logid:f365dd96) Seq 77353-4517673376206130891:
Response Received:
2024-09-26 16:58:39,977 DEBUG [c.c.a.t.Request]
(DirectAgent-128:ctx-052ce805) (logid:f365dd96) Seq 77353-4517673376206130891:
Processing: { Ans: , MgmtId: 108597816265214, via:
77353(xcp-sr246-u1314.aisrp.local), Ver: v1, Flags: 10,
[{"org.apache.cloudstack.storage.command.CreateObjectAnswer":{"data":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"e1971e58-5b6e-4f59-a256-154280cf4d16","id":"0","quiescevm":"false","physicalSize":"0","accountId":"0","followRedirects":"false"}},"result":"true","wait":"0","bypassHostMaintenance":"false"}}]
}
2024-09-26 16:58:39,977 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-148:ctx-ef42787d job-72454/job-72455 ctx-d1890501)
(logid:f365dd96) Seq 77353-4517673376206130891: Received: { Ans: , MgmtId:
108597816265214, via: 77353(xcp-sr246-u1314.aisrp.local), Ver: v1, Flags: 10, {
CreateObjectAnswer } }
2024-09-26 16:58:40,045 DEBUG [o.a.c.s.h.HeuristicRuleHelper]
(Work-Job-Executor-148:ctx-ef42787d job-72454/job-72455 ctx-d1890501)
(logid:f365dd96) No heuristic rules found for zone with ID [7] and heuristic
type [SNAPSHOT]. Returning null.
2024-09-26 16:58:40,049 DEBUG [c.c.s.StatsCollector]
(Work-Job-Executor-148:ctx-ef42787d job-72454/job-72455 ctx-d1890501)
(logid:f365dd96) Verifying image storage [7]. Capacity: total=[785 GB],
used=[243 GB], threshold=[89.99999761581421%].
2024-09-26 16:58:40,050 DEBUG [c.c.s.StatsCollector]
(Work-Job-Executor-148:ctx-ef42787d job-72454/job-72455 ctx-d1890501)
(logid:f365dd96) Verifying image storage [9]. Capacity: total=[785 GB],
used=[70 GB], threshold=[89.99999761581421%].
2024-09-26 16:58:40,051 DEBUG [c.c.s.StatsCollector]
(Work-Job-Executor-148:ctx-ef42787d job-72454/job-72455 ctx-d1890501)
(logid:f365dd96) Verifying image storage [7]. Capacity: total=[785 GB],
used=[243 GB], threshold=[89.99999761581421%].
2024-09-26 16:58:40,101 DEBUG [o.a.c.s.m.AncientDataMotionStrategy]
(Work-Job-Executor-148:ctx-ef42787d job-72454/job-72455 ctx-d1890501)
(logid:f365dd96) copyAsync inspecting src type SNAPSHOT copyAsync inspecting
dest type SNAPSHOT
2024-09-26 16:58:40,224 ERROR [o.a.c.s.e.DefaultEndPointSelector]
(Work-Job-Executor-148:ctx-ef42787d job-72454/job-72455 ctx-d1890501)
(logid:f365dd96) IR24 select BACKUPSNAPSHOT from primary to secondary 867
dest=867
2024-09-26 16:58:40,236 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
(Work-Job-Executor-148:ctx-ef42787d job-72454/job-72455 ctx-d1890501)
(logid:f365dd96) getCommandHostDelegation: class
org.apache.cloudstack.storage.command.CopyCommand
2024-09-26 16:58:40,238 DEBUG [c.c.h.XenServerGuru]
(Work-Job-Executor-148:ctx-ef42787d job-72454/job-72455 ctx-d1890501)
(logid:f365dd96) We are returning the default host to execute commands because
the source and destination objects are not NFS type.
2024-09-26 16:58:40,243 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-148:ctx-ef42787d job-72454/job-72455 ctx-d1890501)
(logid:f365dd96) Seq 77358-1940207014466653252: Sending { Cmd , MgmtId:
108597816265214, via: 77358(xcp-sr246-u1112.aisrp.local), Ver: v1, Flags:
100111,
[{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"e1971e58-5b6e-4f59-a256-154280cf4d16","volume":{"uuid":"da420806-0ff3-43b8-9ee0-4aa2adfd9b99","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"DM7100_LUN3","name":"cluster5-dm7100_lun3","id":"53","poolType":"PreSetup","host":"localhost","path":"/DM7100_LUN3","port":"0","url":"PreSetup://localhost/DM7100_LUN3/?ROLE=Primary&STOREUUID=DM7100_LUN3","isManaged":"false"}},"name":"ROOT-5468","size":"(20.00
GB)
21474836480","path":"9e2a4d6e-3ab8-46f4-85cf-c7454fc9ea17","volumeId":"6156","vmName":"i-225-5468-VM","accountId":"225","format":"VHD","provisi
oningType":"THIN","poolId":"53","id":"6156","deviceId":"0","cacheMode":"NONE","hypervisorType":"XenServer","directDownload":"false","deployAsIs":"false","followRedirects":"true"},"parentSnapshotPath":"70424963-5933-4e73-b185-cfdc402175cd","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"DM7100_LUN3","name":"cluster5-dm7100_lun3","id":"53","poolType":"PreSetup","host":"localhost","path":"/DM7100_LUN3","port":"0","url":"PreSetup://localhost/DM7100_LUN3/?ROLE=Primary&STOREUUID=DM7100_LUN3","isManaged":"false"}},"vmName":"i-225-5468-VM","name":"backup-test-karasev-3_ROOT-5468_20240926135832","hypervisorType":"XenServer","id":"867","quiescevm":"false","parents":["70424963-5933-4e73-b185-cfdc402175cd"],"physicalSize":"0","accountId":"225","followRedirects":"false"}},"destTO":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"snapshots/225/6156","volume":{"uuid":"da420806-0ff3-43b8-9ee0-4aa2adfd9b99","volumeType":"ROOT","dataStore":{"org.apache.cloudst
ack.storage.to.PrimaryDataStoreTO":{"uuid":"DM7100_LUN3","name":"cluster5-dm7100_lun3","id":"53","poolType":"PreSetup","host":"localhost","path":"/DM7100_LUN3","port":"0","url":"PreSetup://localhost/DM7100_LUN3/?ROLE=Primary&STOREUUID=DM7100_LUN3","isManaged":"false"}},"name":"ROOT-5468","size":"(20.00
GB)
21474836480","path":"9e2a4d6e-3ab8-46f4-85cf-c7454fc9ea17","volumeId":"6156","vmName":"i-225-5468-VM","accountId":"225","format":"VHD","provisioningType":"THIN","poolId":"53","id":"6156","deviceId":"0","cacheMode":"NONE","hypervisorType":"XenServer","directDownload":"false","deployAsIs":"false","followRedirects":"true"},"dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://10.69.105.254/var/NFS","_role":"Image"}},"vmName":"i-225-5468-VM","name":"backup-test-karasev-3_ROOT-5468_20240926135832","hypervisorType":"XenServer","id":"867","quiescevm":"false","physicalSize":"0","accountId":"225","followRedirects":"false"}},"executeInSequence":"true","options":{"snapshot.backup.to.sec
ondary":"true","fullSnapshot":"false"},"options2":{},"wait":"21600","bypassHostMaintenance":"false"}}]
}
2024-09-26 16:58:40,245 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-148:ctx-ef42787d job-72454/job-72455 ctx-d1890501)
(logid:f365dd96) Seq 77358-1940207014466653252: Executing: { Cmd , MgmtId:
108597816265214, via: 77358(xcp-sr246-u1112.aisrp.local), Ver: v1, Flags:
100111,
[{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"e1971e58-5b6e-4f59-a256-154280cf4d16","volume":{"uuid":"da420806-0ff3-43b8-9ee0-4aa2adfd9b99","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"DM7100_LUN3","name":"cluster5-dm7100_lun3","id":"53","poolType":"PreSetup","host":"localhost","path":"/DM7100_LUN3","port":"0","url":"PreSetup://localhost/DM7100_LUN3/?ROLE=Primary&STOREUUID=DM7100_LUN3","isManaged":"false"}},"name":"ROOT-5468","size":"(20.00
GB)
21474836480","path":"9e2a4d6e-3ab8-46f4-85cf-c7454fc9ea17","volumeId":"6156","vmName":"i-225-5468-VM","accountId":"225","format":"VHD","prov
isioningType":"THIN","poolId":"53","id":"6156","deviceId":"0","cacheMode":"NONE","hypervisorType":"XenServer","directDownload":"false","deployAsIs":"false","followRedirects":"true"},"parentSnapshotPath":"70424963-5933-4e73-b185-cfdc402175cd","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"DM7100_LUN3","name":"cluster5-dm7100_lun3","id":"53","poolType":"PreSetup","host":"localhost","path":"/DM7100_LUN3","port":"0","url":"PreSetup://localhost/DM7100_LUN3/?ROLE=Primary&STOREUUID=DM7100_LUN3","isManaged":"false"}},"vmName":"i-225-5468-VM","name":"backup-test-karasev-3_ROOT-5468_20240926135832","hypervisorType":"XenServer","id":"867","quiescevm":"false","parents":["70424963-5933-4e73-b185-cfdc402175cd"],"physicalSize":"0","accountId":"225","followRedirects":"false"}},"destTO":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"snapshots/225/6156","volume":{"uuid":"da420806-0ff3-43b8-9ee0-4aa2adfd9b99","volumeType":"ROOT","dataStore":{"org.apache.clou
dstack.storage.to.PrimaryDataStoreTO":{"uuid":"DM7100_LUN3","name":"cluster5-dm7100_lun3","id":"53","poolType":"PreSetup","host":"localhost","path":"/DM7100_LUN3","port":"0","url":"PreSetup://localhost/DM7100_LUN3/?ROLE=Primary&STOREUUID=DM7100_LUN3","isManaged":"false"}},"name":"ROOT-5468","size":"(20.00
GB)
21474836480","path":"9e2a4d6e-3ab8-46f4-85cf-c7454fc9ea17","volumeId":"6156","vmName":"i-225-5468-VM","accountId":"225","format":"VHD","provisioningType":"THIN","poolId":"53","id":"6156","deviceId":"0","cacheMode":"NONE","hypervisorType":"XenServer","directDownload":"false","deployAsIs":"false","followRedirects":"true"},"dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://10.69.105.254/var/NFS","_role":"Image"}},"vmName":"i-225-5468-VM","name":"backup-test-karasev-3_ROOT-5468_20240926135832","hypervisorType":"XenServer","id":"867","quiescevm":"false","physicalSize":"0","accountId":"225","followRedirects":"false"}},"executeInSequence":"true","options":{"snapshot.backup.to.
secondary":"true","fullSnapshot":"false"},"options2":{},"wait":"21600","bypassHostMaintenance":"false"}}]
}
2024-09-26 16:58:40,246 DEBUG [c.c.s.r.StorageSubsystemCommandHandlerBase]
(DirectAgent-322:ctx-abb19c96) (logid:f365dd96) Executing command CopyCommand:
[{"srcTO":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"e1971e58-5b6e-4f59-a256-154280cf4d16","volume":{"uuid":"da420806-0ff3-43b8-9ee0-4aa2adfd9b99","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"DM7100_LUN3","name":"cluster5-dm7100_lun3","id":53,"poolType":"PreSetup","host":"localhost","path":"/DM7100_LUN3","port":0,"url":"PreSetup://localhost/DM7100_LUN3/?ROLE=Primary&STOREUUID=DM7100_LUN3","isManaged":false}},"name":"ROOT-5468","size":21474836480,"path":"9e2a4d6e-3ab8-46f4-85cf-c7454fc9ea17","volumeId":6156,"vmName":"i-225-5468-VM","accountId":225,"format":"VHD","provisioningType":"THIN","poolId":53,"id":6156,"deviceId":0,"cacheMode":"NONE","hypervisorType":"XenServer","directDownload":false,"deployAsIs":false,"followRedirects":true},"parentSnapshotPath":"704249
63-5933-4e73-b185-cfdc402175cd","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"DM7100_LUN3","name":"cluster5-dm7100_lun3","id":53,"poolType":"PreSetup","host":"localhost","path":"/DM7100_LUN3","port":0,"url":"PreSetup://localhost/DM7100_LUN3/?ROLE=Primary&STOREUUID=DM7100_LUN3","isManaged":false}},"vmName":"i-225-5468-VM","name":"backup-test-karasev-3_ROOT-5468_20240926135832","hypervisorType":"XenServer","id":867,"quiescevm":false,"parents":["70424963-5933-4e73-b185-cfdc402175cd"],"physicalSize":0,"accountId":225,"followRedirects":false}},"destTO":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"snapshots/225/6156","volume":{"uuid":"da420806-0ff3-43b8-9ee0-4aa2adfd9b99","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"DM7100_LUN3","name":"cluster5-dm7100_lun3","id":53,"poolType":"PreSetup","host":"localhost","path":"/DM7100_LUN3","port":0,"url":"PreSetup://localhost/DM7100_LUN3/?ROLE=Primary&ST
OREUUID=DM7100_LUN3","isManaged":false}},"name":"ROOT-5468","size":21474836480,"path":"9e2a4d6e-3ab8-46f4-85cf-c7454fc9ea17","volumeId":6156,"vmName":"i-225-5468-VM","accountId":225,"format":"VHD","provisioningType":"THIN","poolId":53,"id":6156,"deviceId":0,"cacheMode":"NONE","hypervisorType":"XenServer","directDownload":false,"deployAsIs":false,"followRedirects":true},"dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://10.69.105.254/var/NFS","_role":"Image"}},"vmName":"i-225-5468-VM","name":"backup-test-karasev-3_ROOT-5468_20240926135832","hypervisorType":"XenServer","id":867,"quiescevm":false,"physicalSize":0,"accountId":225,"followRedirects":false}},"executeInSequence":true,"options":{"snapshot.backup.to.secondary":"true","fullSnapshot":"false"},"options2":{},"wait":21600,"bypassHostMaintenance":false}].
2024-09-26 16:58:41,264 DEBUG [c.c.h.x.r.XenServerStorageProcessor]
(DirectAgent-322:ctx-abb19c96) (logid:f365dd96) No file SR found for path:
/var/cloud_mount/440acca8-622d-3dfb-825d-b3e62d7bba87/snapshots/225/6156
2024-09-26 16:58:41,264 DEBUG [c.c.h.x.r.XenServerStorageProcessor]
(DirectAgent-322:ctx-abb19c96) (logid:f365dd96) Creating file SR for path
[/var/cloud_mount/440acca8-622d-3dfb-825d-b3e62d7bba87/snapshots/225/6156] on
host [4208d709-b553-47c0-aee2-7942d370b70c]
2024-09-26 16:58:41,495 DEBUG [c.c.a.ApiServlet]
(qtp501107890-4824844:ctx-107e93ec) (logid:1bc5742a) ===START=== 172.16.11.61
-- GET
jobId=f365dd96-c018-454a-874c-37dce9396d96&command=queryAsyncJobResult&response=json
2024-09-26 16:58:41,518 DEBUG [c.c.a.ApiServlet]
(qtp501107890-4824844:ctx-107e93ec ctx-f58fcdf2) (logid:1bc5742a) ===END===
172.16.11.61 -- GET
jobId=f365dd96-c018-454a-874c-37dce9396d96&command=queryAsyncJobResult&response=json
2024-09-26 16:58:41,520 DEBUG [c.c.h.x.r.XenServerStorageProcessor]
(DirectAgent-322:ctx-abb19c96) (logid:f365dd96) Unpluging PBD
[c9ae6161-7db8-f308-db93-4fa060c94f9c] of SR
[f6204e02-6efe-3005-95a3-095f19b458ac] as it is not working properly.
2024-09-26 16:58:41,536 DEBUG [c.c.h.x.r.XenServerStorageProcessor]
(DirectAgent-322:ctx-abb19c96) (logid:f365dd96) Forgetting SR
[f6204e02-6efe-3005-95a3-095f19b458ac] as it is not working properly.
2024-09-26 16:58:41,617 DEBUG [c.c.h.x.r.XenServerStorageProcessor]
(DirectAgent-322:ctx-abb19c96) (logid:f365dd96) Could not create file SR
[/var/cloud_mount/440acca8-622d-3dfb-825d-b3e62d7bba87/snapshots/225/6156] on
host [4208d709-b553-47c0-aee2-7942d370b70c].
2024-09-26 16:58:41,617 DEBUG [c.c.h.x.r.XenServerStorageProcessor]
(DirectAgent-322:ctx-abb19c96) (logid:f365dd96) Exception in backupsnapshot
stage due to com.cloud.utils.exception.CloudRuntimeException: Could not
retrieve an already used file SR for path
[/var/cloud_mount/440acca8-622d-3dfb-825d-b3e62d7bba87/snapshots/225/6156] or
create a new file SR on host [4208d709-b553-47c0-aee2-7942d370b70c]
2024-09-26 16:58:41,617 WARN [c.c.h.x.r.XenServerStorageProcessor]
(DirectAgent-322:ctx-abb19c96) (logid:f365dd96) BackupSnapshot Failed due to
Exception in backupsnapshot stage due to
com.cloud.utils.exception.CloudRuntimeException: Could not retrieve an already
used file SR for path
[/var/cloud_mount/440acca8-622d-3dfb-825d-b3e62d7bba87/snapshots/225/6156] or
create a new file SR on host [4208d709-b553-47c0-aee2-7942d370b70c]
2024-09-26 16:58:44,625 DEBUG [c.c.a.ApiServlet]
(qtp501107890-4825573:ctx-e72bd43c) (logid:d5a19d4e) ===START=== 172.16.11.61
-- GET
jobId=f365dd96-c018-454a-874c-37dce9396d96&command=queryAsyncJobResult&response=json
2024-09-26 16:58:44,651 DEBUG [c.c.a.ApiServlet]
(qtp501107890-4825573:ctx-e72bd43c ctx-707eaf3e) (logid:d5a19d4e) ===END===
172.16.11.61 -- GET
jobId=f365dd96-c018-454a-874c-37dce9396d96&command=queryAsyncJobResult&response=json
2024-09-26 16:58:47,762 DEBUG [c.c.a.ApiServlet]
(qtp501107890-4824845:ctx-517f2c19) (logid:c6e8be8b) ===START=== 172.16.11.61
-- GET
jobId=f365dd96-c018-454a-874c-37dce9396d96&command=queryAsyncJobResult&response=json
2024-09-26 16:58:47,784 DEBUG [c.c.a.ApiServlet]
(qtp501107890-4824845:ctx-517f2c19 ctx-9a6894c4) (logid:c6e8be8b) ===END===
172.16.11.61 -- GET
jobId=f365dd96-c018-454a-874c-37dce9396d96&command=queryAsyncJobResult&response=json
2024-09-26 16:58:47,863 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-322:ctx-abb19c96) (logid:f365dd96) Seq 77358-1940207014466653252:
Response Received:
2024-09-26 16:58:47,864 DEBUG [c.c.a.t.Request]
(DirectAgent-322:ctx-abb19c96) (logid:f365dd96) Seq 77358-1940207014466653252:
Processing: { Ans: , MgmtId: 108597816265214, via:
77358(xcp-sr246-u1112.aisrp.local), Ver: v1, Flags: 110,
[{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"result":"false","details":"BackupSnapshot
Failed due to Exception in backupsnapshot stage due to
com.cloud.utils.exception.CloudRuntimeException: Could not retrieve an already
used file SR for path
[/var/cloud_mount/440acca8-622d-3dfb-825d-b3e62d7bba87/snapshots/225/6156] or
create a new file SR on host
[4208d709-b553-47c0-aee2-7942d370b70c]","wait":"0","bypassHostMaintenance":"false"}}]
}
2024-09-26 16:58:47,864 DEBUG [c.c.a.m.AgentAttache]
(DirectAgent-322:ctx-abb19c96) (logid:f365dd96) Seq 77358-1940207014466653252:
No more commands found
2024-09-26 16:58:47,864 DEBUG [c.c.a.t.Request]
(Work-Job-Executor-148:ctx-ef42787d job-72454/job-72455 ctx-d1890501)
(logid:f365dd96) Seq 77358-1940207014466653252: Received: { Ans: , MgmtId:
108597816265214, via: 77358(xcp-sr246-u1112.aisrp.local), Ver: v1, Flags: 110,
{ CopyCmdAnswer } }
2024-09-26 16:58:47,885 DEBUG [c.c.s.s.SnapshotManagerImpl]
(Work-Job-Executor-148:ctx-ef42787d job-72454/job-72455 ctx-d1890501)
(logid:f365dd96) Failed to create snapshotBackupSnapshot Failed due to
Exception in backupsnapshot stage due to
com.cloud.utils.exception.CloudRuntimeException: Could not retrieve an already
used file SR for path
[/var/cloud_mount/440acca8-622d-3dfb-825d-b3e62d7bba87/snapshots/225/6156] or
create a new file SR on host [4208d709-b553-47c0-aee2-7942d370b70c]
2024-09-26 16:58:47,885 DEBUG [c.c.r.ResourceLimitManagerImpl]
(Work-Job-Executor-148:ctx-ef42787d job-72454/job-72455 ctx-d1890501)
(logid:f365dd96) Updating resource Type = snapshot count for Account = 225
Operation = decreasing Amount = 1
2024-09-26 16:58:47,894 DEBUG [c.c.r.ResourceLimitManagerImpl]
(Work-Job-Executor-148:ctx-ef42787d job-72454/job-72455 ctx-d1890501)
(logid:f365dd96) Updating resource Type = secondary_storage count for Account =
225 Operation = decreasing Amount = (20.00 GB) 21474836480
2024-09-26 16:58:47,901 ERROR [o.a.c.s.v.VolumeServiceImpl]
(Work-Job-Executor-148:ctx-ef42787d job-72454/job-72455 ctx-d1890501)
(logid:f365dd96) Take snapshot: 6156 failed
2024-09-26 16:58:47,902 ERROR [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-148:ctx-ef42787d job-72454/job-72455 ctx-d1890501)
(logid:f365dd96) Invocation exception, caused by:
com.cloud.utils.exception.CloudRuntimeException: BackupSnapshot Failed due to
Exception in backupsnapshot stage due to
com.cloud.utils.exception.CloudRuntimeException: Could not retrieve an already
used file SR for path
[/var/cloud_mount/440acca8-622d-3dfb-825d-b3e62d7bba87/snapshots/225/6156] or
create a new file SR on host [4208d709-b553-47c0-aee2-7942d370b70c]
2024-09-26 16:58:47,902 INFO [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-148:ctx-ef42787d job-72454/job-72455 ctx-d1890501)
(logid:f365dd96) Rethrow exception
com.cloud.utils.exception.CloudRuntimeException: BackupSnapshot Failed due to
Exception in backupsnapshot stage due to
com.cloud.utils.exception.CloudRuntimeException: Could not retrieve an already
used file SR for path
[/var/cloud_mount/440acca8-622d-3dfb-825d-b3e62d7bba87/snapshots/225/6156] or
create a new file SR on host [4208d709-b553-47c0-aee2-7942d370b70c]
2024-09-26 16:58:47,902 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-148:ctx-ef42787d job-72454/job-72455) (logid:f365dd96) Done
with run of VM work job: com.cloud.vm.VmWorkTakeVolumeSnapshot for VM 5468, job
origin: 72454
2024-09-26 16:58:47,902 ERROR [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-148:ctx-ef42787d job-72454/job-72455) (logid:f365dd96)
Unable to complete AsyncJobVO: {id:72455, userId: 298, accountId: 225,
instanceType: null, instanceId: null, cmd:
com.cloud.vm.VmWorkTakeVolumeSnapshot, cmdInfo:
rO0ABXNyACVjb20uY2xvdWQudm0uVm1Xb3JrVGFrZVZvbHVtZVNuYXBzaG90BL5gG4Li1c8CAAdaAAthc3luY0JhY2t1cFoACXF1aWVzY2VWbUwADGxvY2F0aW9uVHlwZXQAKUxjb20vY2xvdWQvc3RvcmFnZS9TbmFwc2hvdCRMb2NhdGlvblR5cGU7TAAIcG9saWN5SWR0ABBMamF2YS9sYW5nL0xvbmc7TAAKc25hcHNob3RJZHEAfgACTAAIdm9sdW1lSWRxAH4AAkwAB3pvbmVJZHN0ABBMamF2YS91dGlsL0xpc3Q7eHIAE2NvbS5jbG91ZC52bS5WbVdvcmufmbZW8CVnawIABEoACWFjY291bnRJZEoABnVzZXJJZEoABHZtSWRMAAtoYW5kbGVyTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO3hwAAAAAAAAAOEAAAAAAAABKgAAAAAAABVcdAAUVm9sdW1lQXBpU2VydmljZUltcGwAAHBzcgAOamF2YS5sYW5nLkxvbmc7i-SQzI8j3wIAAUoABXZhbHVleHIAEGphdmEubGFuZy5OdW1iZXKGrJUdC5TgiwIAAHhwAAAAAAAAAABzcQB-AAgAAAAAAAADY3NxAH4ACAAAAAAAABgMcA,
cmdVersion: 0, status: IN_PROGRESS, processSt
atus: 0, resultCode: 0, result: null, initMsid: 108597816265214, completeMsid:
null, lastUpdated: null, lastPolled: null, created: Thu Sep 26 16:58:32 MSK
2024, removed: null}, job origin:72454
2024-09-26 16:58:47,904 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-148:ctx-ef42787d job-72454/job-72455) (logid:f365dd96)
Complete async job-72455, jobStatus: FAILED, resultCode: 0, result:
rO0ABXNyAC9jb20uY2xvdWQudXRpbHMuZXhjZXB0aW9uLkNsb3VkUnVudGltZUV4Y2VwdGlvblZNT3AAAAACAwABSQALY3NFcnJvckNvZGV4cgAaamF2YS5sYW5nLlJ1bnRpbWVFeGNlcHRpb26eXwZHCjSD5QIAAHhyABNqYXZhLmxhbmcuRXhjZXB0aW9u0P0fPho7HMQCAAB4cgATamF2YS5sYW5nLlRocm93YWJsZdXGNSc5d7jLAwAETAAFY2F1c2V0ABVMamF2YS9sYW5nL1Rocm93YWJsZTtMAA1kZXRhaWxNZXNzYWdldAASTGphdmEvbGFuZy9TdHJpbmc7WwAKc3RhY2tUcmFjZXQAHltMamF2YS9sYW5nL1N0YWNrVHJhY2VFbGVtZW50O0wAFHN1cHByZXNzZWRFeGNlcHRpb25zdAAQTGphdmEvdXRpbC9MaXN0O3hwcQB-AAh0ATxCYWNrdXBTbmFwc2hvdCBGYWlsZWQgZHVlIHRvIEV4Y2VwdGlvbiBpbiBiYWNrdXBzbmFwc2hvdCBzdGFnZSBkdWUgdG8gY29tLmNsb3VkLnV0aWxzLmV4Y2VwdGlvbi5DbG91ZFJ1bnRpbWVFeGNlcHRpb246IENvdWxkIG5vdCByZXRyaWV2ZSBhbiBhbHJlYWR5IHVzZWQgZmlsZSBTUiBmb3IgcGF0aCBbL3Zhci9jbG91ZF9tb3VudC80NDBhY2NhOC02MjJkLTNkZmItODI1ZC1iM2U2MmQ3YmJhODcvc25hcHNob3RzL
zIyNS82MTU2XSBvciBjcmVhdGUgYSBuZXcgZmlsZSBTUiBvbiBob3N0IFs0MjA4ZDcwOS1iNTUzLTQ3YzAtYWVlMi03OTQyZDM3MGI3MGNddXIAHltMamF2YS5sYW5nLlN0YWNrVHJhY2VFbGVtZW50OwJGKjw8_SI5AgAAeHAAAAAtc3IAG2phdmEubGFuZy5TdGFja1RyYWNlRWxlbWVudGEJxZomNt2FAgAIQgAGZm9ybWF0SQAKbGluZU51bWJlckwAD2NsYXNzTG9hZGVyTmFtZXEAfgAFTAAOZGVjbGFyaW5nQ2xhc3NxAH4ABUwACGZpbGVOYW1lcQB-AAVMAAptZXRob2ROYW1lcQB-AAVMAAptb2R1bGVOYW1lcQB-AAVMAA1tb2R1bGVWZXJzaW9ucQB-AAV4cAEAAAF-dAADYXBwdAA6b3JnLmFwYWNoZS5jbG91ZHN0YWNrLnN0b3JhZ2Uuc25hcHNob3QuU25hcHNob3RTZXJ2aWNlSW1wbHQAGFNuYXBzaG90U2VydmljZUltcGwuamF2YXQADmJhY2t1cFNuYXBzaG90cHBzcQB-AAwBAAAAwXEAfgAOdAA-b3JnLmFwYWNoZS5jbG91ZHN0YWNrLnN0b3JhZ2Uuc25hcHNob3QuRGVmYXVsdFNuYXBzaG90U3RyYXRlZ3l0ABxEZWZhdWx0U25hcHNob3RTdHJhdGVneS5qYXZhcQB-ABFwcHNxAH4ADAEAAAWfcQB-AA50AC5jb20uY2xvdWQuc3RvcmFnZS5zbmFwc2hvdC5TbmFwc2hvdE1hbmFnZXJJbXBsdAAYU25hcHNob3RNYW5hZ2VySW1wbC5qYXZhdAAZYmFja3VwU25hcHNob3RUb1NlY29uZGFyeXBwc3EAfgAMAQAABWxxAH4ADnEAfgAWcQB-ABd0AAx0YWtlU25hcHNob3RwcHNxAH4ADAD_____cHQAMGpkay5pbnRlcm5hbC5yZW
ZsZWN0LkdlbmVyYXRlZE1ldGhvZEFjY2Vzc29yMTk1N3B0AAZpbnZva2VwcHNxAH4ADAIAAAArcHQAMWpkay5pbnRlcm5hbC5yZWZsZWN0LkRlbGVnYXRpbmdNZXRob2RBY2Nlc3NvckltcGx0ACFEZWxlZ2F0aW5nTWV0aG9kQWNjZXNzb3JJbXBsLmphdmFxAH4AHXQACWphdmEuYmFzZXQABzExLjAuMjRzcQB-AAwCAAACNnB0ABhqYXZhLmxhbmcucmVmbGVjdC5NZXRob2R0AAtNZXRob2QuamF2YXEAfgAdcQB-ACFxAH4AInNxAH4ADAEAAAFYcQB-AA50AChvcmcuc3ByaW5nZnJhbWV3b3JrLmFvcC5zdXBwb3J0LkFvcFV0aWxzdAANQW9wVXRpbHMuamF2YXQAHmludm9rZUpvaW5wb2ludFVzaW5nUmVmbGVjdGlvbnBwc3EAfgAMAQAAAMZxAH4ADnQAPG9yZy5zcHJpbmdmcmFtZXdvcmsuYW9wLmZyYW1ld29yay5SZWZsZWN0aXZlTWV0aG9kSW52b2NhdGlvbnQAH1JlZmxlY3RpdmVNZXRob2RJbnZvY2F0aW9uLmphdmF0AA9pbnZva2VKb2lucG9pbnRwcHNxAH4ADAEAAACjcQB-AA5xAH4AK3EAfgAsdAAHcHJvY2VlZHBwc3EAfgAMAQAAAGFxAH4ADnQAP29yZy5zcHJpbmdmcmFtZXdvcmsuYW9wLmludGVyY2VwdG9yLkV4cG9zZUludm9jYXRpb25JbnRlcmNlcHRvcnQAIEV4cG9zZUludm9jYXRpb25JbnRlcmNlcHRvci5qYXZhcQB-AB1wcHNxAH4ADAEAAAC6cQB-AA5xAH4AK3EAfgAscQB-AC9wcHNxAH4ADAEAAADXcQB-AA50ADRvcmcuc3ByaW5nZnJhbWV3b3JrLmFvcC5mcmFtZXdvcmsuSmRrRHluYW1pY0FvcFByb3h
5dAAXSmRrRHluYW1pY0FvcFByb3h5LmphdmFxAH4AHXBwc3EAfgAMAP____9wdAAXY29tLnN1bi5wcm94eS4kUHJveHkyMzhwcQB-ABpwcHNxAH4ADAEAAArYcQB-AA50ADZvcmcuYXBhY2hlLmNsb3Vkc3RhY2suc3RvcmFnZS52b2x1bWUuVm9sdW1lU2VydmljZUltcGx0ABZWb2x1bWVTZXJ2aWNlSW1wbC5qYXZhcQB-ABpwcHNxAH4ADAEAAA68cQB-AA50ACZjb20uY2xvdWQuc3RvcmFnZS5Wb2x1bWVBcGlTZXJ2aWNlSW1wbHQAGVZvbHVtZUFwaVNlcnZpY2VJbXBsLmphdmF0AB1vcmNoZXN0cmF0ZVRha2VWb2x1bWVTbmFwc2hvdHBwc3EAfgAMAQAAE7lxAH4ADnEAfgA9cQB-AD5xAH4AP3Bwc3EAfgAMAP____9wdAAwamRrLmludGVybmFsLnJlZmxlY3QuR2VuZXJhdGVkTWV0aG9kQWNjZXNzb3IxOTU2cHEAfgAdcHBzcQB-AAwCAAAAK3BxAH4AH3EAfgAgcQB-AB1xAH4AIXEAfgAic3EAfgAMAgAAAjZwcQB-ACRxAH4AJXEAfgAdcQB-ACFxAH4AInNxAH4ADAEAAABpcQB-AA50ACJjb20uY2xvdWQudm0uVm1Xb3JrSm9iSGFuZGxlclByb3h5dAAaVm1Xb3JrSm9iSGFuZGxlclByb3h5LmphdmF0AA9oYW5kbGVWbVdvcmtKb2JwcHNxAH4ADAEAABPHcQB-AA5xAH4APXEAfgA-cQB-AEhwcHNxAH4ADAD_____cHQAMGpkay5pbnRlcm5hbC5yZWZsZWN0LkdlbmVyYXRlZE1ldGhvZEFjY2Vzc29yMTAzNnBxAH4AHXBwc3EAfgAMAgAAACtwcQB-AB9xAH4AIHEAfgAdcQB-ACFxAH4AInNxAH4ADAIAAAI2cHEAfgAkcQB-ACVx
AH4AHXEAfgAhcQB-ACJzcQB-AAwBAAABWHEAfgAOcQB-ACdxAH4AKHEAfgApcHBzcQB-AAwBAAAAxnEAfgAOcQB-ACtxAH4ALHEAfgAtcHBzcQB-AAwBAAAAo3EAfgAOcQB-ACtxAH4ALHEAfgAvcHBzcQB-AAwBAAAAYXEAfgAOcQB-ADFxAH4AMnEAfgAdcHBzcQB-AAwBAAAAunEAfgAOcQB-ACtxAH4ALHEAfgAvcHBzcQB-AAwBAAAA13EAfgAOcQB-ADVxAH4ANnEAfgAdcHBzcQB-AAwA_____3B0ABdjb20uc3VuLnByb3h5LiRQcm94eTI0N3BxAH4ASHBwc3EAfgAMAQAAAGZxAH4ADnQAIGNvbS5jbG91ZC52bS5WbVdvcmtKb2JEaXNwYXRjaGVydAAYVm1Xb3JrSm9iRGlzcGF0Y2hlci5qYXZhdAAGcnVuSm9icHBzcQB-AAwBAAACjnEAfgAOdAA_b3JnLmFwYWNoZS5jbG91ZHN0YWNrLmZyYW1ld29yay5qb2JzLmltcGwuQXN5bmNKb2JNYW5hZ2VySW1wbCQ1dAAYQXN5bmNKb2JNYW5hZ2VySW1wbC5qYXZhdAAMcnVuSW5Db250ZXh0cHBzcQB-AAwBAAAAMHEAfgAOdAA-b3JnLmFwYWNoZS5jbG91ZHN0YWNrLm1hbmFnZWQuY29udGV4dC5NYW5hZ2VkQ29udGV4dFJ1bm5hYmxlJDF0ABtNYW5hZ2VkQ29udGV4dFJ1bm5hYmxlLmphdmF0AANydW5wcHNxAH4ADAEAAAA3cQB-AA50AEJvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0JDF0ABpEZWZhdWx0TWFuYWdlZENvbnRleHQuamF2YXQABGNhbGxwcHNxAH4ADAEAAABmcQB-AA50AEBvcmcuYXBhY2hlLmNsb
3Vkc3RhY2subWFuYWdlZC5jb250ZXh0LmltcGwuRGVmYXVsdE1hbmFnZWRDb250ZXh0cQB-AGR0AA9jYWxsV2l0aENvbnRleHRwcHNxAH4ADAEAAAA0cQB-AA5xAH4AZ3EAfgBkdAAOcnVuV2l0aENvbnRleHRwcHNxAH4ADAEAAAAtcQB-AA50ADxvcmcuYXBhY2hlLmNsb3Vkc3RhY2subWFuYWdlZC5jb250ZXh0Lk1hbmFnZWRDb250ZXh0UnVubmFibGVxAH4AYHEAfgBhcHBzcQB-AAwBAAACWnEAfgAOcQB-AFtxAH4AXHEAfgBhcHBzcQB-AAwCAAACA3B0AC5qYXZhLnV0aWwuY29uY3VycmVudC5FeGVjdXRvcnMkUnVubmFibGVBZGFwdGVydAAORXhlY3V0b3JzLmphdmFxAH4AZXEAfgAhcQB-ACJzcQB-AAwCAAABCHB0AB9qYXZhLnV0aWwuY29uY3VycmVudC5GdXR1cmVUYXNrdAAPRnV0dXJlVGFzay5qYXZhcQB-AGFxAH4AIXEAfgAic3EAfgAMAgAABGhwdAAnamF2YS51dGlsLmNvbmN1cnJlbnQuVGhyZWFkUG9vbEV4ZWN1dG9ydAAXVGhyZWFkUG9vbEV4ZWN1dG9yLmphdmF0AAlydW5Xb3JrZXJxAH4AIXEAfgAic3EAfgAMAgAAAnRwdAAuamF2YS51dGlsLmNvbmN1cnJlbnQuVGhyZWFkUG9vbEV4ZWN1dG9yJFdvcmtlcnEAfgB2cQB-AGFxAH4AIXEAfgAic3EAfgAMAgAAAz1wdAAQamF2YS5sYW5nLlRocmVhZHQAC1RocmVhZC5qYXZhcQB-AGFxAH4AIXEAfgAic3IAH2phdmEudXRpbC5Db2xsZWN0aW9ucyRFbXB0eUxpc3R6uBe0PKee3gIAAHhweAAAEJp3CAAAAAAAAAAAeA
2024-09-26 16:58:47,905 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-148:ctx-ef42787d job-72454/job-72455) (logid:f365dd96)
Publish async job-72455 complete on message bus
2024-09-26 16:58:47,905 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-148:ctx-ef42787d job-72454/job-72455) (logid:f365dd96) Wake
up jobs related to job-72455
2024-09-26 16:58:47,905 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-148:ctx-ef42787d job-72454/job-72455) (logid:f365dd96)
Update db status for job-72455
2024-09-26 16:58:47,906 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-148:ctx-ef42787d job-72454/job-72455) (logid:f365dd96) Wake
up jobs joined with job-72455 and disjoin all subjobs created from job- 72455
2024-09-26 16:58:47,914 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Work-Job-Executor-148:ctx-ef42787d job-72454/job-72455) (logid:f365dd96) Done
executing com.cloud.vm.VmWorkTakeVolumeSnapshot for job-72455
2024-09-26 16:58:47,916 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(Work-Job-Executor-148:ctx-ef42787d job-72454/job-72455) (logid:f365dd96)
Remove job-72455 from job monitoring
2024-09-26 16:58:47,930 ERROR [o.a.c.a.c.u.s.CreateSnapshotCmd]
(API-Job-Executor-58:ctx-c9d433a7 job-72454 ctx-829ed64e) (logid:f365dd96)
Failed to create snapshot due to an internal error creating snapshot for volume
da420806-0ff3-43b8-9ee0-4aa2adfd9b99
2024-09-26 16:58:47,935 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-58:ctx-c9d433a7 job-72454) (logid:f365dd96) Complete async
job-72454, jobStatus: FAILED, resultCode: 530, result:
org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"530","errortext":"Failed
to create snapshot due to an internal error creating snapshot for volume
da420806-0ff3-43b8-9ee0-4aa2adfd9b99"}
2024-09-26 16:58:47,936 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-58:ctx-c9d433a7 job-72454) (logid:f365dd96) Publish async
job-72454 complete on message bus
2024-09-26 16:58:47,936 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-58:ctx-c9d433a7 job-72454) (logid:f365dd96) Wake up jobs
related to job-72454
2024-09-26 16:58:47,936 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-58:ctx-c9d433a7 job-72454) (logid:f365dd96) Update db status
for job-72454
2024-09-26 16:58:47,937 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-58:ctx-c9d433a7 job-72454) (logid:f365dd96) Wake up jobs
joined with job-72454 and disjoin all subjobs created from job- 72454
2024-09-26 16:58:47,941 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-58:ctx-c9d433a7 job-72454) (logid:f365dd96) Done executing
org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd for job-72454
2024-09-26 16:58:47,941 INFO [o.a.c.f.j.i.AsyncJobMonitor]
(API-Job-Executor-58:ctx-c9d433a7 job-72454) (logid:f365dd96) Remove job-72454
from job monitoring
2024-09-26 16:58:50,891 DEBUG [c.c.a.ApiServlet]
(qtp501107890-4824742:ctx-04767efc) (logid:00d3b60a) ===START=== 172.16.11.61
-- GET
jobId=f365dd96-c018-454a-874c-37dce9396d96&command=queryAsyncJobResult&response=json
2024-09-26 16:58:50,913 DEBUG [c.c.a.ApiServlet]
(qtp501107890-4824742:ctx-04767efc ctx-ff2eca28) (logid:00d3b60a) ===END===
172.16.11.61 -- GET
jobId=f365dd96-c018-454a-874c-37dce9396d96&command=queryAsyncJobResult&response=json
~~~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
