Re: [ovirt-users] ovirt 4.1 hosted engine: clear Task error: Operation is not allowed in this task state: ("can't clean in state running", )
Hi, 2017-10-11 16:13 GMT+02:00 Adam Litke : > What is the status of your Datacenter? > The status of datacenter is "operational" > Are these hosts both operational? > yes > Are you experiencing other problems with your storage other than the > inconsistent task state? > No, what kind of problem? > Do you see the KeyError: 'VERSION' message related to domain > *b6730d64-2cf8-42a3-8f08-24b8cc2c0cd8 > also on Node02? * > Is from node 2 > *Did you experience any disaster (power outage, FC storage outage, > network, etc) around the time this started happening?* > No ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] ovirt 4.1 hosted engine: clear Task error: Operation is not allowed in this task state: ("can't clean in state running", )
What is the status of your Datacenter? Are these hosts both operational? Are you experiencing other problems with your storage other than the inconsistent task state?Do you see the KeyError: 'VERSION' message related to domain *b6730d64-2cf8-42a3-8f08-24b8cc2c0cd8 also on Node02? Did you experience any disaster (power outage, FC storage outage, network, etc) around the time this started happening?* On Wed, Oct 11, 2017 at 9:21 AM, yayo (j) wrote: > Hi all, > > ovirt 4.1 hosted engine on 2 node cluster and FC LUN Storage > > I'm trying to clear some task pending from months using vdsClient but I > can't do anything. Below are the steps (on node 1, the SPM): > > 1. Show all tasks: > > > > > > > > > > > > > > > > > > > > *# vdsClient -s 0 getAllTasksInfofd319af4-d160-48ce-b682-5a908333a5e1 : > verb = createVolume id = > fd319af4-d160-48ce-b682-5a908333a5e19bbc2bc4-3c73-4814-a785-6ea737904528 : >verb = prepareMerge id = > 9bbc2bc4-3c73-4814-a785-6ea737904528e70feb21-964d-49d9-9b5a-8e3f70a92db1 : >verb = prepareMerge id = > e70feb21-964d-49d9-9b5a-8e3f70a92db1cf064461-f0ab-4e44-a68f-b2d58fa83a21 : >verb = prepareMerge id = > cf064461-f0ab-4e44-a68f-b2d58fa83a2185b7cf4e-d658-4785-94f0-391fe9616b41 : >verb = prepareMerge id = > 85b7cf4e-d658-4785-94f0-391fe9616b417416627a-fe50-4353-b129-e01bba066a66 : >verb = prepareMerge id = > 7416627a-fe50-4353-b129-e01bba066a66* > > > 2. Stop all tasks (repeted for every task): > > *# vdsClient -s 0 stopTask 7416627a-fe50-4353-b129-e01bba066a66 * > Task is aborted: u'7416627a-fe50-4353-b129-e01bba066a66' - code 411 > > 3. Tring to clear tasks: > > * # vdsClient -s 0 clearTask 7416627a-fe50-4353-b129-e01bba066a66* > *Operation is not allowed in this task state: ("can't clean in state > running",)* > > > > *On Node 01 (the SPM) I have multiple errors > in /var/log/vdsm/vdsm.log like this:* > > *2017-10-11 15:09:53,719+0200 INFO (jsonrpc/3) [storage.TaskManager.Task] > (Task='9519d4db-2960-4b88-82f2-e4c1094eac54') aborting: Task is aborted: > u'Operation is not allowed in this task state: ("can\'t clean in state > running",)' - code 100 (task:1175)* > *2017-10-11 15:09:53,719+0200 ERROR (jsonrpc/3) [storage.Dispatcher] > FINISH clearTask error=Operation is not allowed in this task state: ("can't > clean in state running",) (dispatcher:78)* > *2017-10-11 15:09:53,720+0200 INFO (jsonrpc/3) [jsonrpc.JsonRpcServer] > RPC call Task.clear failed (error 410) in 0.01 seconds (__init__:539)* > *2017-10-11 15:09:53,743+0200 INFO (jsonrpc/6) [vdsm.api] START > clearTask(taskID=u'7416627a-fe50-4353-b129-e01bba066a66', spUUID=None, > options=None) from=:::192.168.0.226,36724, flow_id=7cd340ec (api:46)* > *2017-10-11 15:09:53,743+0200 INFO (jsonrpc/6) [vdsm.api] FINISH > clearTask error=Operation is not allowed in this task state: ("can't clean > in state running",) from=:::192.168.0.226,36724, flow_id=7cd340ec > (api:50)* > *2017-10-11 15:09:53,743+0200 ERROR (jsonrpc/6) [storage.TaskManager.Task] > (Task='0e12e052-2aca-480d-b50f-5de01ddebe35') Unexpected error (task:870)* > *Traceback (most recent call last):* > * File "/usr/share/vdsm/storage/task.py", line 877, in _run* > *return fn(*args, **kargs)* > * File "", line 2, in clearTask* > * File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, in > method* > *ret = func(*args, **kwargs)* > * File "/usr/share/vdsm/storage/hsm.py", line 2258, in clearTask* > *return self.taskMng.clearTask(taskID=taskID)* > * File "/usr/share/vdsm/storage/taskManager.py", line 175, in clearTask* > *t.clean()* > * File "/usr/share/vdsm/storage/task.py", line 1047, in clean* > *raise se.TaskStateError("can't clean in state %s" % self.state)* > *TaskStateError: Operation is not allowed in this task state: ("can't > clean in state running",)* > > > *On Node 02 (is a 2 node cluster) I have other errors (I don't know if are > related):* > > *2017-10-11 15:11:57,083+0200 INFO (jsonrpc/7) [storage.LVM] Refreshing > lvs: vg=b50c1f5c-aa2c-4a53-9f89-83517fa70d3b lvs=['leases'] (lvm:1291)* > *2017-10-11 15:11:57,084+0200 INFO (jsonrpc/7) [storage.LVM] Refreshing > LVs (vg=b50c1f5c-aa2c-4a53-9f89-83517fa70d3b, lvs=['leases']) (lvm:1319)* > *2017-10-11 15:11:57,124+0200 INFO (jsonrpc/7) [storage.VolumeManifest] > b50c1f5c-aa2c-4a53-9f89-83517fa70d3b/d42f671e-1745-46c1-9e1c-2833245675fc/c86afaa5-6ca8-4fcb-a27e-ffbe0133fe23 > info is {'status': 'OK', 'domain': 'b50c1f5c-aa2c-4a53-9f89-83517fa70d3b', > 'voltype': 'LEAF', 'description': 'hosted-engine.metadata', 'parent': > '----', 'format': 'RAW', 'generation': 0, > 'image': 'd42f671e-1745-46c1-9e1c-2833245675fc', 'ctime': '1499437345', > 'disktype': '2', 'legality': 'LEGAL', 'mtime': '0', 'apparentsize': > '134217728', 'children': [], 'pool': '', 'capacity': '134217728', 'uuid': > u'c86afaa5-6ca8-4fcb-a27e-ffbe0133fe23', 'truesize': '134217
[ovirt-users] ovirt 4.1 hosted engine: clear Task error: Operation is not allowed in this task state: ("can't clean in state running", )
Hi all, ovirt 4.1 hosted engine on 2 node cluster and FC LUN Storage I'm trying to clear some task pending from months using vdsClient but I can't do anything. Below are the steps (on node 1, the SPM): 1. Show all tasks: *# vdsClient -s 0 getAllTasksInfofd319af4-d160-48ce-b682-5a908333a5e1 : verb = createVolume id = fd319af4-d160-48ce-b682-5a908333a5e19bbc2bc4-3c73-4814-a785-6ea737904528 : verb = prepareMerge id = 9bbc2bc4-3c73-4814-a785-6ea737904528e70feb21-964d-49d9-9b5a-8e3f70a92db1 : verb = prepareMerge id = e70feb21-964d-49d9-9b5a-8e3f70a92db1cf064461-f0ab-4e44-a68f-b2d58fa83a21 : verb = prepareMerge id = cf064461-f0ab-4e44-a68f-b2d58fa83a2185b7cf4e-d658-4785-94f0-391fe9616b41 : verb = prepareMerge id = 85b7cf4e-d658-4785-94f0-391fe9616b417416627a-fe50-4353-b129-e01bba066a66 : verb = prepareMerge id = 7416627a-fe50-4353-b129-e01bba066a66* 2. Stop all tasks (repeted for every task): *# vdsClient -s 0 stopTask 7416627a-fe50-4353-b129-e01bba066a66 * Task is aborted: u'7416627a-fe50-4353-b129-e01bba066a66' - code 411 3. Tring to clear tasks: * # vdsClient -s 0 clearTask 7416627a-fe50-4353-b129-e01bba066a66* *Operation is not allowed in this task state: ("can't clean in state running",)* *On Node 01 (the SPM) I have multiple errors in /var/log/vdsm/vdsm.log like this:* *2017-10-11 15:09:53,719+0200 INFO (jsonrpc/3) [storage.TaskManager.Task] (Task='9519d4db-2960-4b88-82f2-e4c1094eac54') aborting: Task is aborted: u'Operation is not allowed in this task state: ("can\'t clean in state running",)' - code 100 (task:1175)* *2017-10-11 15:09:53,719+0200 ERROR (jsonrpc/3) [storage.Dispatcher] FINISH clearTask error=Operation is not allowed in this task state: ("can't clean in state running",) (dispatcher:78)* *2017-10-11 15:09:53,720+0200 INFO (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call Task.clear failed (error 410) in 0.01 seconds (__init__:539)* *2017-10-11 15:09:53,743+0200 INFO (jsonrpc/6) [vdsm.api] START clearTask(taskID=u'7416627a-fe50-4353-b129-e01bba066a66', spUUID=None, options=None) from=:::192.168.0.226,36724, flow_id=7cd340ec (api:46)* *2017-10-11 15:09:53,743+0200 INFO (jsonrpc/6) [vdsm.api] FINISH clearTask error=Operation is not allowed in this task state: ("can't clean in state running",) from=:::192.168.0.226,36724, flow_id=7cd340ec (api:50)* *2017-10-11 15:09:53,743+0200 ERROR (jsonrpc/6) [storage.TaskManager.Task] (Task='0e12e052-2aca-480d-b50f-5de01ddebe35') Unexpected error (task:870)* *Traceback (most recent call last):* * File "/usr/share/vdsm/storage/task.py", line 877, in _run* *return fn(*args, **kargs)* * File "", line 2, in clearTask* * File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, in method* *ret = func(*args, **kwargs)* * File "/usr/share/vdsm/storage/hsm.py", line 2258, in clearTask* *return self.taskMng.clearTask(taskID=taskID)* * File "/usr/share/vdsm/storage/taskManager.py", line 175, in clearTask* *t.clean()* * File "/usr/share/vdsm/storage/task.py", line 1047, in clean* *raise se.TaskStateError("can't clean in state %s" % self.state)* *TaskStateError: Operation is not allowed in this task state: ("can't clean in state running",)* *On Node 02 (is a 2 node cluster) I have other errors (I don't know if are related):* *2017-10-11 15:11:57,083+0200 INFO (jsonrpc/7) [storage.LVM] Refreshing lvs: vg=b50c1f5c-aa2c-4a53-9f89-83517fa70d3b lvs=['leases'] (lvm:1291)* *2017-10-11 15:11:57,084+0200 INFO (jsonrpc/7) [storage.LVM] Refreshing LVs (vg=b50c1f5c-aa2c-4a53-9f89-83517fa70d3b, lvs=['leases']) (lvm:1319)* *2017-10-11 15:11:57,124+0200 INFO (jsonrpc/7) [storage.VolumeManifest] b50c1f5c-aa2c-4a53-9f89-83517fa70d3b/d42f671e-1745-46c1-9e1c-2833245675fc/c86afaa5-6ca8-4fcb-a27e-ffbe0133fe23 info is {'status': 'OK', 'domain': 'b50c1f5c-aa2c-4a53-9f89-83517fa70d3b', 'voltype': 'LEAF', 'description': 'hosted-engine.metadata', 'parent': '----', 'format': 'RAW', 'generation': 0, 'image': 'd42f671e-1745-46c1-9e1c-2833245675fc', 'ctime': '1499437345', 'disktype': '2', 'legality': 'LEGAL', 'mtime': '0', 'apparentsize': '134217728', 'children': [], 'pool': '', 'capacity': '134217728', 'uuid': u'c86afaa5-6ca8-4fcb-a27e-ffbe0133fe23', 'truesize': '134217728', 'type': 'PREALLOCATED', 'lease': {'owners': [], 'version': None}} (volume:272)* *2017-10-11 15:11:57,125+0200 INFO (jsonrpc/7) [vdsm.api] FINISH getVolumeInfo return={'info': {'status': 'OK', 'domain': 'b50c1f5c-aa2c-4a53-9f89-83517fa70d3b', 'voltype': 'LEAF', 'description': 'hosted-engine.metadata', 'parent': '----', 'format': 'RAW', 'generation': 0, 'image': 'd42f671e-1745-46c1-9e1c-2833245675fc', 'ctime': '1499437345', 'disktype': '2', 'legality': 'LEGAL', 'mtime': '0', 'apparentsize': '134217728', 'children': [], 'pool': '', 'capacity': '134217728', 'uuid': u'c86afaa5-6ca8-4fcb-a27e-ffbe0133fe