Hi Enrico,

in the past i got remaining tasks cleared by running **engine-setup** again.
There is a part which clears remaining tasks ...

Best Regards 
Oliver
> Am 25.07.2019 um 15:47 schrieb Enrico <[email protected]>:
> 
>  Hi all,
> my ovirt cluster has got 3 Hypervisors runnig Centos 7.5.1804 vdsm is 
> 4.20.39.1-1.el7,
> ovirt engine is 4.2.4.5-1.el7, the storage systems are HP MSA P2000 and 2050 
> (fibre channel). 
> 
> I need to stop one of the hypervisors for maintenance but this system is the 
> storage pool manager.
> 
> For this reason I decided to manually activate SPM in one of the other nodes 
> but this operation is not 
> successful.
> 
> In the ovirt engine (engine.log) the error is this:
> 
> 2019-07-25 12:39:16,744+02 INFO  
> [org.ovirt.engine.core.bll.storage.pool.ForceSelectSPMCommand] (default 
> task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] Running command: 
> ForceSelectSPMCommand internal: false. Entities affected :  ID: 
> 81c9bd3c-ae0a-467f-bf7f-63ab30cd8d9e Type: VDSAction group MANIPULATE_HOST 
> with role type ADMIN
> 2019-07-25 12:39:16,745+02 INFO  
> [org.ovirt.engine.core.vdsbroker.irsbroker.SpmStopOnIrsVDSCommand] (default 
> task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] START, 
> SpmStopOnIrsVDSCommand( 
> SpmStopOnIrsVDSCommandParameters:{storagePoolId='18d57688-6ed4-43b8-bd7c-0665b55950b7',
>  ignoreFailoverLimit='false'}), log id: 37bf4639
> 2019-07-25 12:39:16,747+02 INFO  
> [org.ovirt.engine.core.vdsbroker.irsbroker.ResetIrsVDSCommand] (default 
> task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] START, ResetIrsVDSCommand( 
> ResetIrsVDSCommandParameters:{storagePoolId='18d57688-6ed4-43b8-bd7c-0665b55950b7',
>  ignoreFailoverLimit='false', vdsId='751f3e99-b95e-4c31-bc38-77f5661a0bdc', 
> ignoreStopFailed='false'}), log id: 2522686f
> 2019-07-25 12:39:16,749+02 INFO  
> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (default 
> task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] START, 
> SpmStopVDSCommand(HostName = infn-vm05.management, 
> SpmStopVDSCommandParameters:{hostId='751f3e99-b95e-4c31-bc38-77f5661a0bdc', 
> storagePoolId='18d57688-6ed4-43b8-bd7c-0665b55950b7'}), log id: 1810fd8b
> 2019-07-25 12:39:16,758+02 ERROR 
> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (default 
> task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] SpmStopVDSCommand::Not 
> stopping SPM on vds 'infn-vm05.management', pool id 
> '18d57688-6ed4-43b8-bd7c-0665b55950b7' as there are uncleared tasks 'Task 
> 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e', status 'running''
> 2019-07-25 12:39:16,758+02 INFO  
> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (default 
> task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] FINISH, SpmStopVDSCommand, 
> log id: 1810fd8b
> 2019-07-25 12:39:16,758+02 INFO  
> [org.ovirt.engine.core.vdsbroker.irsbroker.ResetIrsVDSCommand] (default 
> task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] FINISH, ResetIrsVDSCommand, 
> log id: 2522686f
> 2019-07-25 12:39:16,758+02 INFO  
> [org.ovirt.engine.core.vdsbroker.irsbroker.SpmStopOnIrsVDSCommand] (default 
> task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] FINISH, 
> SpmStopOnIrsVDSCommand, log id: 37bf4639
> 2019-07-25 12:39:16,760+02 ERROR 
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] 
> (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] EVENT_ID: 
> USER_FORCE_SELECTED_SPM_STOP_FAILED(4,096), Failed to force select 
> infn-vm07.management as the SPM due to a failure to stop the current SPM.
> 
> while in the hypervisor (SPM) vdsm.log:
> 
> 2019-07-25 12:39:16,744+02 INFO  
> [org.ovirt.engine.core.bll.storage.pool.ForceSelectSPMCommand] (default 
> task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] Running command: 
> ForceSelectSPMCommand internal: false. Entities affected :  ID: 
> 81c9bd3c-ae0a-467f-bf7f-63ab30cd8d9e Type: VDSAction group MANIPULATE_HOST 
> with role type ADMIN
> 2019-07-25 12:39:16,745+02 INFO  
> [org.ovirt.engine.core.vdsbroker.irsbroker.SpmStopOnIrsVDSCommand] (default 
> task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] START, 
> SpmStopOnIrsVDSCommand( 
> SpmStopOnIrsVDSCommandParameters:{storagePoolId='18d57688-6ed4-43b8-bd7c-0665b55950b7',
>  ignoreFailoverLimit='false'}), log id: 37bf4639
> 2019-07-25 12:39:16,747+02 INFO  
> [org.ovirt.engine.core.vdsbroker.irsbroker.ResetIrsVDSCommand] (default 
> task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] START, ResetIrsVDSCommand( 
> ResetIrsVDSCommandParameters:{storagePoolId='18d57688-6ed4-43b8-bd7c-0665b55950b7',
>  ignoreFailoverLimit='false', vdsId='751f3e99-b95e-4c31-bc38-77f5661a0bdc', 
> ignoreStopFailed='false'}), log id: 2522686f
> 2019-07-25 12:39:16,749+02 INFO  
> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (default 
> task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] START, 
> SpmStopVDSCommand(HostName = infn-vm05.management, 
> SpmStopVDSCommandParameters:{hostId='751f3e99-b95e-4c31-bc38-77f5661a0bdc', 
> storagePoolId='18d57688-6ed4-43b8-bd7c-0665b55950b7'}), log id: 1810fd8b
> 2019-07-25 12:39:16,758+02 ERROR 
> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (default 
> task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] SpmStopVDSCommand::Not 
> stopping SPM on vds 'infn-vm05.management', pool id 
> '18d57688-6ed4-43b8-bd7c-0665b55950b7' as there are uncleared tasks 'Task 
> 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e', status 'running''
> 2019-07-25 12:39:16,758+02 INFO  
> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (default 
> task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] FINISH, SpmStopVDSCommand, 
> log id: 1810fd8b
> 2019-07-25 12:39:16,758+02 INFO  
> [org.ovirt.engine.core.vdsbroker.irsbroker.ResetIrsVDSCommand] (default 
> task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] FINISH, ResetIrsVDSCommand, 
> log id: 2522686f
> 2019-07-25 12:39:16,758+02 INFO  
> [org.ovirt.engine.core.vdsbroker.irsbroker.SpmStopOnIrsVDSCommand] (default 
> task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] FINISH, 
> SpmStopOnIrsVDSCommand, log id: 37bf4639
> 2019-07-25 12:39:16,760+02 ERROR 
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] 
> (default task-30) [7c374384-f884-4dc9-87d0-7af27dce706b] EVENT_ID: 
> USER_FORCE_SELECTED_SPM_STOP_FAILED(4,096), Failed to force select 
> infn-vm07.management as the SPM due to a failure to stop the current SPM.
> 2019-07-25 12:39:18,660+02 INFO  
> [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] 
> (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] Task id 
> 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' has passed pre-polling period time and 
> should be polled. Pre-polling period is 60000 millis. 
> 2019-07-25 12:39:18,660+02 INFO  
> [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] 
> (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] Task id 
> 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' has passed pre-polling period time and 
> should be polled. Pre-polling period is 60000 millis. 
> 2019-07-25 12:39:18,750+02 INFO  
> [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] 
> (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] Task id 
> 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' has passed pre-polling period time and 
> should be polled. Pre-polling period is 60000 millis. 
> 2019-07-25 12:39:18,750+02 ERROR 
> [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] 
> (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] 
> BaseAsyncTask::logEndTaskFailure: Task 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' 
> (Parent Command 'Unknown', Parameters Type 
> 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended with 
> failure:
> 2019-07-25 12:39:18,750+02 INFO  
> [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] 
> (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] 
> SPMAsyncTask::ClearAsyncTask: Attempting to clear task 
> 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e'
> 2019-07-25 12:39:18,751+02 INFO  
> [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] 
> (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] START, 
> SPMClearTaskVDSCommand( 
> SPMTaskGuidBaseVDSCommandParameters:{storagePoolId='18d57688-6ed4-43b8-bd7c-0665b55950b7',
>  ignoreFailoverLimit='false', 
> taskId='fdcf4d1b-82fe-49a6-b233-323ebe568f8e'}), log id: 34ae2b2f
> 2019-07-25 12:39:18,752+02 INFO  
> [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] 
> (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] START, 
> HSMClearTaskVDSCommand(HostName = infn-vm05.management, 
> HSMTaskGuidBaseVDSCommandParameters:{hostId='751f3e99-b95e-4c31-bc38-77f5661a0bdc',
>  taskId='fdcf4d1b-82fe-49a6-b233-323ebe568f8e'}), log id: d3a78ad
> 2019-07-25 12:39:18,757+02 INFO  
> [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] 
> (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] FINISH, 
> HSMClearTaskVDSCommand, log id: d3a78ad
> 2019-07-25 12:39:18,757+02 INFO  
> [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] 
> (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] FINISH, 
> SPMClearTaskVDSCommand, log id: 34ae2b2f
> 2019-07-25 12:39:18,757+02 INFO  
> [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] 
> (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] 
> SPMAsyncTask::ClearAsyncTask: At time of attempt to clear task 
> 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' the response code was 'TaskStateError' 
> and message was 'Operation is not allowed in this task state: ("can't clean 
> in state running",)'. Task will not be     cleaned
> 2019-07-25 12:39:18,757+02 INFO  
> [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] 
> (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] 
> BaseAsyncTask::onTaskEndSuccess: Task 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' 
> (Parent Command 'Unknown', Parameters Type 
> 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended 
> successfully.
> 2019-07-25 12:39:18,757+02 INFO  
> [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] 
> (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] 
> SPMAsyncTask::ClearAsyncTask: Attempting to clear task 
> 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e'
> 2019-07-25 12:39:18,758+02 INFO  
> [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] 
> (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] START, 
> SPMClearTaskVDSCommand( 
> SPMTaskGuidBaseVDSCommandParameters:{storagePoolId='18d57688-6ed4-43b8-bd7c-0665b55950b7',
>  ignoreFailoverLimit='false', 
> taskId='fdcf4d1b-82fe-49a6-b233-323ebe568f8e'}), log id: 42de0c2b
> 2019-07-25 12:39:18,759+02 INFO  
> [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] 
> (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] START, 
> HSMClearTaskVDSCommand(HostName = infn-vm05.management, 
> HSMTaskGuidBaseVDSCommandParameters:{hostId='751f3e99-b95e-4c31-bc38-77f5661a0bdc',
>  taskId='fdcf4d1b-82fe-49a6-b233-323ebe568f8e'}), log id: 4895c79c
> 2019-07-25 12:39:18,764+02 INFO  
> [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] 
> (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] FINISH, 
> HSMClearTaskVDSCommand, log id: 4895c79c
> 2019-07-25 12:39:18,764+02 INFO  
> [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] 
> (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] FINISH, 
> SPMClearTaskVDSCommand, log id: 42de0c2b
> 2019-07-25 12:39:18,764+02 INFO  
> [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] 
> (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] 
> SPMAsyncTask::ClearAsyncTask: At time of attempt to clear task 
> 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' the response code was 'TaskStateError' 
> and message was 'Operation is not allowed in this task state: ("can't clean 
> in state running",)'. Task will not be     cleaned
> 2019-07-25 12:39:18,764+02 INFO  
> [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] 
> (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] Task id 
> 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' has passed pre-polling period time and 
> should be polled. Pre-polling period is 60000 millis. 
> 2019-07-25 12:39:18,764+02 INFO  
> [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] 
> (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] Cleaning zombie tasks: 
> Clearing async task 'Unknown' that started at 'Fri May 03 14:48:50 CEST 2019'
> 2019-07-25 12:39:18,764+02 INFO  
> [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] 
> (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] 
> SPMAsyncTask::ClearAsyncTask: Attempting to clear task 
> 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e'
> 2019-07-25 12:39:18,765+02 INFO  
> [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] 
> (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] START, 
> SPMClearTaskVDSCommand( 
> SPMTaskGuidBaseVDSCommandParameters:{storagePoolId='18d57688-6ed4-43b8-bd7c-0665b55950b7',
>  ignoreFailoverLimit='false', 
> taskId='fdcf4d1b-82fe-49a6-b233-323ebe568f8e'}), log id: da77af2
> 2019-07-25 12:39:18,766+02 INFO  
> [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] 
> (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] START, 
> HSMClearTaskVDSCommand(HostName = infn-vm05.management, 
> HSMTaskGuidBaseVDSCommandParameters:{hostId='751f3e99-b95e-4c31-bc38-77f5661a0bdc',
>  taskId='fdcf4d1b-82fe-49a6-b233-323ebe568f8e'}), log id: 530694fb
> 2019-07-25 12:39:18,771+02 INFO  
> [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] 
> (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] FINISH, 
> HSMClearTaskVDSCommand, log id: 530694fb
> 2019-07-25 12:39:18,771+02 INFO  
> [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] 
> (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] FINISH, 
> SPMClearTaskVDSCommand, log id: da77af2
> 2019-07-25 12:39:18,771+02 INFO  
> [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] 
> (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] 
> SPMAsyncTask::ClearAsyncTask: At time of attempt to clear task 
> 'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' the response code was 'TaskStateError' 
> and message was 'Operation is not allowed in this task state: ("can't clean 
> in state running",)'. Task will not be     cleaned
> 
> there is some relation between this error and a task that has remained 
> hanging, from SPM server:
> 
> # vdsm-client Task  getInfo taskID=fdcf4d1b-82fe-49a6-b233-323ebe568f8e
> {
>     "verb": "prepareMerge", 
>     "id": "fdcf4d1b-82fe-49a6-b233-323ebe568f8e"
> }
> 
> # vdsm-client Task  getStatus taskID=fdcf4d1b-82fe-49a6-b233-323ebe568f8e
> {
>     "message": "running job 1 of 1", 
>     "code": 0, 
>     "taskID": "fdcf4d1b-82fe-49a6-b233-323ebe568f8e", 
>     "taskResult": "", 
>     "taskState": "running"
> }
> 
> How can I solve this problem ?
> 
> Thanks a lot for your help !!
> Best Regards
> Enrico 
> 
>  -- 
> _______________________________________________________________________
> 
> Enrico Becchetti                    Servizio di Calcolo e Reti
> 
> Istituto Nazionale di Fisica Nucleare - Sezione di Perugia
> Via Pascoli,c/o Dipartimento di Fisica  06123 Perugia (ITALY)
> Phone:+39 075 5852777             Mail: Enrico.Becchetti<at>pg.infn.it 
> _______________________________________________________________________
> _______________________________________________
> Users mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
> oVirt Code of Conduct: 
> https://www.ovirt.org/community/about/community-guidelines/
> List Archives: 
> https://lists.ovirt.org/archives/list/[email protected]/message/KQBFVUCEFAA2FVBLG7ZGDJOQZLC2FGCI/

_______________________________________________
Users mailing list -- [email protected]
To unsubscribe send an email to [email protected]
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/[email protected]/message/RVFWB537D3FWWBOM6SDIVHMO6GRG6B4O/

Reply via email to