[ovirt-devel] Re: execution failed: javax.net.ssl.SSLPeerUnverifiedException (was: vdsm.storage.exception.UnknownTask: Task id unknown (was: [oVirt Jenkins] ovirt-system-tests_he-basic-suite-master -
On Tue, Jul 21, 2020 at 3:44 PM Yedidyah Bar David wrote:
> We had several different threads about this failure. I'll use current
> to send a summary:
>
> - Our wildfly build was patched to include newer httpclient/core:
> https://gerrit.ovirt.org/110324
>
> - The appliance was patched to use master snapshot:
> https://gerrit.ovirt.org/110397
>
> - Now he-basic-suite succeeded, first time after many months:
>
> https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/1680/
So that's great news and from now on we should make sure that it stays
green (meaning everybody need to pay the same attention as to the basic
suite)
>
> Thanks to everyone involved!
>
> Best regards,
>
> On Tue, Jul 7, 2020 at 3:41 PM Martin Perina wrote:
> >
> > Hi,
> >
> > I'm not aware of change regarding certificates recently. So is this
> error reproducible outside Jenkins? Or even better is it reproducible on
> some easier flow other than HE installation so we can debug what
> certificate is loaded in VDSM?
> >
> > Thanks,
> > Martin
> >
> > On Tue, Jul 7, 2020 at 2:07 PM Yedidyah Bar David
> wrote:
> >>
> >> On Tue, Jul 7, 2020 at 12:50 PM Yedidyah Bar David
> wrote:
> >> >
> >> > On Wed, Jun 24, 2020 at 2:14 PM Evgeny Slutsky
> wrote:
> >> > >
> >> > > Hi,
> >> > > changing the hostname to include also the domain name fixed the
> cert deployment issue:
> >> > > https://gerrit.ovirt.org/#/c/109842/
> >> > >
> >> > > not sure how it affects the engine certificate content.
> >> > > from my offline discussion with @Martin Perina this was that
> change that could cause it:
> >> > > https://gerrit.ovirt.org/#/c/109636/
> >> > >
> >> > > any thoughts?
> >> >
> >> > Above two patches are merged, but we still fail the same way:
> >> >
> >> >
> https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/1664/
> >> >
> >> >
> https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/1664/artifact/exported-artifacts/test_logs/he-basic-suite-master/post-he_deploy/lago-he-basic-suite-master-host-0/_var_log/ovirt-hosted-engine-setup/engine-logs-2020-07-07T03%3A15%3A01Z/ovirt-engine/engine.log
> >> >
> >> > 2020-07-06 23:04:25,555-04 ERROR
> >> > [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand]
> >> >
> (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-38)
> >> > [fb28ce9] Command 'UploadStreamVDSCommand(HostName =
> >> > lago-he-basic-suite-master-host-0.lago.local,
> >> >
> UploadStreamVDSCommandParameters:{hostId='e096650f-a7d6-4383-b1bb-f2e61327aac0'})'
> >> > execution failed: javax.net.ssl.SSLPeerUnverifiedException:
> >> > Certificate for doesn't
> >> > match any of the subject alternative names:
> >> > [lago-he-basic-suite-master-host-0.lago.local]
> >> >
> >> > Any idea?
> >>
> >> And I now see this is indeed what's failing hosted-engine deploy at:
> >>
> >> 2020-07-07 05:51:58,573-0400 INFO ansible task start {'status': 'OK',
> >> 'ansible_type': 'task', 'ansible_playbook':
> >> '/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml',
> >> 'ansible_task': 'ovirt.hosted_engine_setup : Check OVF_STORE volume
> >> status'}
> >>
> >> (See other thread: [oVirt Jenkins]
> >> ovirt-system-tests_he-basic-suite-master - Build # 1655 - Still
> >> Failing! )
> >>
> >> On a successful run, engine.log has:
> >>
> >> 2020-07-02 18:01:55,527+03 INFO
> >>
> [org.ovirt.engine.core.bll.storage.ovfstore.ProcessOvfUpdateForStorageDomainCommand]
> >> (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-84)
> >> [2b0721d8] Running command: ProcessOvfUpdateForStorageDomainCommand
> >> internal: true. Entities affected : ID:
> >> e102d7b5-1a37-490f-a3e7-20e56c37791f Type: StorageAction group
> >> MANIPULATE_STORAG
> >> E_DOMAIN with role type ADMIN
> >> 2020-07-02 18:01:55,607+03 INFO
> >>
> [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand]
> >> (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-84)
> >> [2b0721d8
> >> ] START, SetVolumeDescriptionVDSCommand(
> >>
> SetVolumeDescriptionVDSCommandParameters:{storagePoolId='b9dccefe-bc61-11ea-8ebe-001a4a231728',
> >> ignoreFailoverLimit='false', storageDomainId='e102d7b
> >> 5-1a37-490f-a3e7-20e56c37791f',
> >> imageGroupId='db934a98-4111-4faf-8cb9-6b36928cd61c',
> >> imageId='f898c40e-1f88-48db-b59b-f2c73162ddb7'}), log id: e203e51
> >> 2020-07-02 18:01:55,609+03 INFO
> >>
> [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand]
> >> (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-84)
> >> [2b0721d8
> >> ] -- executeIrsBrokerCommand: calling 'setVolumeDescription',
> parameters:
> >> 2020-07-02 18:01:55,609+03 INFO
> >>
> [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand]
> >> (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-84)
> >> [2b0721d8
> >> ] ++ spUUID=b9dccefe-bc61-11ea-8ebe-001a4a231728
> >> 2020-07-02 18:01:55,609+03 INFO
> >>
>
[ovirt-devel] Re: execution failed: javax.net.ssl.SSLPeerUnverifiedException (was: vdsm.storage.exception.UnknownTask: Task id unknown (was: [oVirt Jenkins] ovirt-system-tests_he-basic-suite-master -
We had several different threads about this failure. I'll use current
to send a summary:
- Our wildfly build was patched to include newer httpclient/core:
https://gerrit.ovirt.org/110324
- The appliance was patched to use master snapshot:
https://gerrit.ovirt.org/110397
- Now he-basic-suite succeeded, first time after many months:
https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/1680/
Thanks to everyone involved!
Best regards,
On Tue, Jul 7, 2020 at 3:41 PM Martin Perina wrote:
>
> Hi,
>
> I'm not aware of change regarding certificates recently. So is this error
> reproducible outside Jenkins? Or even better is it reproducible on some
> easier flow other than HE installation so we can debug what certificate is
> loaded in VDSM?
>
> Thanks,
> Martin
>
> On Tue, Jul 7, 2020 at 2:07 PM Yedidyah Bar David wrote:
>>
>> On Tue, Jul 7, 2020 at 12:50 PM Yedidyah Bar David wrote:
>> >
>> > On Wed, Jun 24, 2020 at 2:14 PM Evgeny Slutsky wrote:
>> > >
>> > > Hi,
>> > > changing the hostname to include also the domain name fixed the cert
>> > > deployment issue:
>> > > https://gerrit.ovirt.org/#/c/109842/
>> > >
>> > > not sure how it affects the engine certificate content.
>> > > from my offline discussion with @Martin Perina this was that change
>> > > that could cause it:
>> > > https://gerrit.ovirt.org/#/c/109636/
>> > >
>> > > any thoughts?
>> >
>> > Above two patches are merged, but we still fail the same way:
>> >
>> > https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/1664/
>> >
>> > https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/1664/artifact/exported-artifacts/test_logs/he-basic-suite-master/post-he_deploy/lago-he-basic-suite-master-host-0/_var_log/ovirt-hosted-engine-setup/engine-logs-2020-07-07T03%3A15%3A01Z/ovirt-engine/engine.log
>> >
>> > 2020-07-06 23:04:25,555-04 ERROR
>> > [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand]
>> > (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-38)
>> > [fb28ce9] Command 'UploadStreamVDSCommand(HostName =
>> > lago-he-basic-suite-master-host-0.lago.local,
>> > UploadStreamVDSCommandParameters:{hostId='e096650f-a7d6-4383-b1bb-f2e61327aac0'})'
>> > execution failed: javax.net.ssl.SSLPeerUnverifiedException:
>> > Certificate for doesn't
>> > match any of the subject alternative names:
>> > [lago-he-basic-suite-master-host-0.lago.local]
>> >
>> > Any idea?
>>
>> And I now see this is indeed what's failing hosted-engine deploy at:
>>
>> 2020-07-07 05:51:58,573-0400 INFO ansible task start {'status': 'OK',
>> 'ansible_type': 'task', 'ansible_playbook':
>> '/usr/share/ovirt-hosted-engine-setup/ansible/trigger_role.yml',
>> 'ansible_task': 'ovirt.hosted_engine_setup : Check OVF_STORE volume
>> status'}
>>
>> (See other thread: [oVirt Jenkins]
>> ovirt-system-tests_he-basic-suite-master - Build # 1655 - Still
>> Failing! )
>>
>> On a successful run, engine.log has:
>>
>> 2020-07-02 18:01:55,527+03 INFO
>> [org.ovirt.engine.core.bll.storage.ovfstore.ProcessOvfUpdateForStorageDomainCommand]
>> (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-84)
>> [2b0721d8] Running command: ProcessOvfUpdateForStorageDomainCommand
>> internal: true. Entities affected : ID:
>> e102d7b5-1a37-490f-a3e7-20e56c37791f Type: StorageAction group
>> MANIPULATE_STORAG
>> E_DOMAIN with role type ADMIN
>> 2020-07-02 18:01:55,607+03 INFO
>> [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand]
>> (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-84)
>> [2b0721d8
>> ] START, SetVolumeDescriptionVDSCommand(
>> SetVolumeDescriptionVDSCommandParameters:{storagePoolId='b9dccefe-bc61-11ea-8ebe-001a4a231728',
>> ignoreFailoverLimit='false', storageDomainId='e102d7b
>> 5-1a37-490f-a3e7-20e56c37791f',
>> imageGroupId='db934a98-4111-4faf-8cb9-6b36928cd61c',
>> imageId='f898c40e-1f88-48db-b59b-f2c73162ddb7'}), log id: e203e51
>> 2020-07-02 18:01:55,609+03 INFO
>> [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand]
>> (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-84)
>> [2b0721d8
>> ] -- executeIrsBrokerCommand: calling 'setVolumeDescription', parameters:
>> 2020-07-02 18:01:55,609+03 INFO
>> [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand]
>> (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-84)
>> [2b0721d8
>> ] ++ spUUID=b9dccefe-bc61-11ea-8ebe-001a4a231728
>> 2020-07-02 18:01:55,609+03 INFO
>> [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand]
>> (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-84)
>> [2b0721d8
>> ] ++ sdUUID=e102d7b5-1a37-490f-a3e7-20e56c37791f
>> 2020-07-02 18:01:55,609+03 INFO
>> [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand]
>> (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-84)
>> [2b0721d8
>> ] ++
[ovirt-devel] Re: Check OVF_STORE volume status task failures
On Tue, Jul 21, 2020 at 12:56 PM Marcin Sobczyk wrote:
>
> Hi,
>
> this is fixed since some time with [1], but I think that the appliance still
> uses old version of ovirt-engine-wildfly-overlay.
> The fixed version is 19.1.0-2 [2]. Nir, could you take a look at it and bump
> the version in the appliance if necessary?
Should be fixed by https://gerrit.ovirt.org/110397
>
> Thanks, Marcin
>
> [1] https://gerrit.ovirt.org/#/c/110324/
> [2] https://gerrit.ovirt.org/#/c/110324/1/automation/build-artifacts.sh
>
> On 7/20/20 3:53 PM, Artem Hrechanychenko wrote:
>
> Hi all,
> maybe I miss some information, but I'm still have troubles with he
> installation using ost CI.
>
> Is that already fixed ?
>
> https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/10415/
>
> https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/10415/artifact/check-patch.he-basic_suite_master.el8.x86_64/test_logs/he-basic-suite-master/post-he_deploy/lago-he-basic-suite-master-host-0/_var_log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20200720061238-soz41s.log
>
> 2020-07-20 06:36:04,455-0400 INFO
> otopi.ovirt_hosted_engine_setup.ansible_utils
> ansible_utils._process_output:111 TASK [ovirt.hosted_engine_setup : Check
> OVF_STORE volume status]
> 2020-07-20 06:40:22,815-0400 DEBUG
> otopi.ovirt_hosted_engine_setup.ansible_utils
> ansible_utils._process_output:105 {'results': [{'cmd': ['vdsm-client',
> 'Volume', 'getInfo', 'storagepoolID=03a466fe-ca73-11ea-b77e-5452c0a8c863',
> 'storagedomainID=6685cca5-f0e1-4831-acdf-6f7b50596142',
> 'imageID=4d2f7009-5b79-4b44-b0ef-e152bc51649f',
> 'volumeID=7b953d0e-662d-4e72-9fdc-823ea867262b'], 'stdout': '{\n
> "apparentsize": "134217728",\n"capacity": "134217728",\n"children":
> [],\n"ctime": "1595241259",\n"description":
> "{\\"Updated\\":false,\\"Last Updated\\":null,\\"Storage
> Domains\\":[{\\"uuid\\":\\"6685cca5-f0e1-4831-acdf-6f7b50596142\\"}],\\"Disk
> Description\\":\\"OVF_STORE\\"}",\n"disktype": "OVFS",\n"domain":
> "6685cca5-f0e1-4831-acdf-6f7b50596142",\n"format": "RAW",\n
> "generation": 0,\n"image": "4d2f7009-5b79-4b44-b0ef-e152bc51649f",\n
> "lease": {\n"offset": 0,\n"owners": [],\n"path":
> "/rhev/data-center/mnt/lago-he-basic-suite-master-storage:_exports_nfs__he/6685cca5-f0e1-4831-acdf-6f7b50596142/images/4d2f7009-5b79-4b44-b0ef-e152bc51649f/7b953d0e-662d-4e72-9fdc-823ea867262b.lease",\n
> "version": null\n},\n"legality": "LEGAL",\n"mtime":
> "0",\n"parent": "----",\n"pool":
> "",\n"status": "OK",\n"truesize": "134217728",\n"type":
> "PREALLOCATED",\n"uuid": "7b953d0e-662d-4e72-9fdc-823ea867262b",\n
> "voltype": "LEAF"\n}', 'stderr': '', 'rc': 0, 'start': '2020-07-20
> 06:38:13.456845', 'end': '2020-07-20 06:38:13.897280', 'delta':
> '0:00:00.440435', 'changed': True, 'invocation': {'module_args':
> {'_raw_params': 'vdsm-client Volume getInfo
> storagepoolID=03a466fe-ca73-11ea-b77e-5452c0a8c863
> storagedomainID=6685cca5-f0e1-4831-acdf-6f7b50596142
> imageID=4d2f7009-5b79-4b44-b0ef-e152bc51649f
> volumeID=7b953d0e-662d-4e72-9fdc-823ea867262b', 'warn': True, '_uses_shell':
> False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None,
> 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin':
> None}}, 'stdout_lines': ['{', '"apparentsize": "134217728",', '
> "capacity": "134217728",', '"children": [],', '"ctime":
> "1595241259",', '"description": "{\\"Updated\\":false,\\"Last
> Updated\\":null,\\"Storage
> Domains\\":[{\\"uuid\\":\\"6685cca5-f0e1-4831-acdf-6f7b50596142\\"}],\\"Disk
> Description\\":\\"OVF_STORE\\"}",', '"disktype": "OVFS",', '"domain":
> "6685cca5-f0e1-4831-acdf-6f7b50596142",', '"format": "RAW",', '
> "generation": 0,', '"image": "4d2f7009-5b79-4b44-b0ef-e152bc51649f",', '
> "lease": {', '"offset": 0,', '"owners": [],', '
> "path":
> "/rhev/data-center/mnt/lago-he-basic-suite-master-storage:_exports_nfs__he/6685cca5-f0e1-4831-acdf-6f7b50596142/images/4d2f7009-5b79-4b44-b0ef-e152bc51649f/7b953d0e-662d-4e72-9fdc-823ea867262b.lease",',
> '"version": null', '},', '"legality": "LEGAL",', '
> "mtime": "0",', '"parent": "----",', '
> "pool": "",', '"status": "OK",', '"truesize": "134217728",', '
> "type": "PREALLOCATED",', '"uuid":
> "7b953d0e-662d-4e72-9fdc-823ea867262b",', '"voltype": "LEAF"', '}'],
> 'stderr_lines': [], '_ansible_no_log': False, 'failed': True, 'attempts': 12,
> 'item': {'name': 'OVF_STORE', 'image_id':
> '7b953d0e-662d-4e72-9fdc-823ea867262b', 'id':
> '4d2f7009-5b79-4b44-b0ef-e152bc51649f'}, 'ansible_loop_var': 'item',
> '_ansible_item_label': {'name': 'OVF_STORE', 'image_id':
> '7b953d0e-662d-4e72-9fdc-823ea867262b', 'id':
>
[ovirt-devel] Re: Check OVF_STORE volume status task failures
Hi,
this is fixed since some time with [1], but I think that the appliance
still uses old version of ovirt-engine-wildfly-overlay.
The fixed version is 19.1.0-2 [2]. Nir, could you take a look at it and
bump the version in the appliance if necessary?
Thanks, Marcin
[1] https://gerrit.ovirt.org/#/c/110324/
[2] https://gerrit.ovirt.org/#/c/110324/1/automation/build-artifacts.sh
On 7/20/20 3:53 PM, Artem Hrechanychenko wrote:
Hi all,
maybe I miss some information, but I'm still have troubles with he
installation using ost CI.
Is that already fixed ?
https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/10415/
https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/10415/artifact/check-patch.he-basic_suite_master.el8.x86_64/test_logs/he-basic-suite-master/post-he_deploy/lago-he-basic-suite-master-host-0/_var_log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20200720061238-soz41s.log
2020-07-20 06:36:04,455-0400 INFO otopi.ovirt_hosted_engine_setup.ansible_utils
ansible_utils._process_output:111 TASK [ovirt.hosted_engine_setup : Check
OVF_STORE volume status]
2020-07-20 06:40:22,815-0400 DEBUG otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils._process_output:105 {'results': [{'cmd': ['vdsm-client', 'Volume', 'getInfo', 'storagepoolID=03a466fe-ca73-11ea-b77e-5452c0a8c863', 'storagedomainID=6685cca5-f0e1-4831-acdf-6f7b50596142', 'imageID=4d2f7009-5b79-4b44-b0ef-e152bc51649f', 'volumeID=7b953d0e-662d-4e72-9fdc-823ea867262b'], 'stdout': '{\n"apparentsize": "134217728",\n"capacity": "134217728",\n"children": [],\n"ctime": "1595241259",\n"description": "{\\"Updated\\":false,\\"Last Updated\\":null,\\"Storage Domains\\":[{\\"uuid\\":\\"6685cca5-f0e1-4831-acdf-6f7b50596142\\"}],\\"Disk Description\\":\\"OVF_STORE\\"}",\n"disktype": "OVFS",\n"domain": "6685cca5-f0e1-4831-acdf-6f7b50596142",\n"format": "RAW",\n"generation": 0,\n"image": "4d2f7009-5b79-4b44-b0ef-e152bc51649f",\n"lease": {\n"offset": 0,\n"owners": [],\n"path": "/rhev/data-center/mnt/lago-he-basic-suite-master-storage:_exports_nfs__he/6685cca5-f0e1-4831-acdf-6f7b50596142/images/4d2f7009-5b79-4b44-b0ef-e152bc51649f/7b953d0e-662d-4e72-9fdc-823ea867262b.lease",\n"version": null\n},\n"legality": "LEGAL",\n"mtime": "0",\n"parent": "----",\n"pool": "",\n"status": "OK",\n"truesize": "134217728",\n"type": "PREALLOCATED",\n"uuid": "7b953d0e-662d-4e72-9fdc-823ea867262b",\n"voltype": "LEAF"\n}', 'stderr': '', 'rc': 0, 'start': '2020-07-20 06:38:13.456845', 'end': '2020-07-20
06:38:13.897280', 'delta': '0:00:00.440435', 'changed': True, 'invocation': {'module_args': {'_raw_params': 'vdsm-client Volume getInfo storagepoolID=03a466fe-ca73-11ea-b77e-5452c0a8c863 storagedomainID=6685cca5-f0e1-4831-acdf-6f7b50596142 imageID=4d2f7009-5b79-4b44-b0ef-e152bc51649f volumeID=7b953d0e-662d-4e72-9fdc-823ea867262b', 'warn': True, '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': ['{', '"apparentsize": "134217728",', '"capacity": "134217728",', '"children": [],', '"ctime": "1595241259",', '"description": "{\\"Updated\\":false,\\"Last Updated\\":null,\\"Storage Domains\\":[{\\"uuid\\":\\"6685cca5-f0e1-4831-acdf-6f7b50596142\\"}],\\"Disk Description\\":\\"OVF_STORE\\"}",', '"disktype": "OVFS",', '"domain": "6685cca5-f0e1-4831-acdf-6f7b50596142",', '"format": "RAW",', '"generation": 0,', '"image": "4d2f7009-5b79-4b44-b0ef-e152bc51649f",', '"lease": {', '"offset": 0,', '"owners": [],', '"path": "/rhev/data-center/mnt/lago-he-basic-suite-master-storage:_exports_nfs__he/6685cca5-f0e1-4831-acdf-6f7b50596142/images/4d2f7009-5b79-4b44-b0ef-e152bc51649f/7b953d0e-662d-4e72-9fdc-823ea867262b.lease",', '"version": null', '},', '"legality": "LEGAL",', '"mtime": "0",', '"parent": "----",', '"pool": "",', '"status": "OK",', '"truesize": "134217728",', '"type": "PREALLOCATED",', '
"uuid": "7b953d0e-662d-4e72-9fdc-823ea867262b",', '"voltype": "LEAF"', '}'], 'stderr_lines': [], '_ansible_no_log': False, 'failed': True, 'attempts': 12, 'item': {'name': 'OVF_STORE', 'image_id': '7b953d0e-662d-4e72-9fdc-823ea867262b', 'id': '4d2f7009-5b79-4b44-b0ef-e152bc51649f'}, 'ansible_loop_var': 'item', '_ansible_item_label': {'name': 'OVF_STORE', 'image_id': '7b953d0e-662d-4e72-9fdc-823ea867262b', 'id': '4d2f7009-5b79-4b44-b0ef-e152bc51649f'}}, {'cmd': ['vdsm-client', 'Volume', 'getInfo', 'storagepoolID=03a466fe-ca73-11ea-b77e-5452c0a8c863', 'storagedomainID=6685cca5-f0e1-4831-acdf-6f7b50596142', 'imageID=044e384a-dedf-4589-8dfb-beca170138ee', 'volumeID=033d64fd-6f93-42be-84bc-082b03095ef3'], 'stdout': '{\n
[ovirt-devel] ovirt-engine has been tagged (ovirt-engine-4.4.1.10)
___ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-le...@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/7VWGN5AMWNKTDR5BBVU4GF3JXWGDTB6N/
[ovirt-devel] Re: device compatibility interface for live migration with assigned devices
On 2020/7/20 下午6:39, Sean Mooney wrote: On Mon, 2020-07-20 at 11:41 +0800, Jason Wang wrote: On 2020/7/18 上午12:12, Alex Williamson wrote: On Thu, 16 Jul 2020 16:32:30 +0800 Yan Zhao wrote: On Thu, Jul 16, 2020 at 12:16:26PM +0800, Jason Wang wrote: On 2020/7/14 上午7:29, Yan Zhao wrote: hi folks, we are defining a device migration compatibility interface that helps upper layer stack like openstack/ovirt/libvirt to check if two devices are live migration compatible. The "devices" here could be MDEVs, physical devices, or hybrid of the two. e.g. we could use it to check whether - a src MDEV can migrate to a target MDEV, - a src VF in SRIOV can migrate to a target VF in SRIOV, - a src MDEV can migration to a target VF in SRIOV. (e.g. SIOV/SRIOV backward compatibility case) The upper layer stack could use this interface as the last step to check if one device is able to migrate to another device before triggering a real live migration procedure. we are not sure if this interface is of value or help to you. please don't hesitate to drop your valuable comments. (1) interface definition The interface is defined in below way: __userspace /\ \ / \write / read \ /__ ___\|/_ | migration_version | | migration_version |-->check migration - - compatibility device Adevice B a device attribute named migration_version is defined under each device's sysfs node. e.g. (/sys/bus/pci/devices/\:00\:02.0/$mdev_UUID/migration_version). Are you aware of the devlink based device management interface that is proposed upstream? I think it has many advantages over sysfs, do you consider to switch to that? Advantages, such as? My understanding for devlink(netlink) over sysfs (some are mentioned at the time of vDPA sysfs mgmt API discussion) are: i tought netlink was used more a as a configuration protocoal to qurry and confire nic and i guess other devices in its devlink form requireint a tool to be witten that can speak the protocal to interact with. the primary advantate of sysfs is that everything is just a file. there are no addtional depleenceis needed Well, if you try to build logic like introspection on top for a sophisticated hardware, you probably need to have library on top. And it's attribute per file is pretty inefficient. and unlike netlink there are not interoperatblity issues in a coanitnerised env. if you are using diffrenet version of libc and gcc in the contaienr vs the host my understanding is tools like ethtool from ubuntu deployed in a container on a centos host can have issue communicating with the host kernel. Kernel provides stable ABI for userspace, so it's not something that we can't fix. if its jsut a file unless the format the data is returnin in chagnes or the layout of sysfs changes its compatiable regardless of what you use to read it. I believe you can't change sysfs layout which is part of uABI. But as I mentioned below, sysfs has several drawbacks. It's not harm to compare between different approach when you start a new device management API. Thanks - existing users (NIC, crypto, SCSI, ib), mature and stable - much better error reporting (ext_ack other than string or errno) - namespace aware - do not couple with kobject Thanks ___ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-le...@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/N525IDOKAXQ3IGK6LUQJ65AARPAEDIBV/
[ovirt-devel] Re: device compatibility interface for live migration with assigned devices
On Fri, Jul 17, 2020 at 10:12:58AM -0600, Alex Williamson wrote:
<...>
> > yes, in another reply, Alex proposed to use an interface in json format.
> > I guess we can define something like
> >
> > { "self" :
> > [
> > { "pciid" : "8086591d",
> > "driver" : "i915",
> > "gvt-version" : "v1",
> > "mdev_type" : "i915-GVTg_V5_2",
> > "aggregator" : "1",
> > "pv-mode" : "none",
> > }
> > ],
> > "compatible" :
> > [
> > { "pciid" : "8086591d",
> > "driver" : "i915",
> > "gvt-version" : "v1",
> > "mdev_type" : "i915-GVTg_V5_2",
> > "aggregator" : "1"
> > "pv-mode" : "none",
> > },
> > { "pciid" : "8086591d",
> > "driver" : "i915",
> > "gvt-version" : "v1",
> > "mdev_type" : "i915-GVTg_V5_4",
> > "aggregator" : "2"
> > "pv-mode" : "none",
> > },
> > { "pciid" : "8086591d",
> > "driver" : "i915",
> > "gvt-version" : "v2",
> > "mdev_type" : "i915-GVTg_V5_4",
> > "aggregator" : "2"
> > "pv-mode" : "none, ppgtt, context",
> > }
> > ...
> > ]
> > }
> >
> > But as those fields are mostly vendor specific, the userspace can
> > only do simple string comparing, I guess the list would be very long as
> > it needs to enumerate all possible targets.
>
>
> This ignores so much of what I tried to achieve in my example :(
>
sorry, I just was eager to show and confirm the way to list all compatible
combination of mdev_type and mdev attributes.
>
> > also, in some fileds like "gvt-version", is there a simple way to express
> > things like v2+?
>
>
> That's not a reasonable thing to express anyway, how can you be certain
> that v3 won't break compatibility with v2? Sean proposed a versioning
> scheme that accounts for this, using an x.y.z version expressing the
> major, minor, and bugfix versions, where there is no compatibility
> across major versions, minor versions have forward compatibility (ex. 1
> -> 2 is ok, 2 -> 1 is not) and bugfix version number indicates some
> degree of internal improvement that is not visible to the user in terms
> of features or compatibility, but provides a basis for preferring
> equally compatible candidates.
>
right. if self version is v1, it can't know its compatible version is
v2. it can only be done in reverse. i.e.
when self version is v2, it can list its compatible version is v1 and
v2.
and maybe later when self version is v3, there's no v1 in its compatible
list.
In this way, do you think we still need the complex x.y.z versioning scheme?
>
> > If the userspace can read this interface both in src and target and
> > check whether both src and target are in corresponding compatible list, I
> > think it will work for us.
> >
> > But still, kernel should not rely on userspace's choice, the opaque
> > compatibility string is still required in kernel. No matter whether
> > it would be exposed to userspace as an compatibility checking interface,
> > vendor driver would keep this part of code and embed the string into the
> > migration stream. so exposing it as an interface to be used by libvirt to
> > do a safety check before a real live migration is only about enabling
> > the kernel part of check to happen ahead.
>
> As you indicate, the vendor driver is responsible for checking version
> information embedded within the migration stream. Therefore a
> migration should fail early if the devices are incompatible. Is it
but as I know, currently in VFIO migration protocol, we have no way to
get vendor specific compatibility checking string in migration setup stage
(i.e. .save_setup stage) before the device is set to _SAVING state.
In this way, for devices who does not save device data in precopy stage,
the migration compatibility checking is as late as in stop-and-copy
stage, which is too late.
do you think we need to add the getting/checking of vendor specific
compatibility string early in save_setup stage?
> really libvirt's place to second guess what it has been directed to do?
if libvirt uses the scheme of reading compatibility string at source and
writing for checking at the target, it can not be called "a second guess".
It's not a guess, but a confirmation.
> Why would we even proceed to design a user parse-able version interface
> if we still have a dependency on an opaque interface? Thanks,
one reason is that libvirt can't trust the parsing result from
openstack.
Another reason is that libvirt can use this opaque interface easier than
another parsing by itself, in the fact that it would not introduce more
burden to kernel who would write this part of code anyway, no matter
libvirt uses it or not.
Thanks
Yan
___
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct:
[ovirt-devel] Re: device compatibility interface for live migration with assigned devices
On Mon, 2020-07-20 at 11:41 +0800, Jason Wang wrote: > On 2020/7/18 上午12:12, Alex Williamson wrote: > > On Thu, 16 Jul 2020 16:32:30 +0800 > > Yan Zhao wrote: > > > > > On Thu, Jul 16, 2020 at 12:16:26PM +0800, Jason Wang wrote: > > > > On 2020/7/14 上午7:29, Yan Zhao wrote: > > > > > hi folks, > > > > > we are defining a device migration compatibility interface that helps > > > > > upper > > > > > layer stack like openstack/ovirt/libvirt to check if two devices are > > > > > live migration compatible. > > > > > The "devices" here could be MDEVs, physical devices, or hybrid of the > > > > > two. > > > > > e.g. we could use it to check whether > > > > > - a src MDEV can migrate to a target MDEV, > > > > > - a src VF in SRIOV can migrate to a target VF in SRIOV, > > > > > - a src MDEV can migration to a target VF in SRIOV. > > > > > (e.g. SIOV/SRIOV backward compatibility case) > > > > > > > > > > The upper layer stack could use this interface as the last step to > > > > > check > > > > > if one device is able to migrate to another device before triggering > > > > > a real > > > > > live migration procedure. > > > > > we are not sure if this interface is of value or help to you. please > > > > > don't > > > > > hesitate to drop your valuable comments. > > > > > > > > > > > > > > > (1) interface definition > > > > > The interface is defined in below way: > > > > > > > > > >__userspace > > > > > /\ \ > > > > >/ \write > > > > > / read \ > > > > > /__ ___\|/_ > > > > > | migration_version | | migration_version |-->check migration > > > > > - - compatibility > > > > >device Adevice B > > > > > > > > > > > > > > > a device attribute named migration_version is defined under each > > > > > device's > > > > > sysfs node. e.g. > > > > > (/sys/bus/pci/devices/\:00\:02.0/$mdev_UUID/migration_version). > > > > > > > > Are you aware of the devlink based device management interface that is > > > > proposed upstream? I think it has many advantages over sysfs, do you > > > > consider to switch to that? > > > > Advantages, such as? > > > My understanding for devlink(netlink) over sysfs (some are mentioned at > the time of vDPA sysfs mgmt API discussion) are: i tought netlink was used more a as a configuration protocoal to qurry and confire nic and i guess other devices in its devlink form requireint a tool to be witten that can speak the protocal to interact with. the primary advantate of sysfs is that everything is just a file. there are no addtional depleenceis needed and unlike netlink there are not interoperatblity issues in a coanitnerised env. if you are using diffrenet version of libc and gcc in the contaienr vs the host my understanding is tools like ethtool from ubuntu deployed in a container on a centos host can have issue communicating with the host kernel. if its jsut a file unless the format the data is returnin in chagnes or the layout of sysfs changes its compatiable regardless of what you use to read it. > > - existing users (NIC, crypto, SCSI, ib), mature and stable > - much better error reporting (ext_ack other than string or errno) > - namespace aware > - do not couple with kobject > > Thanks > ___ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-le...@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/NZGZCNEUJIZWVQCLTDBRAN5WCVF7TDVD/
[ovirt-devel] Re: device compatibility interface for live migration with assigned devices
On 2020/7/18 上午12:12, Alex Williamson wrote: On Thu, 16 Jul 2020 16:32:30 +0800 Yan Zhao wrote: On Thu, Jul 16, 2020 at 12:16:26PM +0800, Jason Wang wrote: On 2020/7/14 上午7:29, Yan Zhao wrote: hi folks, we are defining a device migration compatibility interface that helps upper layer stack like openstack/ovirt/libvirt to check if two devices are live migration compatible. The "devices" here could be MDEVs, physical devices, or hybrid of the two. e.g. we could use it to check whether - a src MDEV can migrate to a target MDEV, - a src VF in SRIOV can migrate to a target VF in SRIOV, - a src MDEV can migration to a target VF in SRIOV. (e.g. SIOV/SRIOV backward compatibility case) The upper layer stack could use this interface as the last step to check if one device is able to migrate to another device before triggering a real live migration procedure. we are not sure if this interface is of value or help to you. please don't hesitate to drop your valuable comments. (1) interface definition The interface is defined in below way: __userspace /\ \ / \write / read \ /__ ___\|/_ | migration_version | | migration_version |-->check migration - - compatibility device Adevice B a device attribute named migration_version is defined under each device's sysfs node. e.g. (/sys/bus/pci/devices/\:00\:02.0/$mdev_UUID/migration_version). Are you aware of the devlink based device management interface that is proposed upstream? I think it has many advantages over sysfs, do you consider to switch to that? Advantages, such as? My understanding for devlink(netlink) over sysfs (some are mentioned at the time of vDPA sysfs mgmt API discussion) are: - existing users (NIC, crypto, SCSI, ib), mature and stable - much better error reporting (ext_ack other than string or errno) - namespace aware - do not couple with kobject Thanks ___ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-le...@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/PKAGNMWYGBZ6LEJ3ULVZUJOLU6EULLJG/
[ovirt-devel] Re: [ARM64] Possiblity to support oVirt on ARM64
Hi Nir, Thanks alot for the quick response and information, actually I also worked a little bit on libvirt and other from our team has experiences in qemu, those projects works OK on ARM platform, it is just the capability might be less when comparing to other platforms, for example, myself just added the getHost() capability for ARM64 in libvirt and now planning to work on compareCPU() API, after that we will have better migration workflow on ARM platform. For resources, actually, yes, I think we can provide those for oVirt community, could you tell me more details about what kind of resources are more suitable, our current resources are mostly VMs from public cloud, will that be OK? BR, Zhenyu On Mon, Jul 20, 2020 at 2:07 AM Nir Soffer wrote: > On Sun, Jul 19, 2020 at 5:04 PM Zhenyu Zheng > wrote: > > > > Hi oVirt, > > > > We are currently trying to make oVirt work on ARM64 platform, since I'm > quite new to oVirt community, I'm wondering what is the current status > about ARM64 support in the oVirt upstream, as I saw the oVirt Wikipedia > page mentioned there is an ongoing efforts to support ARM platform. We have > a small team here and we are willing to also help to make this work. > > Hi Zhenyu, > > I think this is a great idea, both supporting more hardware, and > enlarging the oVirt > community. > > Regarding hardware support we depend mostly on libvirt and qemu, and I > don't know > that is the status. Adding relevant lists and people. > > I don't know about any effort on oVirt side, but last week I added > arm builds for > ovirt-imageio and it works: > > https://copr.fedorainfracloud.org/coprs/nsoffer/ovirt-imageio-preview/build/1555705/ > > We have many dependendencies, but oVirt itself is mostly python and java, > with > tiny bits in C or using ctypes, so it should not be too hard. > > I think the first thing is getting some hardware for testing. Do you > have such hardware, > or have some contacts that can help to get hardware contribution for this? > > Nir > > ___ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-le...@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/MUYF2KTQVSI4LANQORMLCYQHKT4UUTUX/
