On Wed, Sep 6, 2017 at 1:22 PM, Arsène Gschwind <[email protected]> wrote:
> > > On 09/05/2017 09:02 AM, Arsène Gschwind wrote: > > > > On 09/04/2017 07:59 PM, Simone Tiraboschi wrote: > > > On Mon, Sep 4, 2017 at 7:32 PM, Arsène Gschwind <[email protected] > > wrote: > >> >> >> On 09/04/2017 06:32 PM, Simone Tiraboschi wrote: >> >> >> >> On Mon, Sep 4, 2017 at 6:24 PM, Arsène Gschwind < >> [email protected]> wrote: >> >>> >>> >>> On 09/04/2017 02:51 PM, Simone Tiraboschi wrote: >>> >>> >>> >>> On Mon, Sep 4, 2017 at 2:21 PM, Arsène Gschwind < >>> [email protected]> wrote: >>> >>>> >>>> >>>> On 09/04/2017 02:01 PM, Simone Tiraboschi wrote: >>>> >>>> >>>> >>>> On Mon, Sep 4, 2017 at 1:55 PM, Arsène Gschwind < >>>> [email protected]> wrote: >>>> >>>>> >>>>> >>>>> On 09/04/2017 01:52 PM, Simone Tiraboschi wrote: >>>>> >>>>> >>>>> >>>>> On Mon, Sep 4, 2017 at 12:23 PM, Arsène Gschwind < >>>>> [email protected]> wrote: >>>>> >>>>>> Hi Simone, >>>>>> >>>>>> On 09/04/2017 11:14 AM, Simone Tiraboschi wrote: >>>>>> >>>>>> >>>>>> >>>>>> On Mon, Sep 4, 2017 at 10:56 AM, Arsène Gschwind < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Hi Didi, >>>>>>> >>>>>>> On 09/04/2017 10:15 AM, Yedidyah Bar David wrote: >>>>>>> >>>>>>> On Mon, Sep 4, 2017 at 10:16 AM, Arsène >>>>>>> Gschwind<[email protected]> <[email protected]> wrote: >>>>>>> >>>>>>> Hi all, >>>>>>> >>>>>>> A while ago I had some problem with hosted-engine network which wasn't >>>>>>> set >>>>>>> correctly at deploy time, so I finally decided to redeploy the hosted >>>>>>> engine >>>>>>> in the hope the network will be set correctly this time. I've followed >>>>>>> this >>>>>>> procedure: >>>>>>> >>>>>>> Stop all VMs >>>>>>> Full backup of HE DB and export to safe place >>>>>>> Cleanup HE storage following https://access.redhat.com/solutions/2121581 >>>>>>> Reboot Hosts >>>>>>> Re-deploy HE until DB recovery >>>>>>> Recover DB adding the following param: >>>>>>> --he-remove-storage-vm Removes the hosted-engine storage >>>>>>> domain, all its entities and the hosted-engine VM during restore. >>>>>>> --he-remove-hosts Removes all the hosted-engine hosts >>>>>>> during restore. >>>>>>> >>>>>>> Finalize HE deployment. >>>>>>> >>>>>>> Everything did run without errors and I'm able to access Web UI. >>>>>>> >>>>>>> But now I don't see my HE VM and its respective Storage Domain, the logs >>>>>>> says it isn't able to import it. I see all other SD and I'm able to >>>>>>> manage >>>>>>> my VMs as before. >>>>>>> >>>>>>> Please find attached engine.log >>>>>>> >>>>>>> I think this is your problem: >>>>>>> >>>>>>> 2017-09-04 03:26:14,272+02 INFO >>>>>>> [org.ovirt.engine.core.bll.storage.domain.AddExistingBlockStorageDomainCommand] >>>>>>> (org.ovirt.thread.pool-6-thread-24) [2383eaa0] There are existing luns >>>>>>> in the system which are part of VG id >>>>>>> 'vvIoS2-fZTZ-99Ox-Ltzq-pr8U-SL2z-wbTU8g' >>>>>>> >>>>>>> I don't see a VG with this ID, here the IDs I see on the hosts: >>>>>>> >>>>>>> VG #PV #LV #SN Attr VSize >>>>>>> VFree >>>>>>> 6b62cc06-fc44-4c38-af6d-bfd9cbe73246 1 10 0 wz--n- 99.62g >>>>>>> 14.50g >>>>>>> b0414c06-d984-4001-a998-fd9a2e79fb83 2 70 0 wz--n- 10.00t >>>>>>> 2.31t >>>>>>> b2e30961-7cff-4cca-83d6-bee3a4f890ee 2 47 0 wz--n- 5.27t >>>>>>> 2.50t >>>>>>> >>>>>> >>>>>> >>>>>> Could you please repeat the command on host adm-kvmh70 ? >>>>>> >>>>>> 2017-09-04 09:04:18,163+02 INFO [org.ovirt.engine.core.bll.st >>>>>> orage.domain.ImportHostedEngineStorageDomainCommand] >>>>>> (org.ovirt.thread.pool-6-thread-34) [247a3718] Running command: >>>>>> ImportHostedEngineStorageDomainCommand internal: true. >>>>>> 2017-09-04 09:04:18,189+02 INFO [org.ovirt.engine.core.vdsbro >>>>>> ker.vdsbroker.GetVGInfoVDSCommand] (org.ovirt.thread.pool-6-thread-34) >>>>>> [7d2e6cb2] START, GetVGInfoVDSCommand(HostName = adm-kvmh70, >>>>>> GetVGInfoVDSCommandParameters:{runAsync='true', >>>>>> hostId='acbacabb-6c4a-43fd-a1e2-2d7ff2f6f98b', >>>>>> VGID='vvIoS2-fZTZ-99Ox-Ltzq-pr8U-SL2z-wbTU8g'}), log id: 6693b98a >>>>>> 2017-09-04 09:04:18,232+02 INFO [org.ovirt.engine.core.vdsbro >>>>>> ker.vdsbroker.GetVGInfoVDSCommand] (org.ovirt.thread.pool-6-thread-34) >>>>>> [7d2e6cb2] FINISH, GetVGInfoVDSCommand, return: >>>>>> [LUNs:{id='repl_HostedEngine', >>>>>> physicalVolumeId='kYN8Jj-FBDw-MhxI-XcoZ-w1zH-eQL8-IRIgzO', >>>>>> volumeGroupId='vvIoS2-fZTZ-99Ox-Ltzq-pr8U-SL2z-wbTU8g', >>>>>> serial='SHITACHI_OPEN-V_50488888', lunMapping='4', >>>>>> vendorId='HITACHI', productId='OPEN-V', lunConnections='[]', >>>>>> deviceSize='100', pvSize='0', peCount='797', peAllocatedCount='681', >>>>>> vendorName='HITACHI', pathsDictionary='[sdf=true, sdu=true, sdk=true, >>>>>> sdp=true]', pathsCapacity='[sdf=100, sdu=100, sdk=100, sdp=100]', >>>>>> lunType='FCP', status='null', diskId='null', diskAlias='null', >>>>>> storageDomainId='6b62cc06-fc44-4c38-af6d-bfd9cbe73246', >>>>>> storageDomainName='null', discardMaxSize='268435456', >>>>>> discardZeroesData='true'}], log id: 6693b98a >>>>>> 2017-09-04 09:04:18,245+02 INFO [org.ovirt.engine.core.bll.st >>>>>> orage.domain.AddExistingBlockStorageDomainCommand] >>>>>> (org.ovirt.thread.pool-6-thread-34) [7d2e6cb2] There are existing >>>>>> luns in the system which are part of VG id 'vvIoS2-fZTZ-99Ox-Ltzq-pr8U-SL >>>>>> 2z-wbTU8g' >>>>>> 2017-09-04 09:04:18,245+02 WARN [org.ovirt.engine.core.bll.st >>>>>> orage.domain.AddExistingBlockStorageDomainCommand] >>>>>> (org.ovirt.thread.pool-6-thread-34) [7d2e6cb2] Validation of action >>>>>> 'AddExistingBlockStorageDomain' failed for user SYSTEM. Reasons: >>>>>> VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__ADD,ACTION_TYPE_FAIL >>>>>> ED_IMPORT_STORAGE_DOMAIN_EXTERNAL_LUN_DISK_EXIST >>>>>> >>>>>> I don't know which command you are talking about, I didn't run any >>>>>> command since it tries to import the SD automatically. >>>>>> >>>>> >>>>> Sorry, can you please run vgdisplay on adm-kvmh70 ? >>>>> >>>>> No problem! here the result: >>>>> >>>>> [root@adm-kvmh70 ~]# vgs >>>>> VG #PV #LV #SN Attr VSize >>>>> VFree >>>>> 6b62cc06-fc44-4c38-af6d-bfd9cbe73246 1 10 0 wz--n- 99.62g >>>>> 14.50g >>>>> b0414c06-d984-4001-a998-fd9a2e79fb83 2 70 0 wz--n- 10.00t >>>>> 2.31t >>>>> b2e30961-7cff-4cca-83d6-bee3a4f890ee 2 47 0 wz--n- 5.27t >>>>> 2.50t >>>>> vg_adm-kvmh70 1 3 0 wz--n- 277.90g >>>>> 218.68g >>>>> >>>>> >>>> OK, and >>>> vgdisplay 6b62cc06-fc44-4c38-af6d-bfd9cbe73246 >>>> >>>> [root@adm-kvmh70 ~]# vgdisplay 6b62cc06-fc44-4c38-af6d-bfd9cbe73246 >>>> --- Volume group --- >>>> VG Name 6b62cc06-fc44-4c38-af6d-bfd9cbe73246 >>>> System ID >>>> Format lvm2 >>>> Metadata Areas 2 >>>> Metadata Sequence No 23 >>>> VG Access read/write >>>> VG Status resizable >>>> MAX LV 0 >>>> Cur LV 10 >>>> Open LV 3 >>>> Max PV 0 >>>> Cur PV 1 >>>> Act PV 1 >>>> VG Size 99.62 GiB >>>> PE Size 128.00 MiB >>>> Total PE 797 >>>> Alloc PE / Size 681 / 85.12 GiB >>>> Free PE / Size 116 / 14.50 GiB >>>> VG UUID vvIoS2-fZTZ-99Ox-Ltzq-pr8U-SL2z-wbTU8g >>>> >>> >>> OK, vvIoS2-fZTZ-99Ox-Ltzq-pr8U-SL2z-wbTU8g is the uuid of the VG that >>> engine says that it includes the LUN used by the hosted-engine SD as >>> detected by the engine. >>> Its name is 6b62cc06-fc44-4c38-af6d-bfd9cbe73246 which is also the uuid >>> of that storage domain. >>> >>> Now we need to understand what's there. >>> >>> Could you please check if an SD with uuid >>> 6b62cc06-fc44-4c38-af6d-bfd9cbe73246 >>> is visible in the engine? >>> >>> The visible SD in the engine >>> >>> [oVirt shell (connected)]# list storagedomains >>> >>> id : 072fbaa1-08f3-4a40-9f34-a5ca22dd1d74 >>> name : ovirt-image-repository >>> description: Public Glance repository for oVirt >>> >>> id : b2e30961-7cff-4cca-83d6-bee3a4f890ee >>> name : xxxxxxxxxxx >>> >>> id : b0414c06-d984-4001-a998-fd9a2e79fb83 >>> name : xxxxxxxxxxx >>> description: xxxxxxxxxxx >>> >>> id : 71fa1c61-4a68-469f-9a56-835190d32075 >>> name : xxxxxxxxxxx >>> >>> id : 9817e64b-fdd9-4d4c-91ee-f3482bb0cc56 >>> name : xxxxxxxxxxxx >>> description: xxxxxxxxxxxx >>> >>> Is adm-kvmh70 involved in hosted-engine? did you redeployed it? >>> >>> Yes adm-kvmh70 should be a host hosting HE and yes as written before I >>> did redeploy HE because of a network interface problem. I did cleanup the >>> lun using the RedHat guide, redeploy HE and restore the DB using the >>> mentioned parameter to remove HE information. >>> >> >> Could you please set adm-kvmh70 to maintenance mode from the engine and >> then reactivate it or better reboot it while in maintenance mode? >> I suspect that 6b62cc06-fc44-4c38-af6d-bfd9cbe73246 is just a left over >> of the previous hosted-engine storage domain kept open on the host. >> >> I did reboot adm-kvmh70 while in maintenance mode and also reboot the >> engine but it didn't change anything. The import failure is still there... >> I've noticed a ovirt-ha-agent error: >> >> ovirt-ha-agent ovirt_hosted_engine_ha.agent.h >> osted_engine.HostedEngine.config ERROR Unable to identify the OVF_STORE >> volume, falling back to initial vm.conf. Please ensure you already added >> your first data domain for regular VMs >> >> Could that be related? >> > > This is just a side effects of that: the OVF_STORE disk is going to be > created by the engine once it correctly imported the hosted-engine storage > domain. > > Could you please share again your up-to-date engine.log? > > >> Thanks >> >> Please find the latest engine log attached. > > Thanks > > When analyzing the log I do see some warning about existing LUNs which are > not part of the VG, does this means the LUNs are existing in the DB? > That LUN is reported by VDSM to the engine. Could you please share your /etc/ovirt-hosted-engine/hosted-engine.conf? > If yes would it be possible to remove them from the engine DB? > > Thanks for your help, > Arsene > > > >> >>> >>> >>> In the past did you tried to manually import the hosted-engine SD or >>> something similar? >>> >>> No, before the "redeployment" the HE SD was auto imported successfully. >>> >>> Thanks a lot >>> >>> >>> >>> >>> thanks >>> >>> >>> >>> >>>> >>>> Thanks >>>> >>>> ? >>>> >>>> >>>>> >>>>> >>>>> >>>>>> >>>>>> Thanks, >>>>>> Arsene >>>>>> >>>>>> >>>>>> >>>>>>> >>>>>>> >>>>>>> Thanks for any help to resolve that issue. >>>>>>> >>>>>>> I guess you can try to remove this disk/lun from the engine and let it >>>>>>> retry. >>>>>>> >>>>>>> Could let me know how to remove that lun from the engine? >>>>>>> >>>>>>> If the only disk is of the hosted-engine, I guess it should have been >>>>>>> removed by '--he-remove-storage-vm' - if so, please open a bug >>>>>>> describing your flow in detail. Thanks. >>>>>>> >>>>>>> It seems that this option didn't remove the he storage information >>>>>>> and that this VG is still the old one. >>>>>>> >>>>>>> Many thanks for your help >>>>>>> Rgds, >>>>>>> Arsene >>>>>>> >>>>>>> Best, >>>>>>> >>>>>>> >>>>>>> Arsène >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> Arsène Gschwind >>>>>>> Fa. Sapify AG im Auftrag der Universität Basel >>>>>>> IT Services >>>>>>> Klingelbergstr. 70 | CH-4056 Basel | Switzerland >>>>>>> Tel. +41 79 449 25 63 <+41%2079%20449%2025%2063> | >>>>>>> http://its.unibas.ch >>>>>>> ITS-ServiceDesk: [email protected] | +41 61 267 14 11 >>>>>>> <+41%2061%20267%2014%2011> >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Users mailing >>>>>>> [email protected]http://lists.ovirt.org/mailman/listinfo/users >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> *Arsène Gschwind* >>>>>>> Fa. Sapify AG im Auftrag der Universität Basel >>>>>>> IT Services >>>>>>> Klingelbergstr. 70 | CH-4056 Basel | Switzerland >>>>>>> Tel. +41 79 449 25 63 <+41%2079%20449%2025%2063> | >>>>>>> http://its.unibas.ch >>>>>>> >>>>>>> ITS-ServiceDesk: [email protected] | +41 61 267 14 11 >>>>>>> <+41%2061%20267%2014%2011> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Users mailing list >>>>>>> [email protected] >>>>>>> http://lists.ovirt.org/mailman/listinfo/users >>>>>>> >>>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> *Arsène Gschwind* >>>>>> Fa. Sapify AG im Auftrag der Universität Basel >>>>>> IT Services >>>>>> Klingelbergstr. 70 | CH-4056 Basel | Switzerland >>>>>> Tel. +41 79 449 25 63 <+41%2079%20449%2025%2063> | http://its.unibas.ch >>>>>> >>>>>> ITS-ServiceDesk: [email protected] | +41 61 267 14 11 >>>>>> <+41%2061%20267%2014%2011> >>>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> *Arsène Gschwind* >>>>> Fa. Sapify AG im Auftrag der Universität Basel >>>>> IT Services >>>>> Klingelbergstr. 70 | CH-4056 Basel | Switzerland >>>>> Tel. +41 79 449 25 63 <+41%2079%20449%2025%2063> | http://its.unibas.ch >>>>> >>>>> ITS-ServiceDesk: [email protected] | +41 61 267 14 11 >>>>> <+41%2061%20267%2014%2011> >>>>> >>>> >>>> >>>> -- >>>> >>>> *Arsène Gschwind* >>>> Fa. Sapify AG im Auftrag der Universität Basel >>>> IT Services >>>> Klingelbergstr. 70 | CH-4056 Basel | Switzerland >>>> Tel. +41 79 449 25 63 <+41%2079%20449%2025%2063> | http://its.unibas.ch >>>> >>>> ITS-ServiceDesk: [email protected] | +41 61 267 14 11 >>>> <+41%2061%20267%2014%2011> >>>> >>> >>> >>> -- >>> >>> *Arsène Gschwind* >>> Fa. Sapify AG im Auftrag der Universität Basel >>> IT Services >>> Klingelbergstr. 70 | CH-4056 Basel | Switzerland >>> Tel. +41 79 449 25 63 <+41%2079%20449%2025%2063> | http://its.unibas.ch >>> >>> ITS-ServiceDesk: [email protected] | +41 61 267 14 11 >>> <+41%2061%20267%2014%2011> >>> >> >> >> -- >> >> *Arsène Gschwind* >> Fa. Sapify AG im Auftrag der Universität Basel >> IT Services >> Klingelbergstr. 70 | CH-4056 Basel | Switzerland >> Tel. +41 79 449 25 63 <+41%2079%20449%2025%2063> | http://its.unibas.ch >> >> ITS-ServiceDesk: [email protected] | +41 61 267 14 11 >> <+41%2061%20267%2014%2011> >> > > > -- > > *Arsène Gschwind* > Fa. Sapify AG im Auftrag der Universität Basel > IT Services > Klingelbergstr. 70 | CH-4056 Basel | Switzerland > Tel. +41 79 449 25 63 <+41%2079%20449%2025%2063> | http://its.unibas.ch > ITS-ServiceDesk: [email protected] | +41 61 267 14 11 > <+41%2061%20267%2014%2011> > > > _______________________________________________ > Users mailing [email protected]http://lists.ovirt.org/mailman/listinfo/users > > > -- > > *Arsène Gschwind* > Fa. Sapify AG im Auftrag der Universität Basel > IT Services > Klingelbergstr. 70 | CH-4056 Basel | Switzerland > Tel. +41 79 449 25 63 <+41%2079%20449%2025%2063> | http://its.unibas.ch > ITS-ServiceDesk: [email protected] | +41 61 267 14 11 > <+41%2061%20267%2014%2011> > > _______________________________________________ > Users mailing list > [email protected] > http://lists.ovirt.org/mailman/listinfo/users > >
_______________________________________________ Users mailing list [email protected] http://lists.ovirt.org/mailman/listinfo/users
