On Mon, Sep 21, 2020 at 3:30 PM Jeremey Wise <jeremey.w...@gmail.com> wrote:
>
>
> Old System
> Three servers..  Centos 7 -> Lay down VDO (dedup / compression) add those VDO 
> volumes as bricks to gluster.
>
> New cluster (remove boot drives and run wipe of all data drives)
>
> Goal: use first 512GB Drives to ignite the cluster and get things on feet and 
> stage infrastructure things.  Then use one of the 1TB drives in each server 
> for my "production" volume.  And second 1TB drive in each server as staging.  
> I want to be able to "learn" and not loose days / weeks of data... so disk 
> level rather give up capacity for sake of "oh.. well .. that messed up.. 
> rebuild.
>
> After minimal install.  Setup of network..  run HCI wizard.
>
> It failed various times along build... lack SELInux permissive, .. did not 
> wipe 1TB drives with hope of importing old Gluster file system / VDO voluemes 
> to import my five or six custom and important VMs. (OCP cluster bootstrap 
> environment, Plex servers, DNS / DHCP / Proxy HA cluster nodes et....)
>
> Gave up on too many HCI failures about disk.. so wiped drives (will use 
> external NAS to repopulate important VMs back (or so is plan... see other 
> posting on no import of qcow2 images / xml :P )
>
> Ran into next batch of issues about use of true device ID  ... as name too 
> long... but /dev/sd?  makes me nervious as I have seen many systems with 
> issues when they use this old and should be depricated means to address disk 
> ID:  use UUID  or raw ID... 
> "/dev/disk/by-id/ata-Samsung_SSD_850_PRO_512GB_S250NXAGA15787L
>
> Started getting errors about HCI failing with "excluded by filter" errors.

I'm not sure I follow your long story, but this error is caused by too
strict lvm
filter in /etc/lvm/lvm.conf.

Edit this file and remove the line that looks like this:

    filter = 
["a|^/dev/disk/by-id/lvm-pv-uuid-80ovnb-mZIO-J65Y-rl9n-YAY7-h0Q9-Aezk8D$|",
"r|.*|"]

Then install gluster, it will stop complaining about the filter.

At the end of the installation, you are going to add the hosts to
engine. At this point
a new lvm filter will be created, considering all the mounted logical volumes.

Maybe gluster setup should warn about lvm filter or remove it before
the installation.

> wiped drives ( gdisk /dev/sd?  => x => z => y => y)
>
> filters errors I could not fiture out what they were.. .. error of "filter 
> exists"  to me meant ..  you have one.. remove it so I can remove drive.
>
> Did full dd if=/dev/zero of=dev/sd? ..  still same issue
> filtered in multipath just for grins.... still same issue.
>
> Posted to forums.. nobody had ideas 
> https://forums.centos.org/viewtopic.php?f=54&t=75687   Posted to slack 
> gluster channel.. they looked at it and could not figure out...
>
> Wiped systems.. started over.   This time the HCI wizard deployed.
>
> My guess... is once I polished setup to make sure wizard did not attempt 
> before SELinux set to permissive (vs disable)  drives all wiped (even though 
> they SHOULD just be ignored..  I I think VDO scanned and saw VDO definition 
> on drive so freeked some ansible wizard script out).
>
> Now cluster is up..  but then went to add "production"  gluster +VDO and 
> "staging"  gluster + vdo volumes... and having issues.
>
> Sorry for long back story but I think this will add color to issues.
>
> My Thoughts as to root issues
> 1) HCI wizard has issues just using drives told, and ignoring other data 
> drives in system ... VDO as example I saw notes about failed attempt ... but 
> it should not have touched that volume... just used one it needed and igored 
> rest.
> 2) HCI wizard bug of ignoring user set /dev/sd?  for each server again, was 
> another failure attempt where clean up may not have run. (noted this in 
> posting about manual edit .. and apply button :P to ingest)
> 3) HCI wizard bug of name I was using of device ID vs /sd?  which is IMAO ... 
> bad form.. but name too long.. again. another cleanup where things may not 
> have fully cleaned.. or I forgot to click clean ...  where system was left in 
> non-pristine state
> 2) HCI wizard does NOT clean itself up properly if it fails ... or when I ran 
> clean up, maybe it did not complete and I closed wizard which then created 
> this orphaned state.
> 3) HCI Setup and post setup needs to add filtering
>
>
>   With a perfect and pristine process  .. it ran.   But only when all other 
> learning and requirements to get it just right were setup first.  oVirt HCI 
> is Soooo very close to being a great platform , well thought out and 
> production class.  Just needs some more nerds beating on it to find these 
> cracks, and get the GUI and setup polished.
>
> My $0.002
>
>
> On Mon, Sep 21, 2020 at 8:06 AM Nir Soffer <nsof...@redhat.com> wrote:
>>
>> On Mon, Sep 21, 2020 at 9:02 AM Jeremey Wise <jeremey.w...@gmail.com> wrote:
>> >
>> >
>> >
>> >
>> >
>> >
>> > vdo: ERROR - Device /dev/sdc excluded by a filter
>> >
>> >
>> >
>> >
>> >
>> > Other server
>> >
>> > vdo: ERROR - Device 
>> > /dev/mapper/nvme.126f-4141303030303030303030303030303032343538-53504343204d2e32205043496520535344-00000001p1
>> >  excluded by a filter.
>> >
>> >
>> >
>> > All systems when I go to create VDO volume on blank drives.. I get this 
>> > filter error.  All disk outside of the HCI wizard setup are now blocked 
>> > from creating new Gluster volume group.
>> >
>> > Here is what I see in /dev/lvm/lvm.conf |grep filter
>> > [root@odin ~]# cat /etc/lvm/lvm.conf |grep filter
>> > filter = 
>> > ["a|^/dev/disk/by-id/lvm-pv-uuid-e1fvwo-kEfX-v3lT-SKBp-cgze-TwsO-PtyvmC$|",
>> >  
>> > "a|^/dev/disk/by-id/lvm-pv-uuid-mr9awW-oQH5-F4IX-CbEO-RgJZ-x4jK-e4YZS1$|", 
>> > "r|.*|"]
>>
>> This filter is correct for a normal oVirt host. But gluster wants to
>> use more local disks,
>> so you should:
>>
>> 1. remove the lvm filter
>> 2. configure gluster
>> 3. create the lvm filter
>>
>> This will create a filter including all the mounted logical volumes
>> created by gluster.
>>
>> Can you explain how do you reproduce this?
>>
>> The lvm filter is created when you add a host to engine. Did you add the host
>> to engine before configuring gluster? Or maybe you are trying to add a host 
>> that
>> was used previously by oVirt?
>>
>> In the last case, removing the filter before installing gluster will
>> fix the issue.
>>
>> Nir
>>
>> > [root@odin ~]# ls -al /dev/disk/by-id/
>> > total 0
>> > drwxr-xr-x. 2 root root 1220 Sep 18 14:32 .
>> > drwxr-xr-x. 6 root root  120 Sep 18 14:32 ..
>> > lrwxrwxrwx. 1 root root    9 Sep 18 22:40 
>> > ata-INTEL_SSDSC2BB080G4_BTWL40350DXP080KGN -> ../../sda
>> > lrwxrwxrwx. 1 root root   10 Sep 18 22:40 
>> > ata-INTEL_SSDSC2BB080G4_BTWL40350DXP080KGN-part1 -> ../../sda1
>> > lrwxrwxrwx. 1 root root   10 Sep 18 22:40 
>> > ata-INTEL_SSDSC2BB080G4_BTWL40350DXP080KGN-part2 -> ../../sda2
>> > lrwxrwxrwx. 1 root root    9 Sep 18 14:32 
>> > ata-Micron_1100_MTFDDAV512TBN_17401F699137 -> ../../sdb
>> > lrwxrwxrwx. 1 root root    9 Sep 18 22:40 
>> > ata-WDC_WDS100T2B0B-00YS70_183533804564 -> ../../sdc
>> > lrwxrwxrwx. 1 root root   10 Sep 18 16:40 dm-name-cl-home -> ../../dm-2
>> > lrwxrwxrwx. 1 root root   10 Sep 18 16:40 dm-name-cl-root -> ../../dm-0
>> > lrwxrwxrwx. 1 root root   10 Sep 18 16:40 dm-name-cl-swap -> ../../dm-1
>> > lrwxrwxrwx. 1 root root   11 Sep 18 16:40 
>> > dm-name-gluster_vg_sdb-gluster_lv_data -> ../../dm-11
>> > lrwxrwxrwx. 1 root root   10 Sep 18 16:40 
>> > dm-name-gluster_vg_sdb-gluster_lv_engine -> ../../dm-6
>> > lrwxrwxrwx. 1 root root   11 Sep 18 16:40 
>> > dm-name-gluster_vg_sdb-gluster_lv_vmstore -> ../../dm-12
>> > lrwxrwxrwx. 1 root root   10 Sep 18 23:35 
>> > dm-name-nvme.126f-4141303030303030303030303030303032343538-53504343204d2e32205043496520535344-00000001
>> >  -> ../../dm-3
>> > lrwxrwxrwx. 1 root root   10 Sep 18 23:49 
>> > dm-name-nvme.126f-4141303030303030303030303030303032343538-53504343204d2e32205043496520535344-00000001p1
>> >  -> ../../dm-4
>> > lrwxrwxrwx. 1 root root   10 Sep 18 14:32 dm-name-vdo_sdb -> ../../dm-5
>> > lrwxrwxrwx. 1 root root   10 Sep 18 16:40 
>> > dm-uuid-LVM-GpvYIuypEfrR7nEDn5uHPenKwjrsn4ADc49gc6PWLRBCoJ2B3JC9tDJejyx5eDPT
>> >  -> ../../dm-1
>> > lrwxrwxrwx. 1 root root   10 Sep 18 16:40 
>> > dm-uuid-LVM-GpvYIuypEfrR7nEDn5uHPenKwjrsn4ADOMNJfgcat9ZLOpcNO7FyG8ixcl5s93TU
>> >  -> ../../dm-2
>> > lrwxrwxrwx. 1 root root   10 Sep 18 16:40 
>> > dm-uuid-LVM-GpvYIuypEfrR7nEDn5uHPenKwjrsn4ADzqPGk0yTQ19FIqgoAfsCxWg7cDMtl71r
>> >  -> ../../dm-0
>> > lrwxrwxrwx. 1 root root   10 Sep 18 16:40 
>> > dm-uuid-LVM-ikNfztYY7KGT1SI2WYXPz4DhM2cyTelOq6Om5comvRFWJDbtVZAKtE5YGl4jciP9
>> >  -> ../../dm-6
>> > lrwxrwxrwx. 1 root root   11 Sep 18 16:40 
>> > dm-uuid-LVM-ikNfztYY7KGT1SI2WYXPz4DhM2cyTelOqVheASEgerWSEIkjM1BR3us3D9ekHt0L
>> >  -> ../../dm-11
>> > lrwxrwxrwx. 1 root root   11 Sep 18 16:40 
>> > dm-uuid-LVM-ikNfztYY7KGT1SI2WYXPz4DhM2cyTelOQz6vXuivIfup6cquKAjPof8wIGOSe4Vz
>> >  -> ../../dm-12
>> > lrwxrwxrwx. 1 root root   10 Sep 18 23:35 
>> > dm-uuid-mpath-nvme.126f-4141303030303030303030303030303032343538-53504343204d2e32205043496520535344-00000001
>> >  -> ../../dm-3
>> > lrwxrwxrwx. 1 root root   10 Sep 18 23:49 
>> > dm-uuid-part1-mpath-nvme.126f-4141303030303030303030303030303032343538-53504343204d2e32205043496520535344-00000001
>> >  -> ../../dm-4
>> > lrwxrwxrwx. 1 root root   10 Sep 18 14:32 
>> > dm-uuid-VDO-472035cc-8d2b-40ac-afe9-fa60b62a887f -> ../../dm-5
>> > lrwxrwxrwx. 1 root root   10 Sep 18 14:32 
>> > lvm-pv-uuid-e1fvwo-kEfX-v3lT-SKBp-cgze-TwsO-PtyvmC -> ../../dm-5
>> > lrwxrwxrwx. 1 root root   10 Sep 18 22:40 
>> > lvm-pv-uuid-mr9awW-oQH5-F4IX-CbEO-RgJZ-x4jK-e4YZS1 -> ../../sda2
>> > lrwxrwxrwx. 1 root root   13 Sep 18 14:32 
>> > nvme-nvme.126f-4141303030303030303030303030303032343538-53504343204d2e32205043496520535344-00000001
>> >  -> ../../nvme0n1
>> > lrwxrwxrwx. 1 root root   15 Sep 18 14:32 
>> > nvme-nvme.126f-4141303030303030303030303030303032343538-53504343204d2e32205043496520535344-00000001-part1
>> >  -> ../../nvme0n1p1
>> > lrwxrwxrwx. 1 root root   13 Sep 18 14:32 
>> > nvme-SPCC_M.2_PCIe_SSD_AA000000000000002458 -> ../../nvme0n1
>> > lrwxrwxrwx. 1 root root   15 Sep 18 14:32 
>> > nvme-SPCC_M.2_PCIe_SSD_AA000000000000002458-part1 -> ../../nvme0n1p1
>> > lrwxrwxrwx. 1 root root    9 Sep 18 22:40 
>> > scsi-0ATA_INTEL_SSDSC2BB08_BTWL40350DXP080KGN -> ../../sda
>> > lrwxrwxrwx. 1 root root   10 Sep 18 22:40 
>> > scsi-0ATA_INTEL_SSDSC2BB08_BTWL40350DXP080KGN-part1 -> ../../sda1
>> > lrwxrwxrwx. 1 root root   10 Sep 18 22:40 
>> > scsi-0ATA_INTEL_SSDSC2BB08_BTWL40350DXP080KGN-part2 -> ../../sda2
>> > lrwxrwxrwx. 1 root root    9 Sep 18 14:32 
>> > scsi-0ATA_Micron_1100_MTFD_17401F699137 -> ../../sdb
>> > lrwxrwxrwx. 1 root root    9 Sep 18 22:40 
>> > scsi-0ATA_WDC_WDS100T2B0B-_183533804564 -> ../../sdc
>> > lrwxrwxrwx. 1 root root    9 Sep 18 22:40 
>> > scsi-1ATA_INTEL_SSDSC2BB080G4_BTWL40350DXP080KGN -> ../../sda
>> > lrwxrwxrwx. 1 root root   10 Sep 18 22:40 
>> > scsi-1ATA_INTEL_SSDSC2BB080G4_BTWL40350DXP080KGN-part1 -> ../../sda1
>> > lrwxrwxrwx. 1 root root   10 Sep 18 22:40 
>> > scsi-1ATA_INTEL_SSDSC2BB080G4_BTWL40350DXP080KGN-part2 -> ../../sda2
>> > lrwxrwxrwx. 1 root root    9 Sep 18 14:32 
>> > scsi-1ATA_Micron_1100_MTFDDAV512TBN_17401F699137 -> ../../sdb
>> > lrwxrwxrwx. 1 root root    9 Sep 18 22:40 
>> > scsi-1ATA_WDC_WDS100T2B0B-00YS70_183533804564 -> ../../sdc
>> > lrwxrwxrwx. 1 root root    9 Sep 18 22:40 scsi-35001b448b9608d90 -> 
>> > ../../sdc
>> > lrwxrwxrwx. 1 root root    9 Sep 18 14:32 scsi-3500a07511f699137 -> 
>> > ../../sdb
>> > lrwxrwxrwx. 1 root root    9 Sep 18 22:40 scsi-355cd2e404b581cc0 -> 
>> > ../../sda
>> > lrwxrwxrwx. 1 root root   10 Sep 18 22:40 scsi-355cd2e404b581cc0-part1 -> 
>> > ../../sda1
>> > lrwxrwxrwx. 1 root root   10 Sep 18 22:40 scsi-355cd2e404b581cc0-part2 -> 
>> > ../../sda2
>> > lrwxrwxrwx. 1 root root   10 Sep 18 23:35 
>> > scsi-nvme.126f-4141303030303030303030303030303032343538-53504343204d2e32205043496520535344-00000001
>> >  -> ../../dm-3
>> > lrwxrwxrwx. 1 root root   10 Sep 18 23:49 
>> > scsi-nvme.126f-4141303030303030303030303030303032343538-53504343204d2e32205043496520535344-00000001-part1
>> >  -> ../../dm-4
>> > lrwxrwxrwx. 1 root root    9 Sep 18 22:40 
>> > scsi-SATA_INTEL_SSDSC2BB08_BTWL40350DXP080KGN -> ../../sda
>> > lrwxrwxrwx. 1 root root   10 Sep 18 22:40 
>> > scsi-SATA_INTEL_SSDSC2BB08_BTWL40350DXP080KGN-part1 -> ../../sda1
>> > lrwxrwxrwx. 1 root root   10 Sep 18 22:40 
>> > scsi-SATA_INTEL_SSDSC2BB08_BTWL40350DXP080KGN-part2 -> ../../sda2
>> > lrwxrwxrwx. 1 root root    9 Sep 18 14:32 
>> > scsi-SATA_Micron_1100_MTFD_17401F699137 -> ../../sdb
>> > lrwxrwxrwx. 1 root root    9 Sep 18 22:40 
>> > scsi-SATA_WDC_WDS100T2B0B-_183533804564 -> ../../sdc
>> > lrwxrwxrwx. 1 root root    9 Sep 18 22:40 wwn-0x5001b448b9608d90 -> 
>> > ../../sdc
>> > lrwxrwxrwx. 1 root root    9 Sep 18 14:32 wwn-0x500a07511f699137 -> 
>> > ../../sdb
>> > lrwxrwxrwx. 1 root root    9 Sep 18 22:40 wwn-0x55cd2e404b581cc0 -> 
>> > ../../sda
>> > lrwxrwxrwx. 1 root root   10 Sep 18 22:40 wwn-0x55cd2e404b581cc0-part1 -> 
>> > ../../sda1
>> > lrwxrwxrwx. 1 root root   10 Sep 18 22:40 wwn-0x55cd2e404b581cc0-part2 -> 
>> > ../../sda2
>> > lrwxrwxrwx. 1 root root   10 Sep 18 23:35 
>> > wwn-0xvme.126f-4141303030303030303030303030303032343538-53504343204d2e32205043496520535344-00000001
>> >  -> ../../dm-3
>> > lrwxrwxrwx. 1 root root   10 Sep 18 23:49 
>> > wwn-0xvme.126f-4141303030303030303030303030303032343538-53504343204d2e32205043496520535344-00000001-part1
>> >  -> ../../dm-4
>> > lrwxrwxrwx. 1 root root   15 Sep 18 14:32 
>> > wwn-nvme.126f-4141303030303030303030303030303032343538-53504343204d2e32205043496520535344-00000001-part1
>> >  -> ../../nvme0n1p1
>> > [root@odin ~]# ls -al /dev/disk/by-id/
>> >
>> > So filter notes three objects:
>> > lvm-pv-uuid-e1fvwo.... -> dm-5  ->vdo_sdb  (used by HCI for all the three 
>> > gluster base volumes )
>> > lvm-pv-uuid-mr9awW... -> sda2 -> boot volume
>> >
>> > [root@odin ~]# lsblk
>> > NAME                                                                       
>> >                           MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
>> > sda                                                                        
>> >                             8:0    0  74.5G  0 disk
>> > ├─sda1                                                                     
>> >                             8:1    0     1G  0 part  /boot
>> > └─sda2                                                                     
>> >                             8:2    0  73.5G  0 part
>> >   ├─cl-root                                                                
>> >                           253:0    0  44.4G  0 lvm   /
>> >   ├─cl-swap                                                                
>> >                           253:1    0   7.5G  0 lvm   [SWAP]
>> >   └─cl-home                                                                
>> >                           253:2    0  21.7G  0 lvm   /home
>> > sdb                                                                        
>> >                             8:16   0   477G  0 disk
>> > └─vdo_sdb                                                                  
>> >                           253:5    0   2.1T  0 vdo
>> >   ├─gluster_vg_sdb-gluster_lv_engine                                       
>> >                           253:6    0   100G  0 lvm   /gluster_bricks/engine
>> >   ├─gluster_vg_sdb-gluster_thinpool_gluster_vg_sdb_tmeta                   
>> >                           253:7    0     1G  0 lvm
>> >   │ └─gluster_vg_sdb-gluster_thinpool_gluster_vg_sdb-tpool                 
>> >                           253:9    0     2T  0 lvm
>> >   │   ├─gluster_vg_sdb-gluster_thinpool_gluster_vg_sdb                     
>> >                           253:10   0     2T  1 lvm
>> >   │   ├─gluster_vg_sdb-gluster_lv_data                                     
>> >                           253:11   0  1000G  0 lvm   /gluster_bricks/data
>> >   │   └─gluster_vg_sdb-gluster_lv_vmstore                                  
>> >                           253:12   0  1000G  0 lvm   
>> > /gluster_bricks/vmstore
>> >   └─gluster_vg_sdb-gluster_thinpool_gluster_vg_sdb_tdata                   
>> >                           253:8    0     2T  0 lvm
>> >     └─gluster_vg_sdb-gluster_thinpool_gluster_vg_sdb-tpool                 
>> >                           253:9    0     2T  0 lvm
>> >       ├─gluster_vg_sdb-gluster_thinpool_gluster_vg_sdb                     
>> >                           253:10   0     2T  1 lvm
>> >       ├─gluster_vg_sdb-gluster_lv_data                                     
>> >                           253:11   0  1000G  0 lvm   /gluster_bricks/data
>> >       └─gluster_vg_sdb-gluster_lv_vmstore                                  
>> >                           253:12   0  1000G  0 lvm   
>> > /gluster_bricks/vmstore
>> > sdc                                                                        
>> >                             8:32   0 931.5G  0 disk
>> > nvme0n1                                                                    
>> >                           259:0    0 953.9G  0 disk
>> > ├─nvme.126f-4141303030303030303030303030303032343538-53504343204d2e32205043496520535344-00000001
>> >      253:3    0 953.9G  0 mpath
>> > │ 
>> > └─nvme.126f-4141303030303030303030303030303032343538-53504343204d2e32205043496520535344-00000001p1
>> >  253:4    0 953.9G  0 part
>> > └─nvme0n1p1
>> >
>> > So I don't think this is LVM filtering things
>> >
>> > Multipath showing weird treatment of the NVMe drive.. but that is outside 
>> > this converstation
>> > [root@odin ~]# multipath -l
>> > nvme.126f-4141303030303030303030303030303032343538-53504343204d2e32205043496520535344-00000001
>> >  dm-3 NVME,SPCC M.2 PCIe SSD
>> > size=954G features='1 queue_if_no_path' hwhandler='0' wp=rw
>> > `-+- policy='service-time 0' prio=0 status=active
>> >   `- 0:1:1:1 nvme0n1 259:0 active undef running
>> > [root@odin ~]#
>> >
>> > Where is getting this filter.
>> > I have done gdisk /dev/sdc ( new 1TB Drive) and shows no partition.  I 
>> > even did a full dd if=/dev/zero   and no change.
>> >
>> > I reloaded OS on system to get through wizard setup.  Now that all three 
>> > nodes are in the HCI cluster..  all six drives (2 x 1TB in each server) 
>> > are now locked from any use due to this error about filter.
>> >
>> > Ideas?
>> >
>> > --
>> > jeremey.w...@gmail.com
>> > _______________________________________________
>> > Users mailing list -- users@ovirt.org
>> > To unsubscribe send an email to users-le...@ovirt.org
>> > Privacy Statement: https://www.ovirt.org/privacy-policy.html
>> > oVirt Code of Conduct: 
>> > https://www.ovirt.org/community/about/community-guidelines/
>> > List Archives: 
>> > https://lists.ovirt.org/archives/list/users@ovirt.org/message/JIG3DCS72QCYYYSWF5XTQIMQSQVXDMSR/
>>
>
>
> --
> jeremey.w...@gmail.com
_______________________________________________
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/W7SDTL46CR2KGSGTKYSK22Z66NZOHGWR/

Reply via email to