Re: [linux-lvm] " Failed to find physical volume
A possibility I just debugged for a non-booting system. If there is a partition table on the underlying device then that device is not detected as an LVM1/2 member in at least one version of udevd, and won't be seen nor turned on automatically by the systemd-udev code. lvm vgchange -ay worked to enable it (emergency mode, it was the root pv--no udevd involvement) and eventually I found the partition table and removed it and the machine would then boot without needing a manual intervention. dd if=/dev/zero of=device bs=512 count=1 was used once we determined there was a partition signature still left (after partition deletion with fdisk, still had a header), examined with dd if=/dev/device bs=512 count=1 | xxd found 4 non-zero bytes in the block. On Sun, Sep 26, 2021 at 9:22 AM alessandro macuz wrote: > > Thanks Roger, Zdenek, > > I have my ZVOL on my NAS exposed as LUNs. The initiator were switched off and > for unknown reason I found my NAS switched off as well. > It had run for long and I feared the worst (CPU/motherboard/etc). Instead > once powered up everything started to work again but the LUNs that seemed to > jeopardized. > I have many ZVOLs used by ESXi in which I have EVE-NG who uses LVM and such > ZVOLs have the same size so I wanted to inspect them to check the hostname. > > Now some LUNs started to work normally, some others still behave weirdly. I > will run pvs on them with extra debugs to see what's going on. > > Many thanks, > > Alex > > Le jeu. 23 sept. 2021 à 23:48, Roger Heflin a écrit : >> >> If you have lvmetad running and in use then the lvm commands ask it >> what the system has on it. >> >> I have seen on random boots fairly separated systems (rhel7 versions, >> and many years newer fedora systems) at random fail to find one or >> more pv.s >> >> I have disabled it at home, and in my day job we have also disabled >> (across 20k+ systems) as we confirmed it had inconsistency issues >> several times on a variety of our newest installs. >> >> Stopping lvmetad and/or restarting it would generally fix it.But >> it was a source of enough random issues(often failure to mount on a >> boot, so often issues that resulted in page-outs to debug) and did >> not speed things up much enough to be worth it even on devices with >> >2000 SAN volumes. >> >> On Thu, Sep 23, 2021 at 8:52 AM Zdenek Kabelac wrote: >> > >> > Dne 22. 09. 21 v 18:48 alessandro macuz napsal(a): >> > > fdisk correctly identifies the extended partition as 8e. >> > > I wonder which kind of data lvmdiskscan and pvs use in order to list LVM >> > > physical volumes. >> > > Does PVS check some specific metadata within the partition without just >> > > relying on the type of partition displayed by fdisk? >> > > >> > > >> > >> > Hi >> > >> > Yes - PVs do have header signature keeping information about PV attributes >> > and also has the storage area to keep lvm2 metadata. >> > >> > Partition flags known to fdisk are irrelevant. >> > >> > >> > Regards >> > >> > Zdenek >> > >> > ___ >> > linux-lvm mailing list >> > linux-lvm@redhat.com >> > https://listman.redhat.com/mailman/listinfo/linux-lvm >> > read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ >> > ___ linux-lvm mailing list linux-lvm@redhat.com https://listman.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] " Failed to find physical volume
Thanks Roger, Zdenek, I have my ZVOL on my NAS exposed as LUNs. The initiator were switched off and for unknown reason I found my NAS switched off as well. It had run for long and I feared the worst (CPU/motherboard/etc). Instead once powered up everything started to work again but the LUNs that seemed to jeopardized. I have many ZVOLs used by ESXi in which I have EVE-NG who uses LVM and such ZVOLs have the same size so I wanted to inspect them to check the hostname. Now some LUNs started to work normally, some others still behave weirdly. I will run pvs on them with extra debugs to see what's going on. Many thanks, Alex Le jeu. 23 sept. 2021 à 23:48, Roger Heflin a écrit : > If you have lvmetad running and in use then the lvm commands ask it > what the system has on it. > > I have seen on random boots fairly separated systems (rhel7 versions, > and many years newer fedora systems) at random fail to find one or > more pv.s > > I have disabled it at home, and in my day job we have also disabled > (across 20k+ systems) as we confirmed it had inconsistency issues > several times on a variety of our newest installs. > > Stopping lvmetad and/or restarting it would generally fix it.But > it was a source of enough random issues(often failure to mount on a > boot, so often issues that resulted in page-outs to debug) and did > not speed things up much enough to be worth it even on devices with > >2000 SAN volumes. > > On Thu, Sep 23, 2021 at 8:52 AM Zdenek Kabelac > wrote: > > > > Dne 22. 09. 21 v 18:48 alessandro macuz napsal(a): > > > fdisk correctly identifies the extended partition as 8e. > > > I wonder which kind of data lvmdiskscan and pvs use in order to list > LVM > > > physical volumes. > > > Does PVS check some specific metadata within the partition without just > > > relying on the type of partition displayed by fdisk? > > > > > > > > > > Hi > > > > Yes - PVs do have header signature keeping information about PV > attributes > > and also has the storage area to keep lvm2 metadata. > > > > Partition flags known to fdisk are irrelevant. > > > > > > Regards > > > > Zdenek > > > > ___ > > linux-lvm mailing list > > linux-lvm@redhat.com > > https://listman.redhat.com/mailman/listinfo/linux-lvm > > read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ > > > ___ linux-lvm mailing list linux-lvm@redhat.com https://listman.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] " Failed to find physical volume
Dne 23. 09. 21 v 20:03 alessandro macuz napsal(a): Thanks Zdenek, so it might be that metadata is corrupted somehow and hence the pvs program doesn't recognize that partition as physical volume? That may explain why lvmdiskscan reports physical disks (by just looking at the partition type 8e) and pvs completely ignores it. Am I correct? Hi Yes - if your disk header part has lost/damaged its content - it will not be recognized as PV - thus completely ignored. Note - the easiest is to check the verbose output of 'pvs -vvv' - where you could follow up what is command doing in relatively 'readable' form - if you can't follow it - just attach to the email for overlook why could be your disk eventually ignored. Note - the other reason could be the device got filtered by some filter - but I assume you've not changed your configuration on your system ? Zdenek Le jeu. 23 sept. 2021 à 15:52, Zdenek Kabelac a écrit : Dne 22. 09. 21 v 18:48 alessandro macuz napsal(a): > fdisk correctly identifies the extended partition as 8e. > I wonder which kind of data lvmdiskscan and pvs use in order to list LVM > physical volumes. > Does PVS check some specific metadata within the partition without just > relying on the type of partition displayed by fdisk? > > Hi Yes - PVs do have header signature keeping information about PV attributes and also has the storage area to keep lvm2 metadata. Partition flags known to fdisk are irrelevant. Regards Zdenek ___ linux-lvm mailing list linux-lvm@redhat.com https://listman.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] " Failed to find physical volume
Thanks Zdenek, so it might be that metadata is corrupted somehow and hence the pvs program doesn't recognize that partition as physical volume? That may explain why lvmdiskscan reports physical disks (by just looking at the partition type 8e) and pvs completely ignores it. Am I correct? Alex Le jeu. 23 sept. 2021 à 15:52, Zdenek Kabelac a écrit : > Dne 22. 09. 21 v 18:48 alessandro macuz napsal(a): > > fdisk correctly identifies the extended partition as 8e. > > I wonder which kind of data lvmdiskscan and pvs use in order to list LVM > > physical volumes. > > Does PVS check some specific metadata within the partition without just > > relying on the type of partition displayed by fdisk? > > > > > > Hi > > Yes - PVs do have header signature keeping information about PV attributes > and also has the storage area to keep lvm2 metadata. > > Partition flags known to fdisk are irrelevant. > > > Regards > > Zdenek > > ___ linux-lvm mailing list linux-lvm@redhat.com https://listman.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] " Failed to find physical volume
If you have lvmetad running and in use then the lvm commands ask it what the system has on it. I have seen on random boots fairly separated systems (rhel7 versions, and many years newer fedora systems) at random fail to find one or more pv.s I have disabled it at home, and in my day job we have also disabled (across 20k+ systems) as we confirmed it had inconsistency issues several times on a variety of our newest installs. Stopping lvmetad and/or restarting it would generally fix it.But it was a source of enough random issues(often failure to mount on a boot, so often issues that resulted in page-outs to debug) and did not speed things up much enough to be worth it even on devices with >2000 SAN volumes. On Thu, Sep 23, 2021 at 8:52 AM Zdenek Kabelac wrote: > > Dne 22. 09. 21 v 18:48 alessandro macuz napsal(a): > > fdisk correctly identifies the extended partition as 8e. > > I wonder which kind of data lvmdiskscan and pvs use in order to list LVM > > physical volumes. > > Does PVS check some specific metadata within the partition without just > > relying on the type of partition displayed by fdisk? > > > > > > Hi > > Yes - PVs do have header signature keeping information about PV attributes > and also has the storage area to keep lvm2 metadata. > > Partition flags known to fdisk are irrelevant. > > > Regards > > Zdenek > > ___ > linux-lvm mailing list > linux-lvm@redhat.com > https://listman.redhat.com/mailman/listinfo/linux-lvm > read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ > ___ linux-lvm mailing list linux-lvm@redhat.com https://listman.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] " Failed to find physical volume
Dne 22. 09. 21 v 18:48 alessandro macuz napsal(a): fdisk correctly identifies the extended partition as 8e. I wonder which kind of data lvmdiskscan and pvs use in order to list LVM physical volumes. Does PVS check some specific metadata within the partition without just relying on the type of partition displayed by fdisk? Hi Yes - PVs do have header signature keeping information about PV attributes and also has the storage area to keep lvm2 metadata. Partition flags known to fdisk are irrelevant. Regards Zdenek ___ linux-lvm mailing list linux-lvm@redhat.com https://listman.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] " Failed to find physical volume
fdisk correctly identifies the extended partition as 8e. I wonder which kind of data lvmdiskscan and pvs use in order to list LVM physical volumes. Does PVS check some specific metadata within the partition without just relying on the type of partition displayed by fdisk? Alex Le mer. 22 sept. 2021 à 01:36, alessandro macuz a écrit : > Hi all, > > I sought the web for the error " Failed to find physical volume > "/dev/zd96p5"." but the closest discussion in the LVM archives I found > was one of mine back in 2014 > > https://listman.redhat.com/archives/linux-lvm/2014-June/msg00016.html > > This time though it seems pvs fails to find the physical volume when > lvmdiskscan does recognize it. Do I have to really upgrade lvm2 or is there > another way to have pvs recognize the volume? > > root@server:/etc# lvm version > LVM version: 2.02.133(2) (2015-10-30) > Library version: 1.02.110 (2015-10-30) > Driver version: 4.34.0 > root@ server :/etc# lvmdiskscan -l > WARNING: only considering LVM devices > /dev/sda5 [ 74.05 GiB] LVM physical volume > /dev/zd304p3 [ 19.00 GiB] LVM physical volume > /dev/zd96p5 [1023.52 GiB] LVM physical volume > * /dev/zd112p5 [1023.52 GiB] LVM physical volume* > 0 LVM physical volume whole disks > 4 LVM physical volumes > root@ server :/etc# pvs -a /dev/zd96p5 > Failed to find physical volume "/dev/zd96p5". > > Thanks, Alex. > ___ linux-lvm mailing list linux-lvm@redhat.com https://listman.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
[linux-lvm] " Failed to find physical volume
Hi all, I sought the web for the error " Failed to find physical volume "/dev/zd96p5"." but the closest discussion in the LVM archives I found was one of mine back in 2014 https://listman.redhat.com/archives/linux-lvm/2014-June/msg00016.html This time though it seems pvs fails to find the physical volume when lvmdiskscan does recognize it. Do I have to really upgrade lvm2 or is there another way to have pvs recognize the volume? root@server:/etc# lvm version LVM version: 2.02.133(2) (2015-10-30) Library version: 1.02.110 (2015-10-30) Driver version: 4.34.0 root@ server :/etc# lvmdiskscan -l WARNING: only considering LVM devices /dev/sda5 [ 74.05 GiB] LVM physical volume /dev/zd304p3 [ 19.00 GiB] LVM physical volume /dev/zd96p5 [1023.52 GiB] LVM physical volume * /dev/zd112p5 [1023.52 GiB] LVM physical volume* 0 LVM physical volume whole disks 4 LVM physical volumes root@ server :/etc# pvs -a /dev/zd96p5 Failed to find physical volume "/dev/zd96p5". Thanks, Alex. ___ linux-lvm mailing list linux-lvm@redhat.com https://listman.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/