[ovirt-users] Re: Ovirt 4.2.7 won't start and drops to emergency console
I did some checking and my disk is not in a state I expected. (The system doesn't even know the VG exists in it's present state) See the results: # pv PV VG Fmt Attr PSize PFree /dev/md127 onn_vmh lvm2 a-- 222.44g 43.66g /dev/sdd1 gluster_vg3 lvm2 a-- <4.00g <2.00g # pvs -a PVVG Fmt Attr PSize PFree /dev/md127onn_vmh lvm2 a-- 222.44g 43.66g /dev/onn_vmh/home --- 0 0 /dev/onn_vmh/ovirt-node-ng-4.2.7.1-0.20181216.0+1 --- 0 0 /dev/onn_vmh/root --- 0 0 /dev/onn_vmh/swap --- 0 0 /dev/onn_vmh/tmp --- 0 0 /dev/onn_vmh/var --- 0 0 /dev/onn_vmh/var_crash --- 0 0 /dev/onn_vmh/var_log --- 0 0 /dev/onn_vmh/var_log_audit --- 0 0 /dev/sda1 --- 0 0 /dev/sdb1 --- 0 0 /dev/sdd1 gluster_vg3 lvm2 a-- <4.00g <2.00g /dev/sde1 --- 0 0 # vgs VG #PV #LV #SN Attr VSize VFree gluster_vg3 1 1 0 wz--n- <4.00g <2.00g onn_vmh 1 11 0 wz--n- 222.44g 43.66g # vgs -a VG #PV #LV #SN Attr VSize VFree gluster_vg3 1 1 0 wz--n- <4.00g <2.00g onn_vmh 1 11 0 wz--n- 222.44g 43.66g # lvs LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert tmpLVgluster_vg3 -wi--- 2.00g home onn_vmh Vwi-aotz-- 1.00g pool00 4.79 ovirt-node-ng-4.2.7.1-0.20181216.0 onn_vmh Vwi---tz-k 146.60g pool00 root ovirt-node-ng-4.2.7.1-0.20181216.0+1 onn_vmh Vwi-aotz-- 146.60g pool00 ovirt-node-ng-4.2.7.1-0.20181216.0 4.81 pool00 onn_vmh twi-aotz-- 173.60g 7.21 2.30 root onn_vmh Vwi-a-tz-- 146.60g pool00 2.92 swap onn_vmh -wi-ao 4.00g tmp onn_vmh Vwi-aotz-- 1.00g pool00 53.66 var onn_vmh Vwi-aotz-- 15.00g pool00 15.75 var_crashonn_vmh Vwi-aotz-- 10.00g pool00 2.86 var_log onn_vmh Vwi-aotz-- 8.00g pool00 14.73 var_log_auditonn_vmh Vwi-aotz-- 2.00g pool00 6.91 # lvs -a LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert tmpLVgluster_vg3 -wi--- 2.00g home onn_vmh Vwi-aotz-- 1.00g pool00 4.79 [lvol0_pmspare] onn_vmh ewi--- 180.00m ovirt-node-ng-4.2.7.1-0.20181216.0 onn_vmh Vwi---tz-k 146.60g pool00 root ovirt-node-ng-4.2.7.1-0.20181216.0+1 onn_vmh Vwi-aotz-- 146.60g pool00 ovirt-node-ng-4.2.7.1-0.20181216.0 4.81 pool00 onn_vmh twi-aotz-- 173.60g
[ovirt-users] Re: Ovirt 4.2.7 won't start and drops to emergency console
> >>>"Try to get all data in advance (before deactivating the VG)". > > Can you clarify? What do you mean by this? Get all necessary info before disabling the VG. For example: pvs vgs lvs -a lvdisplay -m /dev/gluster_vg1/lvthinpool lvdisplay -m /dev/gluster_vg1/lvthinpool_tmeta lvdisplay -m /dev/gluster_vg1/lvthinpool_tdata > >>>"You got 15GiB of metadata, so create your new metadata LV at least 30 > >>>GiB." > > My presumption is that 15 GB is the max size the was specified when it was > initially built but not the actual size right now, but yes, making it larger > does make sense. I was curious to know what the present size is. Also, > lvconvert man page specifies a supported value between 2M and 16G I will > create a larger metadata volume assuming I can get the procedure to work > properly I am having some difficulty with the procedure, see below. If you don't manually create the meta and data LVs (which will create the pool with another command) the size is created by the following formula: (Pool_LV_size / Pool_LV_chunk_size * 64). For details check: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/logical_volume_manager_administration/LV#thinly_provisioned_volume_creation > > My most important question was here: > I am trying to follow the procedure from > https://access.redhat.com/solutions/3251681 I am on step #2. Step 2a and 2b > work without issue. Step 2c gives me an error. > > Here are the values I am using: > > # lvcreate -L 2G gluster_vg3 --name tmpLV > (created with no issues) > > # lvchange -ay gluster_vg3/tmpLV > (command completed with no issues) > > # lvconvert --thinpool gluster_vg1/lvthinpool --poolmetadata > gluster_vg3/tmpLV > Error: > VG name mismatch from position arg (gluster_vg1) and option arg (gluster_vg3) > > Do I need to create the LV on the same disk that failed? (gluster_vg1) According to the error yes - logic shows that you need to be working in the same VG.If you don't have free space on the thin pool, you can add additional PV and extend the pool and then create your thin meta LV. You can also merge the 2 VGs to get the necessary space. Check 'man 8 vgmerge'. Best Regards, Strahil Nikolov ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/WUDPJNESADA4LPJGJNMLQE4H3XQ5S4WX/
[ovirt-users] Re: Ovirt 4.2.7 won't start and drops to emergency console
Sorry for the multiple posts. I had so many thoughts rolling around in my head. I'll try to consolidate my questions here and rephrase the last three responses. >>>"Try to get all data in advance (before deactivating the VG)". Can you clarify? What do you mean by this? >>>"I still can't imagine why the VG will disappear. Try with 'pvscan --cache' >>>to redetect the PV. Afrer all , all VG info is in the PVs' headers and >>>should be visible no matter the VG is deactivated or not." I'll try that and see what happens. >>>"Is this an oVirt Node or a regular CentOS/RHEL" oVirt Node >>>"You got 15GiB of metadata, so create your new metadata LV at least 30 GiB." My presumption is that 15 GB is the max size the was specified when it was initially built but not the actual size right now, but yes, making it larger does make sense. I was curious to know what the present size is. Also, lvconvert man page specifies a supported value between 2M and 16G I will create a larger metadata volume assuming I can get the procedure to work properly I am having some difficulty with the procedure, see below. My most important question was here: I am trying to follow the procedure from https://access.redhat.com/solutions/3251681 I am on step #2. Step 2a and 2b work without issue. Step 2c gives me an error. Here are the values I am using: # lvcreate -L 2G gluster_vg3 --name tmpLV (created with no issues) # lvchange -ay gluster_vg3/tmpLV (command completed with no issues) # lvconvert --thinpool gluster_vg1/lvthinpool --poolmetadata gluster_vg3/tmpLV Error: VG name mismatch from position arg (gluster_vg1) and option arg (gluster_vg3) Do I need to create the LV on the same disk that failed? (gluster_vg1) i.e.- perhaps my command is wrong and should be lvconvert --thinpool gluster_vg3/lvthinpool --poolmetadata gluster_vg3/tmpLV ** Perhaps this command assumes you already have a VG=gluster_vg3 and LV=lvthinpool? ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/37ILSMSJAYHPZNGBKF4RVIN7XHDQGMSM/
[ovirt-users] Re: Ovirt 4.2.7 won't start and drops to emergency console
Trust Red Hat :) At least their approach should be safer. Of course, you can raise a docu bug but RHEL7 is in such phase that it might not be fixed unless this is found in v8. Best Regards, Strahil NikolovOn Oct 2, 2019 05:43, jeremy_tourvi...@hotmail.com wrote: > > http://man7.org/linux/man-pages/man7/lvmthin.7.html > Command to repair a thin pool: > lvconvert --repair VG/ThinPoolLV > > Repair performs the following steps: > > 1. Creates a new, repaired copy of the metadata. > lvconvert runs the thin_repair command to read damaged metadata from > the existing pool metadata LV, and writes a new repaired copy to the > VG's pmspare LV. > > 2. Replaces the thin pool metadata LV. > If step 1 is successful, the thin pool metadata LV is replaced with > the pmspare LV containing the corrected metadata. The previous thin > pool metadata LV, containing the damaged metadata, becomes visible > with the new name ThinPoolLV_tmetaN (where N is 0,1,...). > > If the repair works, the thin pool LV and its thin LVs can be > activated, and the LV containing the damaged thin pool metadata can > be removed. It may be useful to move the new metadata LV (previously > pmspare) to a better PV. > > If the repair does not work, the thin pool LV and its thin LVs are > lost. > > This info seems to conflict with Red Hat advice. Red Hat says if metadat > volume is full don't run a lvconvert --repair operation. Now I am confused. > I am familiar with LVM and comfortable with it but this is my first time > trying to repair thin LVM. The concepts of a metadata volume are new to me. > ___ > Users mailing list -- users@ovirt.org > To unsubscribe send an email to users-le...@ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/FFN3HWUQVRPFSMXHECNJLJOFNF5ODAT6/ ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/NFM7OW2IPDQYOJ7F7FET3E2OWEEHPWNS/
[ovirt-users] Re: Ovirt 4.2.7 won't start and drops to emergency console
Hm...
It's strage it doesn't detect the VG , but could be related to the issue.
Accordi g to this:
lvthinpool_tmeta {
id = "WBut10-rAOP-FzA7-bJvr-ZdxL-lB70-jzz1Tv"
status = ["READ", "WRITE"]
flags = []
creation_time = 1545495487 # 2018-12-22 10:18:07 -0600
creation_host = "vmh.cyber-range.lan"
segment_count = 1
segment1 {
start_extent = 0
extent_count = 16192 # 15.8125 Gigabytes
You got 15GiB of metadata, so create your new metadata LV at least 30 GiB.
Best Regards,
Strahil NikolovOn Oct 2, 2019 04:49, jeremy_tourvi...@hotmail.com wrote:
>
> I don't know why I didn't think to get some more info regarding my storage
> environment and post it here earlier. My gluster_vg1 volume is on /dev/sda1.
> I can access the engine storage directory but I think that is because it is
> not thin provisioned. I guess I was too bogged down in solving the problem
> when I'm stuck in emergency mode. I had to sneaker net my USB drive to my
> system so I could capture some info. Anyhow here it is:
>
> # lsblk
> NAME MAJ:MIN RM SIZE
> RO TYPE MOUNTPOINT
> sda 8:0 0 5.5T
> 0 disk
> └─sda1 8:1 0 5.5T
> 0 part
> sdb 8:16 0 223.6G
> 0 disk
> ├─sdb1 8:17 0 1G
> 0 part /boot
> └─sdb2 8:18 0 222.6G
> 0 part
> └─md127 9:127 0 222.5G
> 0 raid1
> ├─onn_vmh-swap 253:0 0 4G
> 0 lvm [SWAP]
> ├─onn_vmh-pool00_tmeta 253:1 0 1G
> 0 lvm
> │ └─onn_vmh-pool00-tpool 253:3 0 173.6G
> 0 lvm
> │ ├─onn_vmh-ovirt--node--ng--4.2.7.1--0.20181216.0+1 253:4 0 146.6G
> 0 lvm /
> │ ├─onn_vmh-pool00 253:5 0 173.6G
> 0 lvm
> │ ├─onn_vmh-root 253:6 0 146.6G
> 0 lvm
> │ ├─onn_vmh-home 253:7 0 1G
> 0 lvm /home
> │ ├─onn_vmh-tmp 253:8 0 1G
> 0 lvm /tmp
> │ ├─onn_vmh-var 253:9 0 15G
> 0 lvm /var
> │ ├─onn_vmh-var_log 253:10 0 8G
> 0 lvm /var/log
> │ ├─onn_vmh-var_log_audit 253:11 0 2G
> 0 lvm /var/log/audit
> │ └─onn_vmh-var_crash 253:12 0 10G
> 0 lvm /var/crash
> └─onn_vmh-pool00_tdata 253:2 0 173.6G
> 0 lvm
> └─onn_vmh-pool00-tpool 253:3 0 173.6G
> 0 lvm
> ├─onn_vmh-ovirt--node--ng--4.2.7.1--0.20181216.0+1 253:4 0 146.6G
> 0 lvm /
> ├─onn_vmh-pool00 253:5 0 173.6G
> 0 lvm
> ├─onn_vmh-root 253:6 0 146.6G
> 0 lvm
> ├─onn_vmh-home 253:7 0 1G
> 0 lvm /home
> ├─onn_vmh-tmp 253:8 0 1G
> 0 lvm /tmp
> ├─onn_vmh-var 253:9 0 15G
> 0 lvm /var
> ├─onn_vmh-var_log 253:10 0 8G
> 0 lvm /var/log
> ├─onn_vmh-var_log_audit 253:11 0 2G
> 0 lvm /var/log/audit
> └─onn_vmh-var_crash 253:12 0 10G
> 0 lvm /var/crash
> sdc 8:32 0 223.6G
> 0 disk
> └─sdc1 8:33 0 222.6G
> 0 part
> └─md127 9:127 0 222.5G
> 0 raid1
> ├─onn_vmh-swap 253:0 0 4G
> 0 lvm [SWAP]
> ├─onn_vmh-pool00_tmeta 253:1 0 1G
> 0 lvm
> │ └─onn_vmh-pool00-tpool 253:3 0 173.6G
> 0 lvm
> │ ├─onn_vmh-ovirt--node--ng--4.2.7.1--0.20181216.0+1 253:4 0 146.6G
> 0 lvm /
> │ ├─onn_vmh-pool00 253:5 0 173.6G
> 0 lvm
> │ ├─onn_vmh-root 253:6 0 146.6G
> 0 lvm
> │ ├─onn_vmh-home 253:7 0 1G
> 0 lvm /home
> │ ├─onn_vmh-tmp 253:8 0 1G
> 0 lvm /tmp
> │ ├─onn_vmh-var
[ovirt-users] Re: Ovirt 4.2.7 won't start and drops to emergency console
Is this an oVirt Node or a regular CentOS/RHEL ?On Oct 2, 2019 05:06, jeremy_tourvi...@hotmail.com wrote: > > Here is my fstab file: > # /etc/fstab > # Created by anaconda on Fri Dec 21 22:26:32 2018 > # > # Accessible filesystems, by reference, are maintained under '/dev/disk' > # See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info > # > /dev/onn_vmh/ovirt-node-ng-4.2.7.1-0.20181216.0+1 / ext4 defaults,discard 1 1 > UUID=9b9546f9-25d2-42a6-835b-303f32aee4b1 /boot ext4 > defaults 1 2 > /dev/mapper/onn_vmh-home /home ext4 defaults,discard 1 2 > /dev/mapper/onn_vmh-tmp /tmp ext4 defaults,discard 1 2 > /dev/mapper/onn_vmh-var /var ext4 defaults,discard 1 2 > /dev/mapper/onn_vmh-var_log /var/log ext4 defaults,discard 1 2 > /dev/mapper/onn_vmh-var_log_audit /var/log/audit ext4 defaults,discard 1 2 > /dev/mapper/onn_vmh-swap swap swap defaults 0 0 > /dev/gluster_vg1/engine_lv /gluster_bricks/engine xfs > inode64,noatime,nodiratime 0 0 > /dev/gluster_vg1/lv_vmdisks /gluster_bricks/vmstore xfs > inode64,noatime,nodiratime 0 0 > /dev/gluster_vg1/lv_datadisks /gluster_bricks/data xfs > inode64,noatime,nodiratime 0 0 > #/dev/sdd1 /gluster_bricks/iso xfs inode64,noatime,nodiratime 0 0 > ___ > Users mailing list -- users@ovirt.org > To unsubscribe send an email to users-le...@ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/VYQ22EFSGRYDESRQZAAQ6E327J67JANJ/ ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/K46FZ324D4ZSYYX63XZSPO6RU7AOANKR/
[ovirt-users] Re: Ovirt 4.2.7 won't start and drops to emergency console
Try to get all data in advance (before deactivating the VG). I still can't imagine why the VG will disappear. Try with 'pvscan --cache' to redetect the PV. Afrer all , all VG info is in the PVs' headers and should be visible no matter the VG is deactivated or not. Best Regards, Strahil NikolovOn Oct 2, 2019 02:08, jeremy_tourvi...@hotmail.com wrote: > > "lvs -a" does not list the logical volume I am missing. > "lvdisplay -m /dev/gluster_vg1-lvthinpool-tpool_tmeta" does not work either. > Error message is: Volume Group xxx not found. Cannot process volume group > xxx." > > I am trying to follow the procedure from > https://access.redhat.com/solutions/3251681 > I am on step #2 Step 2a and 2b work without issue. Step 2c give me an > error. Here are the values I am using: > # lvcreate -L 2G gluster_vg3 --name tmpLV > # lvchange -ay gluster_vg3/tmpLV > # lvconvert --thinpool gluster_vg1/lvthinpool --poolmetadata > gluster_vg3/tmpLV > > VG name mismatch from position arg (gluster_vg1) and option arg (gluster_vg3) > > Do I need to create the LV on the same disk that failed? (gluster_vg1) > Is creating a new LV on (gluster_vg3) ok for this situation? > ___ > Users mailing list -- users@ovirt.org > To unsubscribe send an email to users-le...@ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/OJN6VVEMQY743WFXHVZOZB3KAEIN6PWM/ ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/PX52EN5FTWBHRROQKTNMMQJFMD6S4CFW/
[ovirt-users] Re: Ovirt 4.2.7 won't start and drops to emergency console
http://man7.org/linux/man-pages/man7/lvmthin.7.html Command to repair a thin pool: lvconvert --repair VG/ThinPoolLV Repair performs the following steps: 1. Creates a new, repaired copy of the metadata. lvconvert runs the thin_repair command to read damaged metadata from the existing pool metadata LV, and writes a new repaired copy to the VG's pmspare LV. 2. Replaces the thin pool metadata LV. If step 1 is successful, the thin pool metadata LV is replaced with the pmspare LV containing the corrected metadata. The previous thin pool metadata LV, containing the damaged metadata, becomes visible with the new name ThinPoolLV_tmetaN (where N is 0,1,...). If the repair works, the thin pool LV and its thin LVs can be activated, and the LV containing the damaged thin pool metadata can be removed. It may be useful to move the new metadata LV (previously pmspare) to a better PV. If the repair does not work, the thin pool LV and its thin LVs are lost. This info seems to conflict with Red Hat advice. Red Hat says if metadat volume is full don't run a lvconvert --repair operation. Now I am confused. I am familiar with LVM and comfortable with it but this is my first time trying to repair thin LVM. The concepts of a metadata volume are new to me. ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/FFN3HWUQVRPFSMXHECNJLJOFNF5ODAT6/
[ovirt-users] Re: Ovirt 4.2.7 won't start and drops to emergency console
Here is my fstab file: # /etc/fstab # Created by anaconda on Fri Dec 21 22:26:32 2018 # # Accessible filesystems, by reference, are maintained under '/dev/disk' # See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info # /dev/onn_vmh/ovirt-node-ng-4.2.7.1-0.20181216.0+1 / ext4 defaults,discard 1 1 UUID=9b9546f9-25d2-42a6-835b-303f32aee4b1 /boot ext4 defaults1 2 /dev/mapper/onn_vmh-home /home ext4 defaults,discard 1 2 /dev/mapper/onn_vmh-tmp /tmp ext4 defaults,discard 1 2 /dev/mapper/onn_vmh-var /var ext4 defaults,discard 1 2 /dev/mapper/onn_vmh-var_log /var/log ext4 defaults,discard 1 2 /dev/mapper/onn_vmh-var_log_audit /var/log/audit ext4 defaults,discard 1 2 /dev/mapper/onn_vmh-swap swapswapdefaults0 0 /dev/gluster_vg1/engine_lv /gluster_bricks/engine xfs inode64,noatime,nodiratime 0 0 /dev/gluster_vg1/lv_vmdisks /gluster_bricks/vmstore xfs inode64,noatime,nodiratime 0 0 /dev/gluster_vg1/lv_datadisks /gluster_bricks/data xfs inode64,noatime,nodiratime 0 0 #/dev/sdd1 /gluster_bricks/iso xfs inode64,noatime,nodiratime 0 0 ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/VYQ22EFSGRYDESRQZAAQ6E327J67JANJ/
[ovirt-users] Re: Ovirt 4.2.7 won't start and drops to emergency console
I don't know why I didn't think to get some more info regarding my storage environment and post it here earlier. My gluster_vg1 volume is on /dev/sda1. I can access the engine storage directory but I think that is because it is not thin provisioned. I guess I was too bogged down in solving the problem when I'm stuck in emergency mode. I had to sneaker net my USB drive to my system so I could capture some info. Anyhow here it is: # lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:00 5.5T 0 disk └─sda1 8:10 5.5T 0 part sdb 8:16 0 223.6G 0 disk ├─sdb1 8:17 0 1G 0 part /boot └─sdb2 8:18 0 222.6G 0 part └─md1279:127 0 222.5G 0 raid1 ├─onn_vmh-swap 253:00 4G 0 lvm [SWAP] ├─onn_vmh-pool00_tmeta 253:10 1G 0 lvm │ └─onn_vmh-pool00-tpool 253:30 173.6G 0 lvm │ ├─onn_vmh-ovirt--node--ng--4.2.7.1--0.20181216.0+1 253:40 146.6G 0 lvm / │ ├─onn_vmh-pool00 253:50 173.6G 0 lvm │ ├─onn_vmh-root 253:60 146.6G 0 lvm │ ├─onn_vmh-home 253:70 1G 0 lvm /home │ ├─onn_vmh-tmp 253:80 1G 0 lvm /tmp │ ├─onn_vmh-var 253:9015G 0 lvm /var │ ├─onn_vmh-var_log 253:10 0 8G 0 lvm /var/log │ ├─onn_vmh-var_log_audit253:11 0 2G 0 lvm /var/log/audit │ └─onn_vmh-var_crash253:12 010G 0 lvm /var/crash └─onn_vmh-pool00_tdata 253:20 173.6G 0 lvm └─onn_vmh-pool00-tpool 253:30 173.6G 0 lvm ├─onn_vmh-ovirt--node--ng--4.2.7.1--0.20181216.0+1 253:40 146.6G 0 lvm / ├─onn_vmh-pool00 253:50 173.6G 0 lvm ├─onn_vmh-root 253:60 146.6G 0 lvm ├─onn_vmh-home 253:70 1G 0 lvm /home ├─onn_vmh-tmp 253:80 1G 0 lvm /tmp ├─onn_vmh-var 253:9015G 0 lvm /var ├─onn_vmh-var_log 253:10 0 8G 0 lvm /var/log ├─onn_vmh-var_log_audit253:11 0 2G 0 lvm /var/log/audit └─onn_vmh-var_crash253:12 010G 0 lvm /var/crash sdc 8:32 0 223.6G 0 disk └─sdc1 8:33 0 222.6G 0 part └─md1279:127 0 222.5G 0 raid1 ├─onn_vmh-swap 253:00 4G 0 lvm [SWAP] ├─onn_vmh-pool00_tmeta 253:10 1G 0 lvm │ └─onn_vmh-pool00-tpool 253:30 173.6G 0 lvm │ ├─onn_vmh-ovirt--node--ng--4.2.7.1--0.20181216.0+1 253:40 146.6G 0 lvm / │ ├─onn_vmh-pool00 253:50 173.6G 0 lvm │ ├─onn_vmh-root 253:60 146.6G 0 lvm │ ├─onn_vmh-home 253:70 1G 0 lvm /home │ ├─onn_vmh-tmp 253:80 1G 0 lvm /tmp │ ├─onn_vmh-var 253:9015G 0 lvm /var │ ├─onn_vmh-var_log 253:10 0 8G 0 lvm /var/log │ ├─onn_vmh-var_log_audit253:11 0 2G 0 lvm /var/log/audit │ └─onn_vmh-var_crash253:12 010G 0 lvm /var/crash └─onn_vmh-pool00_tdata 253:20 173.6G 0 lvm └─onn_vmh-pool00-tpool 253:30 173.6G 0 lvm ├─onn_vmh-ovirt--node--ng--4.2.7.1--0.20181216.0+1 253:40 146.6G 0 lvm / ├─onn_vmh-pool00 253:50 173.6G 0 lvm ├─onn_vmh-root 253:6
[ovirt-users] Re: Ovirt 4.2.7 won't start and drops to emergency console
"lvs -a" does not list the logical volume I am missing. "lvdisplay -m /dev/gluster_vg1-lvthinpool-tpool_tmeta" does not work either. Error message is: Volume Group xxx not found. Cannot process volume group xxx." I am trying to follow the procedure from https://access.redhat.com/solutions/3251681 I am on step #2 Step 2a and 2b work without issue. Step 2c give me an error. Here are the values I am using: # lvcreate -L 2G gluster_vg3 --name tmpLV # lvchange -ay gluster_vg3/tmpLV # lvconvert --thinpool gluster_vg1/lvthinpool --poolmetadata gluster_vg3/tmpLV VG name mismatch from position arg (gluster_vg1) and option arg (gluster_vg3) Do I need to create the LV on the same disk that failed? (gluster_vg1) Is creating a new LV on (gluster_vg3) ok for this situation? ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/OJN6VVEMQY743WFXHVZOZB3KAEIN6PWM/
[ovirt-users] Re: Ovirt 4.2.7 won't start and drops to emergency console
You can view all LVs via 'lvs -a' and create a new metadata LV of bigger size. Of course , lvdisplay -m /dev/gluster_vg1-lvthinpool-tpool_tmeta shoould also work. Best Regards, Strahil NikolovOn Oct 1, 2019 03:11, jeremy_tourvi...@hotmail.com wrote: > > vgs displays everything EXCEPT gluster_vg1 > "dmsetup ls" does not list the VG in question. That is why I couldn't run > the lvchange command. They were not active or even detected by the system. > > OK, I found my problem, and a solution: > https://access.redhat.com/solutions/3251681 > # cd /var/log > # grep -ri gluster_vg1-lvthinpool-tpool > My metadata is 100% full !!! > > Now, how do I find out how big my original metadata size was so I can make > the new one the correct size? > ___ > Users mailing list -- users@ovirt.org > To unsubscribe send an email to users-le...@ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/6PI2N24GVWL64UHVZ3BWF3KTJBRL2ALU/ ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/U5EQGBF6ZH6V6WXNZQSOJ326ZSGDNFF4/
[ovirt-users] Re: Ovirt 4.2.7 won't start and drops to emergency console
vgs displays everything EXCEPT gluster_vg1 "dmsetup ls" does not list the VG in question. That is why I couldn't run the lvchange command. They were not active or even detected by the system. OK, I found my problem, and a solution: https://access.redhat.com/solutions/3251681 # cd /var/log # grep -ri gluster_vg1-lvthinpool-tpool My metadata is 100% full !!! Now, how do I find out how big my original metadata size was so I can make the new one the correct size? ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/6PI2N24GVWL64UHVZ3BWF3KTJBRL2ALU/
[ovirt-users] Re: Ovirt 4.2.7 won't start and drops to emergency console
What happens when it complain that there is no VGs ? When you run 'vgs' what is the output? Also, take a look into https://www.redhat.com/archives/linux-lvm/2016-February/msg00012.html I have the feeling that you need to disable all lvs - not only the thin pool, but also the thin LVs (first). Best Regards, Strahil NikolovOn Sep 29, 2019 23:00, jeremy_tourvi...@hotmail.com wrote: > > Yes, I can take the downtime. Actually, I don't have any choice at the > moment because it is a single node setup. :) I think this is a distributed > volume from the research I have performed. I posted the lvchange command in > my last post, this was the result- I ran the command lvchange -an > /dev/gluster_vg1/lv_datadisks When I do this I get the message "Volume group > "gluster_vg1" not found. Cannot process volume group gluster_vg1". I also > tried the command the way you specified with just the LV and get the same > results. > > I had placed the system in Global maintenance mode prior to the reboot. Upon > reboot I got the messages about the various gluster volumes not being able to > be mounted because of timeout issues. That is what started my OP. I think > we are both thinking along the same lines regarding the issue. I think the > question is how do you fix a volume that the system won't mount? It does > seem likely that the thinpool needs to be repaired but what do you do if you > can't even perform the first step in the procedure? > > ___ > Users mailing list -- users@ovirt.org > To unsubscribe send an email to users-le...@ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/2X73CAASB4UCOK6Y25EBMGQBZBKKG7JV/ ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/4C7BUN2XVBAL6CRTBYIZCXDBO2LVULYZ/
[ovirt-users] Re: Ovirt 4.2.7 won't start and drops to emergency console
Yes, I can take the downtime. Actually, I don't have any choice at the moment because it is a single node setup. :) I think this is a distributed volume from the research I have performed. I posted the lvchange command in my last post, this was the result- I ran the command lvchange -an /dev/gluster_vg1/lv_datadisks When I do this I get the message "Volume group "gluster_vg1" not found. Cannot process volume group gluster_vg1". I also tried the command the way you specified with just the LV and get the same results. I had placed the system in Global maintenance mode prior to the reboot. Upon reboot I got the messages about the various gluster volumes not being able to be mounted because of timeout issues. That is what started my OP. I think we are both thinking along the same lines regarding the issue. I think the question is how do you fix a volume that the system won't mount? It does seem likely that the thinpool needs to be repaired but what do you do if you can't even perform the first step in the procedure? ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/2X73CAASB4UCOK6Y25EBMGQBZBKKG7JV/
[ovirt-users] Re: Ovirt 4.2.7 won't start and drops to emergency console
Can you suffer downtime ? You can try something like this (I'm improvising): Set to global maintenance (either via UI or hosted-engine --set-maintenance --mode=global) Stop the engine. Stop ovirt-ha-agent ovirt-ha-broker vdsmd supervdsmd sanlock glusterd. Stop all gluster processes via thr script in /usr/share/gluster*/something-named-kill Disable all LVs in the vg via: lvchange -an LV If it doesn't work try disabling the VG: vgchange -an VG and then enabling it again: vgchange -ay VG Last try to repair the thinpool via: lvchange --repair VG/thinpool Last disable the VG and give the system a reboot to verify everything works. Best Regards, Strahil NikolovOn Sep 29, 2019 14:39, jeremy_tourvi...@hotmail.com wrote: > > Thank you for the reply. Please pardon my ignorance, I'm not very good with > GlusterFS. I don't think this is a replicated volume (though I could be > wrong) I built a single node hyperconverged hypervisor. I was reviewing my > gdeploy file from when I originally built the system. I have the following > values: > PV = /dev/sda > VG1= gluster_vg1 > LV1= engine_lv (thick) > LV2 = gluster_vg1 thinpool > LV3 = lv_vmdisks (thinpooll) > LV4 = lv_datadisks (thinpool) > > So according to the article I read in my OP, it says to deactivate the > volumes under the thin pool as the first step. I ran the command lvchange > -an /dev/gluster_vg1/lv_datadisks When I do this I am told Volume group > "gluster_vg1" not found. Cannot process volume group gluster_vg1. > > That seems consistent with the timeout error message. How do you fix this if > you can't access the volumes? Thoughts? > > > ___ > Users mailing list -- users@ovirt.org > To unsubscribe send an email to users-le...@ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/HZXE3NCSFWCUIEN3GKRWKTCH3QZDA4PF/ ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/BURS4TDOUPHKOF54LSVU3UYUZIBT7T2X/
[ovirt-users] Re: Ovirt 4.2.7 won't start and drops to emergency console
Thank you for the reply. Please pardon my ignorance, I'm not very good with GlusterFS. I don't think this is a replicated volume (though I could be wrong) I built a single node hyperconverged hypervisor. I was reviewing my gdeploy file from when I originally built the system. I have the following values: PV = /dev/sda VG1= gluster_vg1 LV1= engine_lv (thick) LV2 = gluster_vg1 thinpool LV3 = lv_vmdisks (thinpooll) LV4 = lv_datadisks (thinpool) So according to the article I read in my OP, it says to deactivate the volumes under the thin pool as the first step. I ran the command lvchange -an /dev/gluster_vg1/lv_datadisks When I do this I am told Volume group "gluster_vg1" not found. Cannot process volume group gluster_vg1. That seems consistent with the timeout error message. How do you fix this if you can't access the volumes? Thoughts? ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/HZXE3NCSFWCUIEN3GKRWKTCH3QZDA4PF/
[ovirt-users] Re: Ovirt 4.2.7 won't start and drops to emergency console
If it's a replicated volume - then you can safely rebuild your bricks and don't even tryhto repair. There is no guarantee that the issue will not reoccur. Best Regards, Strahil NikolovOn Sep 29, 2019 00:22, jeremy_tourvi...@hotmail.com wrote: > > I see evidence that appears to be a problem with gluster. /var/log/messages > has multiple occurrences of: WARNING: /dev/gluster_vg1/lv_: Thin's > tihin-pool needs inspection. Also, if I run vgs I am returned info for my > other volume groups but not gluster_vg1. Lastly, when reviewing journalctl > -xb | grep -i timed I see messages from systemd > Job dev-gluster_vg1-lv_vmdisks.device/start timed.out. > Timed out waiting for device dev-glustet_vg1-lv_vmdisks.device > > Thses messages are happening for both > /dev/gluster_vg1/lv_datadisks > /dev/gluster_vg1/lv_vmdisks > > I did review the article here but I am unable to change to VG1. > https://mellowhost.com/billing/index.php?rp=/knowledgebase/65/How-to-Repair-a-lvm-thin-pool.html > > > Can anyone assist with a procedure on how to repair? > ___ > Users mailing list -- users@ovirt.org > To unsubscribe send an email to users-le...@ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/VCHU2Q4RD5QWNMHFCSXFMWFIHANJGUE2/ ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/KMMBVAHMV7AIPBEGSPWKZ435QVK3YNEA/
